date:20130426

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642878#comment-13642878
]

T Jake Luciani commented on CASSANDRA-5417:
---

I agree the dremel approach would make the on disk format more efficient for
composites. I also think there is some lower hanging fruit to be had if we
simply cache the current composite compare results so it isn't so cpu intense.
Especially for the compaction case.

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

CompositeType happens to be very useful and is now widely used: CQL3 heavily
rely on it, and super columns are now using it too internally. Besides,
CompositeType has been advised as a replacement of super columns on the
thrift side for a while, so it's safe to assume that it's generally used
there too.
CompositeType has initially been introduced as just another AbstractType.
Meaning that the storage engine has no nothing whatsoever of composites
being, well, composite. This has the following drawbacks:
* Because internally a composite value is handled as just a ByteBuffer, we
end up doing a lot of extra work. Typically, each time we compare 2 composite
value, we end up deserializing the components (which, while it doesn't copy
data per-se because we just slice the global ByteBuffer, still waste some cpu
cycles and allocate a bunch of ByteBuffer objects). And since compare can be
called *a lot*, this is likely not negligible.
* This make CQL3 code uglier than necessary. Basically, CQL3 makes extensive
use of composites, and since it gets backs ByteBuffer from the internal
columns, it always have to check if it's actually a compositeType or not, and
then split it and pick the different parts it needs. It's only an API
problem, but having things exposed as composites directly would definitively
make thinks cleaner. In particular, in most cases, CQL3 don't care whether it
has a composite with only one component or a non-really-composite value, but
we still always distinguishes both cases. Lastly, if we do expose composites
more directly internally, it's not a lot more work to internalize better
the different parts of the cell name that CQL3 uses (what's the clustering
key, what's the actuall CQL3 column name, what's the collection element),
making things cleaner. Last but not least, there is currently a bunch of
places where methods take a ByteBuffer as argument and it's hard to know
whether it expects a cell name or a CQL3 column name. This is pretty error
prone.
* It makes it hard (or impossible) to do a number of performance
improvements. Consider CASSANDRA-4175, I'm not really sure how you can do it
properly (in memory) if cell names are just ByteBuffer (since CQL3 column
names are just one of the component in general). But we also miss
oportunities of sharing prefixes. If we were able to share prefixes of
composite names in memory we would 1) lower the memory footprint and 2)
potentially speed-up comparison (of the prefixes) by checking reference
equality first (also, doing prefix sharing on-disk, which is a separate
concern btw, might be easier to do if we do prefix sharing in memory).
So I suggest pushing CompositeType support inside the storage engine. What I
mean by that concretely would be change the internal {{Column.name}} from
ByteBuffer to some CellName type. A CellName would API-wise just be a list of
ByteBuffer. But in practice, we'd have a specific CellName implementation for
not-really-composite names, and the truly composite implementation will allow
some prefix sharing. From an external API however, nothing would change, we
would pack the composite as usual before sending it back to the client, but
at least internally, comparison won't have to deserialize the components
every time, and CQL3 code will be cleaner.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5443) Add CAS CQL support


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne updated CASSANDRA-5443:


Attachment: 0004-Support-tombstones-when-comparing-for-CAS.txt
0003-Handle-deleted-and-expiring-column-in-paxos-updates.txt
0002-Add-syntax-to-support-conditional-update-delete.txt
0001-Refactor-Update-and-Delete-statement-to-extract-common.txt

Attaching patches for this. The first patch is a refactor whose goal is to make 
the 2nd patch, the one that add the new bits, easier. That refactor also cleans 
up those statements somewhat by better sharing code that is common. The 2nd 
patch adds the following syntax:
{noformat}
UPDATE test SET v1 = 5, v2 = 'foobar', v3 = 5 WHERE k = 0 IF v1 = 3 AND v2 = 
'foo';
DELETE v2 FROM test WHERE k = 0 IF v1 = 5;
{noformat}
A few remarks on that syntax:
* Despites my earlier comments, I've left off the {{ATOMICALLY}} and I admit 
it's seducing to just have the {{IF}} and nothing more. That being said, I 
still think that simplify has a downside, which is that it doesn't emphasis 
much that an update with or without IF have very different perfomance 
characteristics. But let's say I'm now leaning more towards a simpler syntax 
and making sure we're clear on the performance impact in documentations.
* I've used {{AND}} to separate conditions. The other option would be to use a 
comma to mimick the SET clause, but it felt a {{AND}} was sounding better after 
a {{IF}}.

The 2 last patches fix small bugs with the paxos code itself (and can be 
committed independently):
* patch 3 fixes make sure we don't correctly preserve deleted and expiring 
columns in paxos updates.
* patch 4 tweaks the CAS comparison to handle correctly deleted columns in 
expected. That allows to support condition like IF v1 = null.

Now there is a few things that don't work yet:
# updating a row if it doesn't exist yet is not supported. To be more precise, 
we do handle {{null}} in conditions, but that's only good for non primary key 
components. And I'm not sure allowing PK columns in conditions would be a good 
solution because I'm not sure
 {noformat}
 UPDATE test SET v1 = 5, v2 = 'foobar' WHERE k = 0 IF k = null
 {noformat}
 really carries the intent very well (the {{WHERE}} and {{IF}} _sound_
 contradictory). Also, the fact that
 {noformat}
 UPDATE test SET v1 = 5, v2 = 'foobar' WHERE k = 0 IF k = 1
 {noformat}
 suggests to me that it's not really the right syntax. Lastly, it would require 
a bit of ugly special casing in the code (not a blocker but ...).
# the code ignores completely the consistency level so far. That part is 
basically pending CASSANDRA-5442.
# the code does support sets and maps in conditions, but lists are *not* 
supported (an InvalidRequestException is thrown is a list is in the 
conditions). The reason is that for lists, the cell name contains a server side 
generated timeuuid, and so we would need to make sure the timeuuids in the 
_expected_ CF we build match the current ones. And well, that's doable but will 
require some special additional code so I've left it off for now.

Last but not least, so far the code returns a resultset with one boolean column 
named 'result' (so like in my comment above). We can easily change that to a 
count of 0 or 1 expressing the number of affected rows as discussed above, but 
as said I'm not fully convinced by that idea yet. One of the reason being that 
if we do ever allow conditional updates in batches (which would be doable, at 
least for UNLOGGED ones), the number of affected rows wouldn't cut it anymore. 
Now, to be fair, the current 1 column resultSet wouldn't work either, but 
that's why my suggestion would be to change that to return one column per 
partition key (since we do CAS at that level). So for instance
{noformat}
UPDATE test SET v1 = 5, v2 = 'foobar', v3 = 5 WHERE k = 0 IF v1 = 3 AND v2 = 
'foo';
{noformat}
would return the following result set:
{noformat}
upd(k = 0)
--
  true
{noformat}
so basically the same thing that with the attached patch except that the column 
result would indicate the value of the partition key updated, which would work 
if we do add batches (the one downside being that if people starts to use large 
blobs as partition keys it'll get messy, but not sure that's a big concern). 
Opinions?


 Add CAS CQL support
 ---

 Key: CASSANDRA-5443
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5443
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 2.0

 Attachments: 
 0001-Refactor-Update-and-Delete-statement-to-extract-common.txt, 
 0002-Add-syntax-to-support-conditional-update-delete.txt,

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642899#comment-13642899
]

Jonathan Ellis commented on CASSANDRA-5417:
---

We'll have single-pass compaction done RSN (CASSANDRA-4180). Or did you mean
something else for caching results?

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642928#comment-13642928
]

Sylvain Lebresne commented on CASSANDRA-5417:
-

bq. What if we took the approach from CASSANDRA-674 /
http://wiki.apache.org/cassandra/FileFormatDesignDoc, and had an explicit
parent atom instead of implicit via prefix sharing?

Just to be clear we talk of the same thing, when I speak of prefix sharing I'm,
as far as this ticket is concern, talking of in-memory sharing. That being
said, yes, I do think we should move at some point to a file format à la
dremel/CASSANDRA-674. But that's more of a follow up to this ticket imo, it's
not something that ticket is trying to tackle in itself.

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

git commit: Set isRunning flag later in binary protocol server

2013-04-26 Thread slebresne

Updated Branches:
  refs/heads/cassandra-1.2 9793d6f32 - dc5b1e9b8


Set isRunning flag later in binary protocol server

patch by slebresne; reviewed by krummas for CASSANDRA-5467


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dc5b1e9b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dc5b1e9b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dc5b1e9b

Branch: refs/heads/cassandra-1.2
Commit: dc5b1e9b86b42c786b4e1e3752a08d45e6e447e8
Parents: 9793d6f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Apr 26 17:17:53 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Apr 26 17:17:53 2013 +0200

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/transport/Server.java |8 
 2 files changed, 5 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc5b1e9b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d1b59bb..7634742 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@
  * Fix primary range ignores replication strategy (CASSANDRA-5424)
  * Fix shutdown of binary protocol server (CASSANDRA-5507)
  * Fix repair -snapshot not working (CASSANDRA-5512)
+ * Set isRunning flag later in binary protocol server (CASSANDRA-5467)
 Merged from 1.1
  * Add retry mechanism to OTC for non-droppable_verbs (CASSANDRA-5393)
  * Use allocator information to improve memtable memory usage estimate 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc5b1e9b/src/java/org/apache/cassandra/transport/Server.java
--
diff --git a/src/java/org/apache/cassandra/transport/Server.java 
b/src/java/org/apache/cassandra/transport/Server.java
index 74394d2..51a90e8 100644
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@ -95,8 +95,8 @@ public class Server implements CassandraDaemon.Server
 
 public void start()
 {
-if (isRunning.compareAndSet(false, true))
-run();
+run();
+isRunning.set(true);
 }
 
 public void stop()
@@ -110,7 +110,7 @@ public class Server implements CassandraDaemon.Server
 return isRunning.get();
 }
 
-public void run()
+private void run()
 {
 // Configure the server.
 executionHandler = new ExecutionHandler(new 
RequestThreadPoolExecutor());
@@ -137,7 +137,7 @@ public class Server implements CassandraDaemon.Server
 connectionTracker.allChannels.add(channel);
 }
 
-public void close()
+private void close()
 {
 // Close opened connections
 connectionTracker.closeAll();

[1/2] git commit: Set isRunning flag later in binary protocol server

2013-04-26 Thread slebresne

Updated Branches:
  refs/heads/trunk 4a104d904 - a45918b08


Set isRunning flag later in binary protocol server

patch by slebresne; reviewed by krummas for CASSANDRA-5467


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dc5b1e9b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dc5b1e9b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dc5b1e9b

Branch: refs/heads/trunk
Commit: dc5b1e9b86b42c786b4e1e3752a08d45e6e447e8
Parents: 9793d6f
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Apr 26 17:17:53 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Apr 26 17:17:53 2013 +0200

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/transport/Server.java |8 
 2 files changed, 5 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc5b1e9b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d1b59bb..7634742 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -11,6 +11,7 @@
  * Fix primary range ignores replication strategy (CASSANDRA-5424)
  * Fix shutdown of binary protocol server (CASSANDRA-5507)
  * Fix repair -snapshot not working (CASSANDRA-5512)
+ * Set isRunning flag later in binary protocol server (CASSANDRA-5467)
 Merged from 1.1
  * Add retry mechanism to OTC for non-droppable_verbs (CASSANDRA-5393)
  * Use allocator information to improve memtable memory usage estimate 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/dc5b1e9b/src/java/org/apache/cassandra/transport/Server.java
--
diff --git a/src/java/org/apache/cassandra/transport/Server.java 
b/src/java/org/apache/cassandra/transport/Server.java
index 74394d2..51a90e8 100644
--- a/src/java/org/apache/cassandra/transport/Server.java
+++ b/src/java/org/apache/cassandra/transport/Server.java
@@ -95,8 +95,8 @@ public class Server implements CassandraDaemon.Server
 
 public void start()
 {
-if (isRunning.compareAndSet(false, true))
-run();
+run();
+isRunning.set(true);
 }
 
 public void stop()
@@ -110,7 +110,7 @@ public class Server implements CassandraDaemon.Server
 return isRunning.get();
 }
 
-public void run()
+private void run()
 {
 // Configure the server.
 executionHandler = new ExecutionHandler(new 
RequestThreadPoolExecutor());
@@ -137,7 +137,7 @@ public class Server implements CassandraDaemon.Server
 connectionTracker.allChannels.add(channel);
 }
 
-public void close()
+private void close()
 {
 // Close opened connections
 connectionTracker.closeAll();

[2/2] git commit: Merge branch 'cassandra-1.2' into trunk

2013-04-26 Thread slebresne

Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a45918b0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a45918b0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a45918b0

Branch: refs/heads/trunk
Commit: a45918b081bebd7e3919411420074a0a628994cb
Parents: 4a104d9 dc5b1e9
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Apr 26 17:21:26 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Apr 26 17:21:26 2013 +0200

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/transport/Server.java |8 
 2 files changed, 5 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/a45918b0/CHANGES.txt
--

[jira] [Updated] (CASSANDRA-5443) Add CAS CQL support


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5443:
--

Reviewer: iamaleksey

 Add CAS CQL support
 ---

 Key: CASSANDRA-5443
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5443
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 2.0

 Attachments: 
 0001-Refactor-Update-and-Delete-statement-to-extract-common.txt, 
 0002-Add-syntax-to-support-conditional-update-delete.txt, 
 0003-Handle-deleted-and-expiring-column-in-paxos-updates.txt, 
 0004-Support-tombstones-when-comparing-for-CAS.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642957#comment-13642957
]

Sylvain Lebresne commented on CASSANDRA-5417:
-

bq. I've attached a screenshot of the current bottleneck if you want to take a
look.

I'll admit that puzzle me a bit. Is that saying that the
AbstractCellNameType.columnSerializer() method itself is being a hot spot?
Cause that method is simple getter of a final field! But anyway, I'll try to do
some read tests, see if I can reproduce that ~30% slowdown and who is
responsible.

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

[jira] [Commented] (CASSANDRA-5443) Add CAS CQL support


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642958#comment-13642958
 ] 

Jonathan Ellis commented on CASSANDRA-5443:
---

bq. updating a row if it doesn't exist yet is not supported ... it's not really 
the right syntax

We do need to support this use case one way or another.  How about IF NOT 
EXISTS?

 Add CAS CQL support
 ---

 Key: CASSANDRA-5443
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5443
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 2.0

 Attachments: 
 0001-Refactor-Update-and-Delete-statement-to-extract-common.txt, 
 0002-Add-syntax-to-support-conditional-update-delete.txt, 
 0003-Handle-deleted-and-expiring-column-in-paxos-updates.txt, 
 0004-Support-tombstones-when-comparing-for-CAS.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

buildbot failure in ASF Buildbot on cassandra-trunk

2013-04-26 Thread buildbot

The Buildbot has detected a new failure on builder cassandra-trunk while 
building cassandra.
Full details are available at:
 http://ci.apache.org/builders/cassandra-trunk/builds/2613

Buildbot URL: http://ci.apache.org/

Buildslave for this Build: portunus_ubuntu

Build Reason: scheduler
Build Source Stamp: [branch trunk] a45918b081bebd7e3919411420074a0a628994cb
Blamelist: Sylvain Lebresne sylv...@datastax.com

BUILD FAILED: failed shell

sincerely,
 -The Buildbot

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642973#comment-13642973
]

T Jake Luciani commented on CASSANDRA-5417:
---

I can send you my test offline if that helps.

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

[jira] [Commented] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642981#comment-13642981
 ] 

Jonathan Ellis commented on CASSANDRA-5511:
---

Pushed commits to address above, thanks!

 Clean up backwards compatibility complexity for 2.0
 ---

 Key: CASSANDRA-5511
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: CASSANDRA-5511-on_1.2.patch, 
 CASSANDRA-5511-on_trunk.patch


 We've supported rolling upgrades (network-compatible for read/write 
 operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required 
 being on a recent release of the immediately prior major series for this to 
 work as desired.
 Meanwhile, we still support reading sstables at least back to 0.6 and 
 possibly even earlier.  This makes dealing with changes to the sstable quite 
 challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
 2.0 is a good place to drop support for sstables older than 1.2.5.  Our 
 experience with network compatibility demonstrates that this is not an 
 unreasonable burden to impose, and the major version number change suggests 
 that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643012#comment-13643012
 ] 

Sylvain Lebresne commented on CASSANDRA-5501:
-

Marco, could you try to run the query the tracing enabled? I.e. by setting 
'TRACING ON' in cqlsh before running the query. Not sure he'll help, but it 
can't hurt I suppose.

It is definitively weird that the cli and cqlsh would differ here, as they 
should end up with the same call internally basically.

 Missing data on SELECT on secondary index 
 --

 Key: CASSANDRA-5501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5501
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.4
 Environment: linux ubuntu 12.04
Reporter: Marco Matarazzo
 Attachments: query_log.txt


 We have a 3 nodes cluster, and a keyspace with RF = 3.
 From cassandra-cli everything is fine (we actually never use it, I just 
 launched it for a check in this particular case).
 [default@goh_master] get agents where station_id = ascii(1110129);
 ---
 RowKey: 6c8efeb6-7209-11e2-890a-aacc0216
 = (column=, value=, timestamp=1364580868176000)
 = (column=character_points, value=, timestamp=136103068689)
 = (column=component_id, value=0, timestamp=1364580868176000)
 = (column=corporation_id, value=3efc729e-7209-11e2-890a-aacc0216, 
 timestamp=136103068689)
 = (column=entity_id, value=0, timestamp=1364580868176000)
 = (column=manufacturing, value=, timestamp=136103068689)
 = (column=model, value=55, timestamp=136103068689)
 = (column=name, value=Jenny Olifield, timestamp=136103068689)
 = (column=name_check, value=jenny_olifield, timestamp=136103068689)
 = (column=station_id, value=1110129, timestamp=1364580868176000)
 = (column=stats_intellect, value=8, timestamp=136103068689)
 = (column=stats_reflexes, value=8, timestamp=136103068689)
 = (column=stats_stamina, value=7, timestamp=136103068689)
 = (column=stats_technology, value=7, timestamp=136103068689)
 = (column=trading, value=, timestamp=136103068689)
 ---
 RowKey: dc413373-6b06-11e2-8943-aacc0216
 = (column=, value=, timestamp=136656818522)
 = (column=character_points, value=100, timestamp=1364580381651000)
 = (column=component_id, value=, timestamp=1364580381651000)
 = (column=corporation_id, value=574934cc-6b06-11e2-a512-aacc0200, 
 timestamp=1364580381651000)
 = (column=entity_id, value=0, timestamp=1364580381651000)
 = (column=manufacturing, value=, timestamp=1364580381651000)
 = (column=model, value=500018, timestamp=1364580381651000)
 = (column=name, value=Darren Matar, timestamp=1364580381651000)
 = (column=name_check, value=darren_matar, timestamp=1364580381651000)
 = (column=station_id, value=1110129, timestamp=1364580381651000)
 = (column=stats_intellect, value=10, timestamp=1364580381651000)
 = (column=stats_reflexes, value=10, timestamp=1364580381651000)
 = (column=stats_stamina, value=10, timestamp=1364580381651000)
 = (column=stats_technology, value=10, timestamp=1364580381651000)
 = (column=trading, value=1, timestamp=136656818522)
 ---
 RowKey: 0e7074ac-64bd-11e2-8c38-aacc0201
 = (column=, value=, timestamp=1364828039093000)
 = (column=character_points, value=, timestamp=136103068676)
 = (column=component_id, value=0, timestamp=1364828039093000)
 = (column=corporation_id, value=e398294e-64bc-11e2-8c38-aacc0201, 
 timestamp=136103068676)
 = (column=entity_id, value=0, timestamp=1364828039093000)
 = (column=manufacturing, value=1, timestamp=1362517535613000)
 = (column=model, value=58, timestamp=136103068676)
 = (column=name, value=Tom Bishop, timestamp=136103068676)
 = (column=name_check, value=tom_bishop, timestamp=136103068676)
 = (column=station_id, value=1110129, timestamp=1364828039093000)
 = (column=stats_intellect, value=9, timestamp=136103068676)
 = (column=stats_reflexes, value=7, timestamp=136103068676)
 = (column=stats_stamina, value=5, timestamp=136103068676)
 = (column=stats_technology, value=9, timestamp=136103068676)
 = (column=trading, value=, timestamp=136103068676)
 ---
 RowKey: 1b462f09-65f3-4148-a1a6-536b52b3bcfa
 = (column=, value=, timestamp=1366568185096000)
 = (column=character_points, value=100, timestamp=1364580381537000)
 = (column=component_id, value=, timestamp=1364580381537000)
 = (column=corporation_id, value=1d2a8803-d139-4b50-85eb-92cb1082de2e, 
 timestamp=1364580381537000)
 = (column=entity_id, value=0, timestamp=1364580381537000)
 = (column=manufacturing, value=, timestamp=1364580381537000)
 = (column=model, value=53, timestamp=1364580381537000)
 = (column=name, value=Andrea Len, timestamp=1364580381537000)
 = (column=name_check,

[jira] [Commented] (CASSANDRA-5443) Add CAS CQL support


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643028#comment-13643028
 ] 

Sylvain Lebresne commented on CASSANDRA-5443:
-

bq. How about IF NOT EXISTS?

That's definitively reasonable. I'll add a fifth patch to add that syntax 
soonish.

 Add CAS CQL support
 ---

 Key: CASSANDRA-5443
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5443
 Project: Cassandra
  Issue Type: Sub-task
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Sylvain Lebresne
 Fix For: 2.0

 Attachments: 
 0001-Refactor-Update-and-Delete-statement-to-extract-common.txt, 
 0002-Add-syntax-to-support-conditional-update-delete.txt, 
 0003-Handle-deleted-and-expiring-column-in-paxos-updates.txt, 
 0004-Support-tombstones-when-comparing-for-CAS.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5501) Missing data on SELECT on secondary index

2013-04-26 Thread Marco Matarazzo (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643037#comment-13643037
 ] 

Marco Matarazzo commented on CASSANDRA-5501:


Alas, we had to update (and add) data in rows and now the query is correctly 
returning everything. 

I didn't know about the TRACING ON command, it's a great tool :-)

I'm going to still post the result of the command, hoping that, apart from the 
result being correct, it will give you some clues anyway.

cqlsh:goh_master select agent_id,name,station_id,trading from agents where 
station_id='1110129';

 agent_id | name   | station_id | trading
--+++-
 6c8efeb6-7209-11e2-890a-aacc0216 | Jenny Olifield |1110129 |null
 b653d8c4-5fca-11e2-bd3a-aacc0216 |mammolo |1110129 |null
 cf9df102-7394-11e2-890a-aacc0216 | riolla |1110129 |null
 f9c0278b-aa5f-11e2-a860-aacc0216 |   Terrinon |1110129 |null
 cf03e58b-6d6a-11e2-890a-aacc0216 | Fichte |1110129 |null
 7e5d9601-70b5-11e2-a512-aacc0200 |   miao |1110129 |null
 8e50ab8c-63e7-11e2-8c38-aacc0201 | Reaper |1110129 |   0
 bba46192-6c63-11e2-8c38-aacc0201 |   crafter1 |1110129 |null
 5521cda0-7394-11e2-890a-aacc0216 |   olea |1110129 |null
 dc413373-6b06-11e2-8943-aacc0216 |   Darren Matar |1110129 |   1
 0e7074ac-64bd-11e2-8c38-aacc0201 | Tom Bishop |1110129 |null
 02238149-717a-11e2-890a-aacc0216 |   Capt. Andrew |1110129 |null
 d4e5a014-72ac-11e2-890a-aacc0216 |  pluto |1110129 |null
 2a483b11-70b5-11e2-8c38-aacc0201 | alexey |1110129 |null
 1b462f09-65f3-4148-a1a6-536b52b3bcfa | Andrea Len |1110129 |   1
 9a96615a-7a72-11e2-a513-aacc0216 |  padme |1110129 |null
 58670d03-70b6-11e2-8c38-aacc0201 | trilly |1110129 |null


Tracing session: 1bd92f60-ae93-11e2-a990-2f5b109ee83c

 activity   
   | timestamp| source   | source_elapsed
---+--+--+

execute_cql3_query | 19:02:42,519 | 10.10.30.169 |  0
   Message received from 
/10.10.30.169 | 19:02:42,516 | 10.10.30.170 | 21
 Executing indexed scan for [min(-9223372036854775808), 
min(-9223372036854775808)] | 19:02:42,518 | 10.10.30.170 |   1866
  Executing single-partition query on 
agents.agents_station_id | 19:02:42,518 | 10.10.30.170 |   2244
 Parsing 
statement | 19:02:42,519 | 10.10.30.169 | 42
  Acquiring sstable 
references | 19:02:42,519 | 10.10.30.170 |   2787
Peparing 
statement | 19:02:42,519 | 10.10.30.169 |122
 Merging memtable 
contents | 19:02:42,519 | 10.10.30.170 |   3107
 Determining replicas to 
query | 19:02:42,519 | 10.10.30.169 |216
 Key cache hit for sstable 
620 | 19:02:42,520 | 10.10.30.170 |   3806
Merging data from memtables and 1 
sstables | 19:02:42,520 | 10.10.30.170 |   4135
   Read 17 live cells and 0 
tombstoned | 19:02:42,521 | 10.10.30.170 |   4826
  Sending message to 
/10.10.30.170 | 19:02:42,522 | 10.10.30.169 |   2761
Executing single-partition query on 
agents | 19:02:42,522 | 10.10.30.170 |   5490
  Acquiring sstable 
references | 19:02:42,522 | 10.10.30.170 |   5782
 Merging memtable 
contents | 19:02:42,522 | 10.10.30.170 |   6062
 Key cache hit for sstable 
443 | 19:02:42,523 | 10.10.30.170 |   6350
Merging data from memtables and 1 
sstables | 19:02:42,523 | 10.10.30.170 |   6628
Read 1 live cells and 3 
tombstoned | 19:02:42,523 | 10.10.30.170 |   6991

[jira] [Updated] (CASSANDRA-5501) Missing data on SELECT on secondary index

2013-04-26 Thread Marco Matarazzo (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marco Matarazzo updated CASSANDRA-5501:
---

Attachment: tracing.txt

(well-formatted tracing)

 Missing data on SELECT on secondary index 
 --

 Key: CASSANDRA-5501
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5501
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.4
 Environment: linux ubuntu 12.04
Reporter: Marco Matarazzo
 Attachments: query_log.txt, tracing.txt


 We have a 3 nodes cluster, and a keyspace with RF = 3.
 From cassandra-cli everything is fine (we actually never use it, I just 
 launched it for a check in this particular case).
 [default@goh_master] get agents where station_id = ascii(1110129);
 ---
 RowKey: 6c8efeb6-7209-11e2-890a-aacc0216
 = (column=, value=, timestamp=1364580868176000)
 = (column=character_points, value=, timestamp=136103068689)
 = (column=component_id, value=0, timestamp=1364580868176000)
 = (column=corporation_id, value=3efc729e-7209-11e2-890a-aacc0216, 
 timestamp=136103068689)
 = (column=entity_id, value=0, timestamp=1364580868176000)
 = (column=manufacturing, value=, timestamp=136103068689)
 = (column=model, value=55, timestamp=136103068689)
 = (column=name, value=Jenny Olifield, timestamp=136103068689)
 = (column=name_check, value=jenny_olifield, timestamp=136103068689)
 = (column=station_id, value=1110129, timestamp=1364580868176000)
 = (column=stats_intellect, value=8, timestamp=136103068689)
 = (column=stats_reflexes, value=8, timestamp=136103068689)
 = (column=stats_stamina, value=7, timestamp=136103068689)
 = (column=stats_technology, value=7, timestamp=136103068689)
 = (column=trading, value=, timestamp=136103068689)
 ---
 RowKey: dc413373-6b06-11e2-8943-aacc0216
 = (column=, value=, timestamp=136656818522)
 = (column=character_points, value=100, timestamp=1364580381651000)
 = (column=component_id, value=, timestamp=1364580381651000)
 = (column=corporation_id, value=574934cc-6b06-11e2-a512-aacc0200, 
 timestamp=1364580381651000)
 = (column=entity_id, value=0, timestamp=1364580381651000)
 = (column=manufacturing, value=, timestamp=1364580381651000)
 = (column=model, value=500018, timestamp=1364580381651000)
 = (column=name, value=Darren Matar, timestamp=1364580381651000)
 = (column=name_check, value=darren_matar, timestamp=1364580381651000)
 = (column=station_id, value=1110129, timestamp=1364580381651000)
 = (column=stats_intellect, value=10, timestamp=1364580381651000)
 = (column=stats_reflexes, value=10, timestamp=1364580381651000)
 = (column=stats_stamina, value=10, timestamp=1364580381651000)
 = (column=stats_technology, value=10, timestamp=1364580381651000)
 = (column=trading, value=1, timestamp=136656818522)
 ---
 RowKey: 0e7074ac-64bd-11e2-8c38-aacc0201
 = (column=, value=, timestamp=1364828039093000)
 = (column=character_points, value=, timestamp=136103068676)
 = (column=component_id, value=0, timestamp=1364828039093000)
 = (column=corporation_id, value=e398294e-64bc-11e2-8c38-aacc0201, 
 timestamp=136103068676)
 = (column=entity_id, value=0, timestamp=1364828039093000)
 = (column=manufacturing, value=1, timestamp=1362517535613000)
 = (column=model, value=58, timestamp=136103068676)
 = (column=name, value=Tom Bishop, timestamp=136103068676)
 = (column=name_check, value=tom_bishop, timestamp=136103068676)
 = (column=station_id, value=1110129, timestamp=1364828039093000)
 = (column=stats_intellect, value=9, timestamp=136103068676)
 = (column=stats_reflexes, value=7, timestamp=136103068676)
 = (column=stats_stamina, value=5, timestamp=136103068676)
 = (column=stats_technology, value=9, timestamp=136103068676)
 = (column=trading, value=, timestamp=136103068676)
 ---
 RowKey: 1b462f09-65f3-4148-a1a6-536b52b3bcfa
 = (column=, value=, timestamp=1366568185096000)
 = (column=character_points, value=100, timestamp=1364580381537000)
 = (column=component_id, value=, timestamp=1364580381537000)
 = (column=corporation_id, value=1d2a8803-d139-4b50-85eb-92cb1082de2e, 
 timestamp=1364580381537000)
 = (column=entity_id, value=0, timestamp=1364580381537000)
 = (column=manufacturing, value=, timestamp=1364580381537000)
 = (column=model, value=53, timestamp=1364580381537000)
 = (column=name, value=Andrea Len, timestamp=1364580381537000)
 = (column=name_check, value=andrea_len, timestamp=1364580381537000)
 = (column=station_id, value=1110129, timestamp=1364580381537000)
 = (column=stats_intellect, value=10, timestamp=1364580381537000)
 = (column=stats_reflexes, value=10, timestamp=1364580381537000)
 = (column=stats_stamina, value=10,

[jira] [Commented] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0

2013-04-26 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643052#comment-13643052
 ] 

Marcus Eriksson commented on CASSANDRA-5511:


lgtm!

 Clean up backwards compatibility complexity for 2.0
 ---

 Key: CASSANDRA-5511
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: CASSANDRA-5511-on_1.2.patch, 
 CASSANDRA-5511-on_trunk.patch


 We've supported rolling upgrades (network-compatible for read/write 
 operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required 
 being on a recent release of the immediately prior major series for this to 
 work as desired.
 Meanwhile, we still support reading sstables at least back to 0.6 and 
 possibly even earlier.  This makes dealing with changes to the sstable quite 
 challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
 2.0 is a good place to drop support for sstables older than 1.2.5.  Our 
 experience with network compatibility demonstrates that this is not an 
 unreasonable burden to impose, and the major version number change suggests 
 that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643054#comment-13643054
]

Sylvain Lebresne commented on CASSANDRA-5417:
-

Oh, if that's relatively standalone, then yes that would be awesome.

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

[1/4] Removed compatibility with pre-1.2.5 sstables and network messages patch by jbellis; reviewed by marcuse for CASSANDRA-5511

2013-04-26 Thread jbellis

Updated Branches:
  refs/heads/trunk a45918b08 - 7f2c3a8e4


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f2c3a8e/test/unit/org/apache/cassandra/utils/LegacyBloomFilterTest.java
--
diff --git a/test/unit/org/apache/cassandra/utils/LegacyBloomFilterTest.java 
b/test/unit/org/apache/cassandra/utils/LegacyBloomFilterTest.java
deleted file mode 100644
index e7319db..000
--- a/test/unit/org/apache/cassandra/utils/LegacyBloomFilterTest.java
+++ /dev/null
@@ -1,132 +0,0 @@
-/*
-* Licensed to the Apache Software Foundation (ASF) under one
-* or more contributor license agreements.  See the NOTICE file
-* distributed with this work for additional information
-* regarding copyright ownership.  The ASF licenses this file
-* to you under the Apache License, Version 2.0 (the
-* License); you may not use this file except in compliance
-* with the License.  You may obtain a copy of the License at
-*
-*http://www.apache.org/licenses/LICENSE-2.0
-*
-* Unless required by applicable law or agreed to in writing,
-* software distributed under the License is distributed on an
-* AS IS BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-* KIND, either express or implied.  See the License for the
-* specific language governing permissions and limitations
-* under the License.
-*/
-package org.apache.cassandra.utils;
-
-import java.io.IOException;
-import java.nio.ByteBuffer;
-import java.util.HashSet;
-import java.util.Iterator;
-import java.util.Set;
-import java.io.ByteArrayInputStream;
-import java.io.DataInputStream;
-import org.apache.cassandra.io.util.DataOutputBuffer;
-
-import org.junit.Before;
-import org.junit.Test;
-
-public class LegacyBloomFilterTest
-{
-public LegacyBloomFilter bf;
-
-public LegacyBloomFilterTest()
-{
-bf = LegacyBloomFilter.getFilter(FilterTestHelper.ELEMENTS, 
FilterTestHelper.MAX_FAILURE_RATE);
-}
-
-public static IFilter testSerialize(LegacyBloomFilter f) throws IOException
-{
-f.add(ByteBufferUtil.bytes(a));
-DataOutputBuffer out = new DataOutputBuffer();
-FilterFactory.serialize(f, out, FilterFactory.Type.SHA);
-
-ByteArrayInputStream in = new ByteArrayInputStream(out.getData(), 0, 
out.getLength());
-LegacyBloomFilter f2 = (LegacyBloomFilter) 
FilterFactory.deserialize(new DataInputStream(in), FilterFactory.Type.SHA, 
false);
-
-assert f2.isPresent(ByteBufferUtil.bytes(a));
-assert !f2.isPresent(ByteBufferUtil.bytes(b));
-return f2;
-}
-
-
-@Before
-public void clear()
-{
-bf.clear();
-}
-
-@Test(expected=UnsupportedOperationException.class)
-public void testBloomLimits1()
-{
-int maxBuckets = BloomCalculations.probs.length - 1;
-int maxK = BloomCalculations.probs[maxBuckets].length - 1;
-
-// possible
-BloomCalculations.computeBloomSpec(maxBuckets, 
BloomCalculations.probs[maxBuckets][maxK]);
-
-// impossible, throws
-BloomCalculations.computeBloomSpec(maxBuckets, 
BloomCalculations.probs[maxBuckets][maxK] / 2);
-}
-
-@Test
-public void testOne()
-{
-bf.add(ByteBufferUtil.bytes(a));
-assert bf.isPresent(ByteBufferUtil.bytes(a));
-assert !bf.isPresent(ByteBufferUtil.bytes(b));
-}
-
-@Test
-public void testFalsePositivesInt()
-{
-FilterTestHelper.testFalsePositives(bf, FilterTestHelper.intKeys(), 
FilterTestHelper.randomKeys2());
-}
-
-@Test
-public void testFalsePositivesRandom()
-{
-FilterTestHelper.testFalsePositives(bf, FilterTestHelper.randomKeys(), 
FilterTestHelper.randomKeys2());
-}
-
-@Test
-public void testWords()
-{
-if (KeyGenerator.WordGenerator.WORDS == 0)
-{
-return;
-}
-LegacyBloomFilter bf2 = 
LegacyBloomFilter.getFilter(KeyGenerator.WordGenerator.WORDS / 2, 
FilterTestHelper.MAX_FAILURE_RATE);
-int skipEven = KeyGenerator.WordGenerator.WORDS % 2 == 0 ? 0 : 2;
-FilterTestHelper.testFalsePositives(bf2,
-  new KeyGenerator.WordGenerator(skipEven, 
2),
-  new KeyGenerator.WordGenerator(1, 2));
-}
-
-public void testManyHashes(IteratorByteBuffer keys)
-{
-int MAX_HASH_COUNT = 128;
-SetInteger hashes = new HashSetInteger();
-int collisions = 0;
-while (keys.hasNext())
-{
-hashes.clear();
-for (int hashIndex : LegacyBloomFilter.getHashBuckets(keys.next(), 
MAX_HASH_COUNT, 1024 * 1024))
-{
-hashes.add(hashIndex);
-}
-collisions += (MAX_HASH_COUNT - hashes.size());
-}
-assert collisions = 100;
-}
-
-@Test
-public void testManyRandom()
-{
-testManyHashes(FilterTestHelper.randomKeys());
-}
-}

[jira] [Resolved] (CASSANDRA-5511) Clean up backwards compatibility complexity for 2.0


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-5511.
---

Resolution: Fixed

committed!

{{121 files changed, 370 insertions(+), 2257 deletions(-)}}

 Clean up backwards compatibility complexity for 2.0
 ---

 Key: CASSANDRA-5511
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5511
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
 Fix For: 2.0

 Attachments: CASSANDRA-5511-on_1.2.patch, 
 CASSANDRA-5511-on_trunk.patch


 We've supported rolling upgrades (network-compatible for read/write 
 operations) for several releases, but both 1.0 - 1.1 and 1.1 - 1.2 required 
 being on a recent release of the immediately prior major series for this to 
 work as desired.
 Meanwhile, we still support reading sstables at least back to 0.6 and 
 possibly even earlier.  This makes dealing with changes to the sstable quite 
 challenging; the recently written-and-reverted CASSANDRA-5487 comes to mind.
 2.0 is a good place to drop support for sstables older than 1.2.5.  Our 
 experience with network compatibility demonstrates that this is not an 
 unreasonable burden to impose, and the major version number change suggests 
 that this is a logical time to make such a change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643209#comment-13643209
]

T Jake Luciani commented on CASSANDRA-5417:
---

Sure, I'll post it with the dataset tonight.

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

[jira] [Commented] (CASSANDRA-4180) Single-pass compaction for LCR


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643245#comment-13643245
 ] 

Jonathan Ellis commented on CASSANDRA-4180:
---

Rebased on top of CASSANDRA-5511 with a fix or two: 
http://github.com/jbellis/cassandra/tree/4180-4

Now passes the start 1.2, rebuild and start 4180-4 test.

 Single-pass compaction for LCR
 --

 Key: CASSANDRA-4180
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4180
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Sylvain Lebresne
Assignee: Jonathan Ellis
  Labels: compaction
 Fix For: 2.0

 Attachments: scrub-error.txt


 LazilyCompactedRow reads all data twice to compact a row which is obviously 
 inefficient. The main reason we do that is to compute the row header. 
 However, CASSANDRA-2319 have removed the main part of that row header. What 
 remains is the size in bytes and the number of columns, but it should be 
 relatively simple to remove those, which would then remove the need for the 
 two-phase compaction.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5332) 1.2.3 Nodetool reports 1.1.10 nodes as down

2013-04-26 Thread Arya Goudarzi (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643261#comment-13643261
 ] 

Arya Goudarzi commented on CASSANDRA-5332:
--

This was due to this issue. The patch here fixed it. CASSANDRA-5432

 1.2.3 Nodetool reports 1.1.10 nodes as down
 ---

 Key: CASSANDRA-5332
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5332
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.3
 Environment: Sun Java 6_u39
 Ubuntu 10.04.1
 Cassandra 1.2.3 and 1.1.10
Reporter: Arya Goudarzi

 During exercise of a rolling upgrade from 1.1.10 to 1.2.3, I upgraded one 
 node from 1.1.10 to 1.2.3 today. It appears that node running 1.2.3 reports 
 nodes in 1.1.10 as Down in nodetool. This means that the nodes running 1.1.10 
 see all other nodes including 1.2.3 as Up. Here is the ring and gossip from 
 nodes with 1.1.10 for example. 231.121 is the upgraded node:
 Address DC  RackStatus State   Load
 Effective-Ownership Token
   
  141784319550391026443072753098378663700
 XX.180.36us-east 1b  Up Normal  49.47 GB25.00%
   1808575600
 XX.231.121  us-east 1c  Up Normal  47.08 GB25.00% 
  7089215977519551322153637656637080005
 XX.177.177  us-east 1d  Up Normal  33.64 GB25.00% 
  14178431955039102644307275311465584410
 XX.7.148us-east 1b  Up Normal  41.27 GB25.00% 
  42535295865117307932921825930779602030
 XX.20.9 us-east 1c  Up Normal  38.51 GB25.00% 
  49624511842636859255075463585608106435
 XX.86.255us-east 1d  Up Normal  34.78 GB25.00%
   56713727820156410577229101240436610840
 XX.63.230us-east 1b  Up Normal  38.11 GB25.00%
   85070591730234615865843651859750628460
 XX.163.36   us-east 1c  Up Normal  44.25 GB25.00% 
  92159807707754167187997289514579132865
 XX.31.234us-east 1d  Up Normal  44.66 GB25.00%
   99249023685273718510150927169407637270
 XX.132.169   us-east 1b  Up Normal  44.2 GB 25.00%
   127605887595351923798765477788721654890
 XX.71.63 us-east 1c  Up Normal  38.74 GB25.00%
   134695103572871475120919115443550159295
 XX.197.209  us-east 1d  Up Normal  41.5 GB 25.00% 
  141784319550391026443072753098378663700
 /XX.71.63
   RACK:1c
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.1598705272E10
   DC:us-east
   INTERNAL_IP:XX.194.92
   STATUS:NORMAL,134695103572871475120919115443550159295
   RPC_ADDRESS:XX.194.92
   RELEASE_VERSION:1.1.6
 /XX.86.255
   RACK:1d
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:3.734334162E10
   DC:us-east
   INTERNAL_IP:XX.6.195
   STATUS:NORMAL,56713727820156410577229101240436610840
   RPC_ADDRESS:XX.6.195
   RELEASE_VERSION:1.1.6
 /XX.7.148
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.4316975808E10
   DC:us-east
   INTERNAL_IP:XX.47.250
   STATUS:NORMAL,42535295865117307932921825930779602030
   RPC_ADDRESS:XX.47.250
   RELEASE_VERSION:1.1.6
 /XX.63.230
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.0918593305E10
   DC:us-east
   INTERNAL_IP:XX.89.127
   STATUS:NORMAL,85070591730234615865843651859750628460
   RPC_ADDRESS:XX.89.127
   RELEASE_VERSION:1.1.6
 /XX.132.169
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.745883458E10
   DC:us-east
   INTERNAL_IP:XX.94.161
   STATUS:NORMAL,127605887595351923798765477788721654890
   RPC_ADDRESS:XX.94.161
   RELEASE_VERSION:1.1.6
 /XX.180.36
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:5.311963027E10
   DC:us-east
   INTERNAL_IP:XX.123.112
   STATUS:NORMAL,1808575600
   RPC_ADDRESS:XX.123.112
   RELEASE_VERSION:1.1.6
 /XX.163.36
   RACK:1c
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.7516755022E10
   DC:us-east
   INTERNAL_IP:XX.163.180
   STATUS:NORMAL,92159807707754167187997289514579132865
   RPC_ADDRESS:XX.163.180
   RELEASE_VERSION:1.1.6
 /XX.31.234
   RACK:1d
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.7954372912E10
   DC:us-east
   INTERNAL_IP:XX.192.159
   STATUS:NORMAL,99249023685273718510150927169407637270
   RPC_ADDRESS:XX.192.159
   RELEASE_VERSION:1.1.6
 /XX.197.209
   RACK:1d
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.4558968005E10
   DC:us-east
   INTERNAL_IP:XX.66.205
   STATUS:NORMAL,141784319550391026443072753098378663700

[jira] [Resolved] (CASSANDRA-5332) 1.2.3 Nodetool reports 1.1.10 nodes as down

2013-04-26 Thread Arya Goudarzi (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arya Goudarzi resolved CASSANDRA-5332.
--

Resolution: Duplicate

 1.2.3 Nodetool reports 1.1.10 nodes as down
 ---

 Key: CASSANDRA-5332
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5332
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.3
 Environment: Sun Java 6_u39
 Ubuntu 10.04.1
 Cassandra 1.2.3 and 1.1.10
Reporter: Arya Goudarzi

 During exercise of a rolling upgrade from 1.1.10 to 1.2.3, I upgraded one 
 node from 1.1.10 to 1.2.3 today. It appears that node running 1.2.3 reports 
 nodes in 1.1.10 as Down in nodetool. This means that the nodes running 1.1.10 
 see all other nodes including 1.2.3 as Up. Here is the ring and gossip from 
 nodes with 1.1.10 for example. 231.121 is the upgraded node:
 Address DC  RackStatus State   Load
 Effective-Ownership Token
   
  141784319550391026443072753098378663700
 XX.180.36us-east 1b  Up Normal  49.47 GB25.00%
   1808575600
 XX.231.121  us-east 1c  Up Normal  47.08 GB25.00% 
  7089215977519551322153637656637080005
 XX.177.177  us-east 1d  Up Normal  33.64 GB25.00% 
  14178431955039102644307275311465584410
 XX.7.148us-east 1b  Up Normal  41.27 GB25.00% 
  42535295865117307932921825930779602030
 XX.20.9 us-east 1c  Up Normal  38.51 GB25.00% 
  49624511842636859255075463585608106435
 XX.86.255us-east 1d  Up Normal  34.78 GB25.00%
   56713727820156410577229101240436610840
 XX.63.230us-east 1b  Up Normal  38.11 GB25.00%
   85070591730234615865843651859750628460
 XX.163.36   us-east 1c  Up Normal  44.25 GB25.00% 
  92159807707754167187997289514579132865
 XX.31.234us-east 1d  Up Normal  44.66 GB25.00%
   99249023685273718510150927169407637270
 XX.132.169   us-east 1b  Up Normal  44.2 GB 25.00%
   127605887595351923798765477788721654890
 XX.71.63 us-east 1c  Up Normal  38.74 GB25.00%
   134695103572871475120919115443550159295
 XX.197.209  us-east 1d  Up Normal  41.5 GB 25.00% 
  141784319550391026443072753098378663700
 /XX.71.63
   RACK:1c
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.1598705272E10
   DC:us-east
   INTERNAL_IP:XX.194.92
   STATUS:NORMAL,134695103572871475120919115443550159295
   RPC_ADDRESS:XX.194.92
   RELEASE_VERSION:1.1.6
 /XX.86.255
   RACK:1d
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:3.734334162E10
   DC:us-east
   INTERNAL_IP:XX.6.195
   STATUS:NORMAL,56713727820156410577229101240436610840
   RPC_ADDRESS:XX.6.195
   RELEASE_VERSION:1.1.6
 /XX.7.148
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.4316975808E10
   DC:us-east
   INTERNAL_IP:XX.47.250
   STATUS:NORMAL,42535295865117307932921825930779602030
   RPC_ADDRESS:XX.47.250
   RELEASE_VERSION:1.1.6
 /XX.63.230
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.0918593305E10
   DC:us-east
   INTERNAL_IP:XX.89.127
   STATUS:NORMAL,85070591730234615865843651859750628460
   RPC_ADDRESS:XX.89.127
   RELEASE_VERSION:1.1.6
 /XX.132.169
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.745883458E10
   DC:us-east
   INTERNAL_IP:XX.94.161
   STATUS:NORMAL,127605887595351923798765477788721654890
   RPC_ADDRESS:XX.94.161
   RELEASE_VERSION:1.1.6
 /XX.180.36
   RACK:1b
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:5.311963027E10
   DC:us-east
   INTERNAL_IP:XX.123.112
   STATUS:NORMAL,1808575600
   RPC_ADDRESS:XX.123.112
   RELEASE_VERSION:1.1.6
 /XX.163.36
   RACK:1c
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.7516755022E10
   DC:us-east
   INTERNAL_IP:XX.163.180
   STATUS:NORMAL,92159807707754167187997289514579132865
   RPC_ADDRESS:XX.163.180
   RELEASE_VERSION:1.1.6
 /XX.31.234
   RACK:1d
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.7954372912E10
   DC:us-east
   INTERNAL_IP:XX.192.159
   STATUS:NORMAL,99249023685273718510150927169407637270
   RPC_ADDRESS:XX.192.159
   RELEASE_VERSION:1.1.6
 /XX.197.209
   RACK:1d
   SCHEMA:99dce53b-487e-3e7b-a958-a1cc48d9f575
   LOAD:4.4558968005E10
   DC:us-east
   INTERNAL_IP:XX.66.205
   STATUS:NORMAL,141784319550391026443072753098378663700
   RPC_ADDRESS:XX.66.205
   RELEASE_VERSION:1.1.6
 /XX.177.177
   RACK:1d

git commit: remove unused assignments

2013-04-26 Thread dbrosius

Updated Branches:
  refs/heads/trunk 7f2c3a8e4 - 1aa987402


remove unused assignments


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1aa98740
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1aa98740
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1aa98740

Branch: refs/heads/trunk
Commit: 1aa987402d43b4a6ca7af8725e0e9fcb5477b6d2
Parents: 7f2c3a8
Author: Dave Brosius dbros...@apache.org
Authored: Fri Apr 26 22:23:48 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Fri Apr 26 22:23:48 2013 -0400

--
 .../cassandra/cql3/statements/SelectStatement.java |1 -
 .../apache/cassandra/service/StorageService.java   |4 +---
 2 files changed, 1 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1aa98740/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index 38fa361..e857104 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -273,7 +273,6 @@ public class SelectStatement implements CQLStatement
 // to account for the grouping of columns.
 // Since that doesn't work for maps/sets/lists, we now use the 
compositesToGroup option of SliceQueryFilter.
 // But we must preserve backward compatibility too (for mixed 
version cluster that is).
-int multiplier = cfDef.isCompact ? 1 : (cfDef.metadata.size() + 1);
 int toGroup = cfDef.isCompact ? -1 : cfDef.columns.size();
 ColumnSlice slice = new ColumnSlice(getRequestedBound(Bound.START, 
variables),
 getRequestedBound(Bound.END, 
variables));

http://git-wip-us.apache.org/repos/asf/cassandra/blob/1aa98740/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index d720df2..c950e20 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -1496,9 +1496,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 private void handleStateLeft(InetAddress endpoint, String[] pieces)
 {
 assert pieces.length = 2;
-CollectionToken tokens;
-Integer version = MessagingService.instance().getVersion(endpoint);
-tokens = getTokensFor(endpoint, pieces[1]);
+CollectionToken tokens = getTokensFor(endpoint, pieces[1]);
 
 if (logger.isDebugEnabled())
 logger.debug(Node  + endpoint +  state left, tokens  + tokens);

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643479#comment-13643479
 ] 

T Jake Luciani commented on CASSANDRA-5417:
---

The best thing todo is just give you the uncompacted sstables...

https://docs.google.com/file/d/0B4FSNkh7LrJCc040UTRKZFdtTVk/edit?usp=sharing


You should keep the uncompacted sstables around and reset after each test
The two scenarios I tested were:
  1. Time it takes to perform a major compaction (with and without patch)
  2. Latency of reads for reading across all uncompacted tables (with and 
without patch)

Here is the schema:

{code}
CREATE KEYSPACE mjff WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 1};

use mjff;
CREATE TABLE data (
name text,
type text,
date timestamp,
value double,
PRIMARY KEY(name,type,date)
) WITH COMPACT STORAGE;
{code}

The reader code is simple:

{code}
public class StressReads {

static int threadCount = 2;

public static String[] names = new 
String[]{APPLE,VIOLET,SUNFLOWER,ROSE,PEONY,ORCHID,ORANGE,MAPLE,LILLY,FLOX,DAISY,DAFODIL,CROCUS,CHERRY};
public static String[] types = new String[]{diffSecs,N.samples,
x.mean,x.absolue.deviation,x.standard.deviation,
y.mean,y.absolue.deviation,y.standard.deviation,
z.mean,z.absolue.deviation,z.standard.deviation};

static ThreadLocalCassandra.Client client = new 
ThreadLocalCassandra.Client() {
public Cassandra.Client initialValue() {
   try{
TTransport trans = new TFramedTransport(new 
TSocket(localhost,9160));
trans.open();

TProtocol prot = new TBinaryProtocol(trans);
Cassandra.Client client = new Cassandra.Client(prot);

client.set_keyspace(mjff);

   return client;
   }catch(Exception e){
throw new RuntimeException(err, e);
   }
}
};

static ExecutorService threadPool = 
Executors.newFixedThreadPool(threadCount);

static AtomicLong totalReads = new AtomicLong(0);
static long allReads = 0;
static int countSeconds = 0 ;

static Random rand = new Random();
public static void main(String[] args) throws InterruptedException {

for(int i=0; ithreadCount; i++) {
   threadPool.submit(new Runnable() {
   @Override
   public void run() {
   while(true){
StringBuffer sb = new StringBuffer();
   sb.append(Select value from data where name=');
   sb.append(names[rand.nextInt(names.length)]);
   sb.append(' and type=');
   sb.append(types[rand.nextInt(types.length)]);
   sb.append(' and date  '2012-03-01 00:00:00' LIMIT 
100);

   try {
CqlResult result = 
client.get().execute_cql3_query(ByteBufferUtil.bytes(sb.toString()), 
Compression.NONE, ConsistencyLevel.ONE);

totalReads.addAndGet(result.getRows().size());
   }catch(Exception e){
   e.printStackTrace();
   }
   }
   }
   });
}


while (true) {
Thread.sleep(1000);

long reads = totalReads.getAndSet(0);
   allReads += reads;
System.err.println(Read +reads+ per/sec, avg 
+allReads/++countSeconds);
}
}
}
{code}


 Push composites support in the storage engine
 -

 Key: CASSANDRA-5417
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
 Project: Cassandra
  Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 2.0


 CompositeType happens to be very useful and is now widely used: CQL3 heavily 
 rely on it, and super columns are now using it too internally. Besides, 
 CompositeType has been advised as a replacement of super columns on the 
 thrift side for a while, so it's safe to assume that it's generally used 
 there too.
 CompositeType has initially been introduced as just another AbstractType.  
 Meaning that the storage engine has no nothing whatsoever of composites 
 being, well, composite. This has the following drawbacks:
 * Because internally a composite value is handled as just a ByteBuffer, we 
 end up doing a lot of extra work. Typically, each time we compare 2 composite 
 value, we end up deserializing the components (which, while it doesn't copy 
 data per-se because we just slice the global ByteBuffer, still waste some cpu 
 cycles and allocate a bunch of ByteBuffer objects). And since compare can be 
 called *a lot*,

[jira] [Commented] (CASSANDRA-5417) Push composites support in the storage engine

[
https://issues.apache.org/jira/browse/CASSANDRA-5417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643481#comment-13643481
]

T Jake Luciani commented on CASSANDRA-5417:
---

Oh and for the read test you should explicitly disable minor compactions

Push composites support in the storage engine
-

Key: CASSANDRA-5417
URL: https://issues.apache.org/jira/browse/CASSANDRA-5417
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Fix For: 2.0

[jira] [Created] (CASSANDRA-5518) Clean out token range bisection on bootstrap

Jonathan Ellis created CASSANDRA-5518:
-

 Summary: Clean out token range bisection on bootstrap
 Key: CASSANDRA-5518
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5518
 Project: Cassandra
  Issue Type: Task
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 2.0


Bootstrapping a node by bisecting an existing node's range has never been very 
useful, and with vnodes it's thoroughly obsolete.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5499) Clean out TokenMetadata.getPrimaryRangeFor


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643525#comment-13643525
 ] 

Jonathan Ellis commented on CASSANDRA-5499:
---

CASSANDRA-5518 does one piece.

Unfortunately AbstractReplicationStrategy is all over this, that's the main 
challenge.  Kind of surprising it works.

 Clean out TokenMetadata.getPrimaryRangeFor
 --

 Key: CASSANDRA-5499
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5499
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.2.5


 TokenMetadata.getPrimaryRangeFor is dangerous because it can return invalid 
 results under NetworkTopologyStrategy.  Callers should generally use 
 StorageService.getPrimaryRangesForEndpoint instead.  See CASSANDRA-5424.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5499) Clean out TokenMetadata.getPrimaryRangeFor


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5499:
--

 Priority: Major  (was: Minor)
Fix Version/s: (was: 1.2.5)
   2.0
 Assignee: (was: Jonathan Ellis)

 Clean out TokenMetadata.getPrimaryRangeFor
 --

 Key: CASSANDRA-5499
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5499
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Jonathan Ellis
 Fix For: 2.0


 TokenMetadata.getPrimaryRangeFor is dangerous because it can return invalid 
 results under NetworkTopologyStrategy.  Callers should generally use 
 StorageService.getPrimaryRangesForEndpoint instead.  See CASSANDRA-5424.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4316) Compaction Throttle too bursty with large rows


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13643528#comment-13643528
 ] 

Jonathan Ellis commented on CASSANDRA-4316:
---

I was hoping we could get rid of RandomAccessReader, but that's not realistic; 
we need to be able to re-use readers for multiple operations for performance 
(CASSANDRA-4942), which implies being able to seek, which basically leaves us 
with RAR.

Also, if we throttle at the buffering level, throttling RAR is not really any 
more complex than a SequentialReader would be.

 Compaction Throttle too bursty with large rows
 --

 Key: CASSANDRA-4316
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4316
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Affects Versions: 0.8.0
Reporter: Wayne Lewis
Assignee: Yuki Morishita
 Fix For: 2.0

 Attachments: 4316-1.2.txt, 4316-1.2-v2.txt


 In org.apache.cassandra.db.compaction.CompactionIterable the check for 
 compaction throttling occurs once every 1000 rows. In our workload this is 
 much too large as we have many large rows (16 - 100 MB).
 With a 100 MB row, about 100 GB is read (and possibly written) before the 
 compaction throttle sleeps. This causes bursts of essentially unthrottled 
 compaction IO followed by a long sleep which yields inconsistence performance 
 and high error rates during the bursts.
 We applied a workaround to check throttle every row which solved our 
 performance and error issues:
 line 116 in org.apache.cassandra.db.compaction.CompactionIterable:
 if ((row++ % 1000) == 0)
 replaced with
 if ((row++ % 1) == 0)
 I think the better solution is to calculate how often throttle should be 
 checked based on the throttle rate to apply sleeps more consistently. E.g. if 
 16MB/sec is the limit then check for sleep after every 16MB is read so sleeps 
 are spaced out about every second.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (CASSANDRA-4316) Compaction Throttle too bursty with large rows