[jira] [Assigned] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.

2014-04-28 Thread Chander S Pechetty (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chander S Pechetty reassigned CASSANDRA-6973:
-

Assignee: Chander S Pechetty

 timestamp data type does ISO 8601 formats with 'Z' as time zone.
 

 Key: CASSANDRA-6973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Juho Mäkinen
Assignee: Chander S Pechetty
Priority: Trivial

 The timestamp data type does not support format where time zone is specified 
 with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:
 create table foo(ts timestamp primary key);
 insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
 cqlsh:test insert into foo(ts) values('2014-04-01T20:17:35Z');
 Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a  formatted date 
 (long)
 The example date was copied directly from ISO 8601 Wikipedia page. The 
 standard says that If the time is in UTC, add a Z directly after the time 
 without a space. Z is the zone designator for the zero UTC offset.
 Tested with cqlsh with 2.0.6 version.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


git commit: Multi-threaded scrub/cleanup/upgradesstables

2014-04-28 Thread marcuse
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 fccc123af - c8dcc7515


Multi-threaded scrub/cleanup/upgradesstables

Patch by rspitzer and marcuse for CASSANDRA-5547


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c8dcc751
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c8dcc751
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c8dcc751

Branch: refs/heads/cassandra-2.1
Commit: c8dcc7515e9f1d7f73f2d5e6651862a1b1023bea
Parents: fccc123
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 24 08:57:46 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Apr 28 08:52:00 2014 +0200

--
 CHANGES.txt |   1 +
 .../db/compaction/CompactionManager.java| 328 +--
 2 files changed, 153 insertions(+), 176 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c8dcc751/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5baaefd..1b67357 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -52,6 +52,7 @@
  * fix cassandra stress errors on reads with native protocol (CASSANDRA-7033)
  * Use OpOrder to guard sstable references for reads (CASSANDRA-6919)
  * Preemptive opening of compaction result (CASSANDRA-6916)
+ * Multi-threaded scrub/cleanup/upgradesstables (CASSANDRA-5547)
 Merged from 2.0:
  * Set JMX RMI port to 7199 (CASSANDRA-7087)
  * Use LOCAL_QUORUM for data reads at LOCAL_SERIAL (CASSANDRA-6939)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c8dcc751/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 30d564c..792c962 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -42,6 +42,7 @@ import javax.management.ObjectName;
 import javax.management.openmbean.OpenDataException;
 import javax.management.openmbean.TabularData;
 
+import com.google.common.base.Predicate;
 import com.google.common.base.Throwables;
 import com.google.common.collect.ArrayListMultimap;
 import com.google.common.collect.ConcurrentHashMultiset;
@@ -240,127 +241,130 @@ public class CompactionManager implements 
CompactionManagerMBean
 }
 }
 
-private abstract static class UnmarkingRunnable implements Runnable
+private AllSSTableOpStatus parallelAllSSTableOperation(final 
ColumnFamilyStore cfs, final OneSSTableOperation operation) throws 
ExecutionException, InterruptedException
 {
-private final ColumnFamilyStore cfs;
-private final IterableSSTableReader sstables;
-
-private UnmarkingRunnable(ColumnFamilyStore cfs, 
IterableSSTableReader sstables)
+IterableSSTableReader compactingSSTables = cfs.markAllCompacting();
+if (compactingSSTables == null)
 {
-this.cfs = cfs;
-this.sstables = sstables;
+logger.info(Aborting operation on {}.{} after failing to 
interrupt other compaction operations, cfs.keyspace.getName(), cfs.name);
+return AllSSTableOpStatus.ABORTED;
 }
-
-protected abstract void runMayThrow() throws IOException;
-
-public final void run()
+if (Iterables.isEmpty(compactingSSTables))
 {
-try
-{
-runMayThrow();
-}
-catch (Exception e)
-{
-throw Throwables.propagate(e);
-}
-finally
+logger.info(No sstables for {}.{}, cfs.keyspace.getName(), 
cfs.name);
+return AllSSTableOpStatus.SUCCESSFUL;
+}
+try
+{
+IterableSSTableReader sstables = 
operation.filterSSTables(compactingSSTables);
+ListFutureObject futures = new ArrayList();
+
+for (final SSTableReader sstable : sstables)
 {
-cfs.getDataTracker().unmarkCompacting(sstables);
+futures.add(executor.submit(new CallableObject()
+{
+@Override
+public Object call() throws Exception
+{
+operation.execute(sstable);
+return this;
+}
+}));
 }
+
+for (FutureObject f : futures)
+f.get();
+}
+finally
+{
+cfs.getDataTracker().unmarkCompacting(compactingSSTables);
 }
+return 

[1/2] git commit: Multi-threaded scrub/cleanup/upgradesstables

2014-04-28 Thread marcuse
Repository: cassandra
Updated Branches:
  refs/heads/trunk 7000efa21 - d0ddff1d3


Multi-threaded scrub/cleanup/upgradesstables

Patch by rspitzer and marcuse for CASSANDRA-5547


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c8dcc751
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c8dcc751
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c8dcc751

Branch: refs/heads/trunk
Commit: c8dcc7515e9f1d7f73f2d5e6651862a1b1023bea
Parents: fccc123
Author: Marcus Eriksson marc...@apache.org
Authored: Thu Apr 24 08:57:46 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Apr 28 08:52:00 2014 +0200

--
 CHANGES.txt |   1 +
 .../db/compaction/CompactionManager.java| 328 +--
 2 files changed, 153 insertions(+), 176 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c8dcc751/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 5baaefd..1b67357 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -52,6 +52,7 @@
  * fix cassandra stress errors on reads with native protocol (CASSANDRA-7033)
  * Use OpOrder to guard sstable references for reads (CASSANDRA-6919)
  * Preemptive opening of compaction result (CASSANDRA-6916)
+ * Multi-threaded scrub/cleanup/upgradesstables (CASSANDRA-5547)
 Merged from 2.0:
  * Set JMX RMI port to 7199 (CASSANDRA-7087)
  * Use LOCAL_QUORUM for data reads at LOCAL_SERIAL (CASSANDRA-6939)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c8dcc751/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 30d564c..792c962 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -42,6 +42,7 @@ import javax.management.ObjectName;
 import javax.management.openmbean.OpenDataException;
 import javax.management.openmbean.TabularData;
 
+import com.google.common.base.Predicate;
 import com.google.common.base.Throwables;
 import com.google.common.collect.ArrayListMultimap;
 import com.google.common.collect.ConcurrentHashMultiset;
@@ -240,127 +241,130 @@ public class CompactionManager implements 
CompactionManagerMBean
 }
 }
 
-private abstract static class UnmarkingRunnable implements Runnable
+private AllSSTableOpStatus parallelAllSSTableOperation(final 
ColumnFamilyStore cfs, final OneSSTableOperation operation) throws 
ExecutionException, InterruptedException
 {
-private final ColumnFamilyStore cfs;
-private final IterableSSTableReader sstables;
-
-private UnmarkingRunnable(ColumnFamilyStore cfs, 
IterableSSTableReader sstables)
+IterableSSTableReader compactingSSTables = cfs.markAllCompacting();
+if (compactingSSTables == null)
 {
-this.cfs = cfs;
-this.sstables = sstables;
+logger.info(Aborting operation on {}.{} after failing to 
interrupt other compaction operations, cfs.keyspace.getName(), cfs.name);
+return AllSSTableOpStatus.ABORTED;
 }
-
-protected abstract void runMayThrow() throws IOException;
-
-public final void run()
+if (Iterables.isEmpty(compactingSSTables))
 {
-try
-{
-runMayThrow();
-}
-catch (Exception e)
-{
-throw Throwables.propagate(e);
-}
-finally
+logger.info(No sstables for {}.{}, cfs.keyspace.getName(), 
cfs.name);
+return AllSSTableOpStatus.SUCCESSFUL;
+}
+try
+{
+IterableSSTableReader sstables = 
operation.filterSSTables(compactingSSTables);
+ListFutureObject futures = new ArrayList();
+
+for (final SSTableReader sstable : sstables)
 {
-cfs.getDataTracker().unmarkCompacting(sstables);
+futures.add(executor.submit(new CallableObject()
+{
+@Override
+public Object call() throws Exception
+{
+operation.execute(sstable);
+return this;
+}
+}));
 }
+
+for (FutureObject f : futures)
+f.get();
+}
+finally
+{
+cfs.getDataTracker().unmarkCompacting(compactingSSTables);
 }
+return 

[2/2] git commit: Merge branch 'cassandra-2.1' into trunk

2014-04-28 Thread marcuse
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d0ddff1d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d0ddff1d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d0ddff1d

Branch: refs/heads/trunk
Commit: d0ddff1d3ed44bdd3a902dd208207a810b1d1341
Parents: 7000efa c8dcc75
Author: Marcus Eriksson marc...@apache.org
Authored: Mon Apr 28 09:22:23 2014 +0200
Committer: Marcus Eriksson marc...@apache.org
Committed: Mon Apr 28 09:22:23 2014 +0200

--
 CHANGES.txt |   1 +
 .../db/compaction/CompactionManager.java| 328 +--
 2 files changed, 153 insertions(+), 176 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d0ddff1d/CHANGES.txt
--



[jira] [Updated] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.

2014-04-28 Thread Chander S Pechetty (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chander S Pechetty updated CASSANDRA-6973:
--

Attachment: trunk-6973.txt

The letter X is used to specify ISO 8601 time zone from the 
[SimpleDateFormat|http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#iso8601timezone]
 javadocs. 
The pattern letter 'X' was added in JDK 7 for supporting ISO8601 style time 
zone format. Replaced the TimestampSerializer ISO8601 pattern 'Z' with 'X', and 
added support for missing patterns.
{code borderStyle=solid}
timeZ
time±hh:mm
time±hhmm
time±hh
{code}
This patch by no means covers all the ISO 8601 combinations, but supplements 
the most frequently used patterns specified in the TimestampSerializer class.

 timestamp data type does ISO 8601 formats with 'Z' as time zone.
 

 Key: CASSANDRA-6973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Juho Mäkinen
Assignee: Chander S Pechetty
Priority: Trivial
 Attachments: trunk-6973.txt


 The timestamp data type does not support format where time zone is specified 
 with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:
 create table foo(ts timestamp primary key);
 insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
 cqlsh:test insert into foo(ts) values('2014-04-01T20:17:35Z');
 Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a  formatted date 
 (long)
 The example date was copied directly from ISO 8601 Wikipedia page. The 
 standard says that If the time is in UTC, add a Z directly after the time 
 without a space. Z is the zone designator for the zero UTC offset.
 Tested with cqlsh with 2.0.6 version.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Comment Edited] (CASSANDRA-6973) timestamp data type does ISO 8601 formats with 'Z' as time zone.

2014-04-28 Thread Chander S Pechetty (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982806#comment-13982806
 ] 

Chander S Pechetty edited comment on CASSANDRA-6973 at 4/28/14 7:22 AM:


The letter X is used to specify ISO 8601 time zone from the 
[SimpleDateFormat|http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#iso8601timezone]
 javadocs. 
The pattern letter 'X' was added in JDK 7 for supporting ISO8601 style time 
zone format. Replaced the TimestampSerializer ISO8601 pattern 'Z' with 'X', and 
added support for missing patterns.

timeZ
time±hh:mm
time±hhmm
time±hh

This patch by no means covers all the ISO 8601 combinations, but supplements 
the most frequently used patterns specified in the TimestampSerializer class.


was (Author: chander):
The letter X is used to specify ISO 8601 time zone from the 
[SimpleDateFormat|http://docs.oracle.com/javase/7/docs/api/java/text/SimpleDateFormat.html#iso8601timezone]
 javadocs. 
The pattern letter 'X' was added in JDK 7 for supporting ISO8601 style time 
zone format. Replaced the TimestampSerializer ISO8601 pattern 'Z' with 'X', and 
added support for missing patterns.
{code borderStyle=solid}
timeZ
time±hh:mm
time±hhmm
time±hh
{code}
This patch by no means covers all the ISO 8601 combinations, but supplements 
the most frequently used patterns specified in the TimestampSerializer class.

 timestamp data type does ISO 8601 formats with 'Z' as time zone.
 

 Key: CASSANDRA-6973
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6973
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Juho Mäkinen
Assignee: Chander S Pechetty
Priority: Trivial
 Attachments: trunk-6973.txt


 The timestamp data type does not support format where time zone is specified 
 with 'Z' (as in zulu aka. UTC+0 aka + time zone). Example:
 create table foo(ts timestamp primary key);
 insert into foo(ts) values('2014-04-01T20:17:35+'); -- this works
 cqlsh:test insert into foo(ts) values('2014-04-01T20:17:35Z');
 Bad Request: unable to coerce '2014-04-01T20:17:35Z' to a  formatted date 
 (long)
 The example date was copied directly from ISO 8601 Wikipedia page. The 
 standard says that If the time is in UTC, add a Z directly after the time 
 without a space. Z is the zone designator for the zero UTC offset.
 Tested with cqlsh with 2.0.6 version.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (CASSANDRA-6826) Query returns different number of results depending on fetchsize

2014-04-28 Thread Sylvain Lebresne (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-6826.
-

Resolution: Fixed

Thanks for the feedback. Closing as dup of CASSANDRA-6825 then (for the driver 
issue, I saw your issue and will look it up there).

 Query returns different number of results depending on fetchsize
 

 Key: CASSANDRA-6826
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6826
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: quad-core Windows 7 x64, single node cluster
 Cassandra 2.0.5
Reporter: Bill Mitchell
Assignee: Sylvain Lebresne

 I issue a query across the set of partitioned wide rows for one logical row, 
 where s, l, and partition specify the composite primary key for the row:
 SELECT ec, ea, rd FROM sr WHERE s = ? and partition IN ? and l = ? ALLOW 
 FILTERING;
 If I set fetchSize to only 1000 when the Cluster is configured, the query 
 sometimes does not return all the results.  In the particular case I am 
 chasing, it returns a total of 98586 rows.  If I increase the fetchsize to 
 10, all the 9 actual rows are returned.  This suggests there is some 
 problem with fetchsize re-establishing the position on the next segment of 
 the result set, at least when multiple partitions are being accessed.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6855) Native protocol V3

2014-04-28 Thread Sylvain Lebresne (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13982919#comment-13982919
 ] 

Sylvain Lebresne commented on CASSANDRA-6855:
-

Rebased and pushed an additional commit on the [same 
branch|https://github.com/pcmanus/cassandra/commits/proto-v3-2] to address the 
comments above up to the following remarks.

bq. I don't know if we want to allow negative timestamps (or if those would 
break things), but if not, we should explicitly check for that and throw an IRE.

I don't think there is a point in allowing them. I did added that restriction 
to the protocol and so the code throw a ProtocolException (rather than an IRE); 
I think it makes more sense to force clients to validate on there side.

bq. Can you add some unit tests for serialization/deserialization of UDTs, 
collections, and Events for both the v2 and v3 protocols?

I did added some tests. I'm sure we could improve on those somewhat, and 
possibly extend to other stuffs, but honestly I think further efforts would be 
better spent (in the sense that we'd get a much better and realistic testing of 
this) adding support for the new protocol version to existing drivers and check 
that the change didn't break previous versions (and that's where I intent to 
spend my efforts).


 Native protocol V3
 --

 Key: CASSANDRA-6855
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6855
 Project: Cassandra
  Issue Type: New Feature
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
 Fix For: 2.1 beta2


 I think we need a V3 of the protocol for 2.1. The things that this 
 could/should includes are:
 # Adding an optional Serial CL for protocol batches (like we have for QUERY 
 and EXECUTE). It was an oversight of V2 of not adding it, and now that we can 
 batch conditional updates, it's definitively missing.
 # Proper type codes for UDT. This is not *strictly* needed to be able to 
 support UDT since currently a UDT will be sent as a custom type with his 
 fully class name + arguments. But parsing that is no fun nor convenient for 
 clients. It's also not particular space efficient (though that's probably not 
 a huge concern since with prepared statement you can avoid sending the 
 ResultSet metadata every time).
 # Serialization format for collections. Currently the serialization format 
 only allow for 65K elements, each of 65K bytes size at most. While 
 collections are not meant to store large amount of data, having the 
 limitation in the protocol serialization format is the wrong way to deal with 
 that. Concretely, the current workaround for CASSANDRA-5428 is ugly. I'll 
 note that the current serialization format is also an obstacle to supporting 
 null inside collections (whether or not we want to support null there is a 
 good question, but here again I'm not sure being limited by the serialization 
 format is a good idea).
 # CASSANDRA-6178: I continue to believe that in many case it makes somewhat 
 more sense to have the default timestamp provided by the client (this is a 
 necessary condition for true idempotent retries in particular). I'm 
 absolutely fine making that optional and leaving server-side generated 
 timestamps by default, but since client can already provide timestamp in 
 query string anyway, I don't see a big deal in making it easier for client 
 driver to control that without messing with the query string.
 # Optional names for values in QUERY messages: it has been brought to my 
 attention that while V2 allows to send a query string with values for a 
 one-roundtrip bind-and-execute, a driver can't really support named bind 
 marker with that feature properly without parsing the query. The proposition 
 is thus to make it (optionally) possible to ship the name of the marker each 
 value is supposed to be bound to.
 I think that 1) and 2) are enough reason to make a V3 (even if there is 
 disagreement on the rest that is).
 3) is a little bit more involved tbh but I do think having the current 
 limitations bolted in the protocol serialization format is wrong in the long 
 run, and it turns out that due to UDT we will start storing serialized 
 collections internally so if we want to lift said limitation in the 
 serialization format, we should do it now and everywhere, as doing it 
 afterwards will be a lot more painful.
 4) and 5) are probably somewhat more minor, but at the same time, both are 
 completely optional (a driver won't have to support those if he doesn't 
 want). They are really just about making things more flexible for client 
 drivers and they are not particularly hard to support so I don't see too many 
 reasons not to include them.
 Last but not least, I know that some may find it wrong to do a new protocol 
 version with each major of C*, so let me state my view 

[jira] [Updated] (CASSANDRA-7072) Allow better specification of rack properties file

2014-04-28 Thread Patrick Ziegler (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Ziegler updated CASSANDRA-7072:
---

Attachment: cassandra-2.0.7-7072.txt

Added improvement for the issue

 Allow better specification of rack properties file
 --

 Key: CASSANDRA-7072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7072
 Project: Cassandra
  Issue Type: Improvement
  Components: Config, Core
Reporter: Patrick Ziegler
  Labels: properties, rack, snitch
 Fix For: 2.0.7

 Attachments: cassandra-2.0.7-7072.txt

   Original Estimate: 72h
  Remaining Estimate: 72h

 Currently the SnitchProperties class loads the properties file via the 
 classloader, this causes problems in our test environment where we use an 
 embedded cassandra for testing and cannot specify what file/where from to 
 load it.
 We will extend the functionality to support loading like the cassandra.yaml 
 file is supported, via the System.properties map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-7072) Allow better specification of rack properties file

2014-04-28 Thread Patrick Ziegler (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Ziegler updated CASSANDRA-7072:
---

Attachment: (was: cassandra-2.0.7-7072.txt)

 Allow better specification of rack properties file
 --

 Key: CASSANDRA-7072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7072
 Project: Cassandra
  Issue Type: Improvement
  Components: Config, Core
Reporter: Patrick Ziegler
  Labels: properties, rack, snitch
 Fix For: 2.0.7

   Original Estimate: 72h
  Remaining Estimate: 72h

 Currently the SnitchProperties class loads the properties file via the 
 classloader, this causes problems in our test environment where we use an 
 embedded cassandra for testing and cannot specify what file/where from to 
 load it.
 We will extend the functionality to support loading like the cassandra.yaml 
 file is supported, via the System.properties map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-7072) Allow better specification of rack properties file

2014-04-28 Thread Patrick Ziegler (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Patrick Ziegler updated CASSANDRA-7072:
---

Attachment: cassandra-2.0.7-7072.txt

Added support for System.properties usage for specifying snitch properties

 Allow better specification of rack properties file
 --

 Key: CASSANDRA-7072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7072
 Project: Cassandra
  Issue Type: Improvement
  Components: Config, Core
Reporter: Patrick Ziegler
  Labels: properties, rack, snitch
 Fix For: 2.0.7

 Attachments: cassandra-2.0.7-7072.txt

   Original Estimate: 72h
  Remaining Estimate: 72h

 Currently the SnitchProperties class loads the properties file via the 
 classloader, this causes problems in our test environment where we use an 
 embedded cassandra for testing and cannot specify what file/where from to 
 load it.
 We will extend the functionality to support loading like the cassandra.yaml 
 file is supported, via the System.properties map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (CASSANDRA-7101) Unable to complete request: one or more nodes were unavailable.

2014-04-28 Thread srikanth (JIRA)
srikanth created CASSANDRA-7101:
---

 Summary: Unable to complete request: one or more nodes were 
unavailable.
 Key: CASSANDRA-7101
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7101
 Project: Cassandra
  Issue Type: Bug
  Components: Config
Reporter: srikanth
 Fix For: 2.0.6


I setup multidata center cluster setup in cassandra. i created keysapce using 
network topology and created a table.but i am unable to insert data into 
tables. i am getting following error to insert data into cassandra
Unable to complete request: one or more nodes were unavailable.

Sample Code:

For Creating Keyspace:

CREATE KEYSPACE TestSample
   WITH replication = {'class': 'NetworkTopologyStrategy', 'DC1' : 1, 
'DC2' : 1}
AND durable_writes = false;

For Creating Tables:

CREATE TABLE users (
  user_id int PRIMARY KEY,
  fname text,
  lname text
);

Inserting Into Tables:
INSERT INTO users (user_id,  fname, lname)
  VALUES (1745, 'john', 'smith');




--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7101) Unable to complete request: one or more nodes were unavailable.

2014-04-28 Thread Michael Shuler (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983017#comment-13983017
 ] 

Michael Shuler commented on CASSANDRA-7101:
---

Please show the output of 'nodetool status'.
Please provide the precise commands to reproduce this, console output, system 
log files, when you reproduce your error.
(I assume that you meant you got this error on 2.0.6 and not to be fixed in 
2.0.6 - current release is 2.0.7)

works for me (with or without durable_write):
{noformat}
mshuler@hana:~$ ccm create -s -n 1:1 -v 2.0.7 test7101
Downloading 
http://archive.apache.org/dist/cassandra/2.0.7/apache-cassandra-2.0.7-src.tar.gz
 to /tmp/ccm-VV7qvF.tar.gz (10.848MB)
  11375316  [100.00%]
Extracting /tmp/ccm-VV7qvF.tar.gz as version 2.0.7 ...
Compiling Cassandra 2.0.7 ...
Current cluster is now: test7101
mshuler@hana:~$ ccm node1 status
Datacenter: dc1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  AddressLoad   Owns (effective)  Host ID 
  TokenRack
UN  127.0.0.1  36.01 KB   100.0%
3fb6966c-465d-4337-bc56-78f5bf489e61  -9223372036854775808 
r1
Datacenter: dc2
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  AddressLoad   Owns (effective)  Host ID 
  TokenRack
UN  127.0.0.2  36 KB  100.0%
257c8dca-8c32-4f32-8256-bf316aa0f933  0
r1
mshuler@hana:~$ ccm node1 cqlsh
Connected to test7101 at 127.0.0.1:9160.
[cqlsh 4.1.1 | Cassandra 2.0.7-SNAPSHOT | CQL spec 3.1.1 | Thrift protocol 
19.39.0]
Use HELP for help.
cqlsh CREATE KEYSPACE TestSample WITH replication = {'class': 
'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1};
cqlsh CREATE TABLE TestSample.users (user_id int PRIMARY KEY, fname text, 
lname text);
cqlsh INSERT INTO TestSample.users (user_id, fname, lname) VALUES (1745, 
'john', 'smith');
cqlsh SELECT * from TestSample.users;

 user_id | fname | lname
-+---+---
1745 |  john | smith

(1 rows)

cqlsh 
mshuler@hana:~$ ccm clear
mshuler@hana:~$ ccm status
node1: DOWN
node2: DOWN
mshuler@hana:~$ ccm start test7101
mshuler@hana:~$ ccm node1 status
Datacenter: dc1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  AddressLoad   Owns (effective)  Host ID 
  TokenRack
UN  127.0.0.1  35.94 KB   100.0%
6db6cda8-a4aa-4e5a-8803-87db3d7f9f82  -9223372036854775808 
r1
Datacenter: dc2
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  AddressLoad   Owns (effective)  Host ID 
  TokenRack
UN  127.0.0.2  35.88 KB   100.0%
cef6466e-dfc2-440f-9617-54a905a54805  0
r1
mshuler@hana:~$ ccm node1 cqlsh
Connected to test7101 at 127.0.0.1:9160.
[cqlsh 4.1.1 | Cassandra 2.0.7-SNAPSHOT | CQL spec 3.1.1 | Thrift protocol 
19.39.0]
Use HELP for help.
cqlsh DESC KEYSPACES 

system  system_traces

cqlsh CREATE KEYSPACE TestSample WITH replication = {'class': 
'NetworkTopologyStrategy', 'dc1': 1, 'dc2': 1} AND durable_writes = 'false';
cqlsh CREATE TABLE TestSample.users (user_id int PRIMARY KEY, fname text, 
lname text);
cqlsh INSERT INTO TestSample.users (user_id, fname, lname) VALUES (1745, 
'john', 'smith');
cqlsh SELECT * from TestSample.users;

 user_id | fname | lname
-+---+---
1745 |  john | smith

(1 rows)

cqlsh 
mshuler@hana:~$ ccm remove test7101
mshuler@hana:~$
{noformat}

 Unable to complete request: one or more nodes were unavailable.
 ---

 Key: CASSANDRA-7101
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7101
 Project: Cassandra
  Issue Type: Bug
  Components: Config
Reporter: srikanth
 Fix For: 2.0.6


 I setup multidata center cluster setup in cassandra. i created keysapce using 
 network topology and created a table.but i am unable to insert data into 
 tables. i am getting following error to insert data into cassandra
 Unable to complete request: one or more nodes were unavailable.
 
 Sample Code:
 For Creating Keyspace:
 CREATE KEYSPACE TestSample
WITH replication = {'class': 'NetworkTopologyStrategy', 'DC1' : 1, 
 'DC2' : 1}
 AND durable_writes = false;
 For Creating Tables:
 CREATE TABLE users (
   user_id int PRIMARY KEY,
   fname text,
   lname text
 );
 Inserting Into Tables:
 INSERT INTO users (user_id,  fname, lname)
   VALUES (1745, 'john', 'smith');



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7099) Concurrent instances of same Prepared Statement seeing intermingled result sets

2014-04-28 Thread Bill Mitchell (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983044#comment-13983044
 ] 

Bill Mitchell commented on CASSANDRA-7099:
--

I should clarify, as the title is misleading, that I was seeing more than 
intermingled results.  Intermingled suggests that the results from query 2 came 
back to query 1 and vice versa.  What I saw was the same results being returned 
to two different queries -- something that might happen if, say, there were a 
query results buffer based on PreparedStatement id without looking at the bound 
parameters, so that the second query thought the results were already 
calculated and grabbed up the results from the first.  

 Concurrent instances of same Prepared Statement seeing intermingled result 
 sets
 ---

 Key: CASSANDRA-7099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.7 with single node cluster
 Windows dual-core laptop
 DataStax Java driver 2.0.1
Reporter: Bill Mitchell

 I have a schema in which a wide row is partitioned into smaller rows.  (See 
 CASSANDRA-6826, CASSANDRA-6825 for more detail on this schema.)  In this 
 case, I randomly assigned the rows across the partitions based on the first 
 four hex digits of a hash value modulo the number of partitions.  
 Occasionally I need to retrieve the rows in order of insertion irrespective 
 of the partitioning.  Cassandra, of course, does not support this when paging 
 by fetch size is enabled, so I am issuing a query against each of the 
 partitions to obtain their rows in order, and merging the results:
 SELECT l, partition, cd, rd, ec, ea FROM sr WHERE s = ?, l = ?, partition = ? 
 ORDER BY cd ASC, ec ASC ALLOW FILTERING;
 These parallel queries are all instances of a single PreparedStatement.  
 What I saw was identical values from multiple queries, which by construction 
 should never happen, and after further investigation, discovered that rows 
 from partition 5 are being returned in the result set for the query against 
 another partition, e.g., 1.  This was so unbelievable that I added diagnostic 
 code in my test case to detect this:
 After reading 167 rows, returned partition 5 does not match query partition 4
 The merge logic works fine and delivers correct results when I use LIMIT to 
 avoid fetch size paging.  Even if there were a bug there, it is hard to see 
 how any client error explains ResultSet.one() returning a row whose values 
 don't match the constraints in that ResultSet's query.
 I'm not sure of the exact significance of 167, as I have configured the 
 queryFetchSize for the cluster to 1000, and in this merge logic I divide that 
 by the number of partitions, 7, so the fetchSize for each of these parallel 
 queries was set to 142.  I suspect this is being treated as a minimum 
 fetchSize, and the driver or server is rounding this up to fill a 
 transmission block.  When I prime the pump, issuing the query against each of 
 the partitions, the initial contents of the result sets are correct.  The 
 failure appears after we advance two of these queries to the next page.
 Although I had been experimenting with fetchMoreResults() for prefetching, I 
 disabled that to isolate this problem, so that is not a factor.   
 I have not yet tried preparing separate instances of the query, as I already 
 have common logic to cache and reuse already prepared statements.
 I have not proven that it is a server bug and not a Java driver bug, but on 
 first glance it was not obvious how the Java driver might associate the 
 responses with the wrong requests.  Were that happening, one would expect to 
 see the right overall collection of rows, just to the wrong queries, and not 
 duplicates, which is what I saw.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Comment Edited] (CASSANDRA-7099) Concurrent instances of same Prepared Statement seeing intermingled result sets

2014-04-28 Thread Bill Mitchell (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983044#comment-13983044
 ] 

Bill Mitchell edited comment on CASSANDRA-7099 at 4/28/14 2:33 PM:
---

I should clarify, as the title is misleading, that I was seeing more than 
intermingled results.  Intermingled suggests that the results from query 2 came 
back to query 1 and vice versa.  What I saw was the same results being returned 
to two different queries -- something that might happen if, say, there were a 
query results buffer based on PreparedStatement id ignoring the bound 
parameters, so that the second query thought the results were already 
calculated and grabbed up the results from the first.  


was (Author: wtmitchell3):
I should clarify, as the title is misleading, that I was seeing more than 
intermingled results.  Intermingled suggests that the results from query 2 came 
back to query 1 and vice versa.  What I saw was the same results being returned 
to two different queries -- something that might happen if, say, there were a 
query results buffer based on PreparedStatement id without looking at the bound 
parameters, so that the second query thought the results were already 
calculated and grabbed up the results from the first.  

 Concurrent instances of same Prepared Statement seeing intermingled result 
 sets
 ---

 Key: CASSANDRA-7099
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7099
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.0.7 with single node cluster
 Windows dual-core laptop
 DataStax Java driver 2.0.1
Reporter: Bill Mitchell

 I have a schema in which a wide row is partitioned into smaller rows.  (See 
 CASSANDRA-6826, CASSANDRA-6825 for more detail on this schema.)  In this 
 case, I randomly assigned the rows across the partitions based on the first 
 four hex digits of a hash value modulo the number of partitions.  
 Occasionally I need to retrieve the rows in order of insertion irrespective 
 of the partitioning.  Cassandra, of course, does not support this when paging 
 by fetch size is enabled, so I am issuing a query against each of the 
 partitions to obtain their rows in order, and merging the results:
 SELECT l, partition, cd, rd, ec, ea FROM sr WHERE s = ?, l = ?, partition = ? 
 ORDER BY cd ASC, ec ASC ALLOW FILTERING;
 These parallel queries are all instances of a single PreparedStatement.  
 What I saw was identical values from multiple queries, which by construction 
 should never happen, and after further investigation, discovered that rows 
 from partition 5 are being returned in the result set for the query against 
 another partition, e.g., 1.  This was so unbelievable that I added diagnostic 
 code in my test case to detect this:
 After reading 167 rows, returned partition 5 does not match query partition 4
 The merge logic works fine and delivers correct results when I use LIMIT to 
 avoid fetch size paging.  Even if there were a bug there, it is hard to see 
 how any client error explains ResultSet.one() returning a row whose values 
 don't match the constraints in that ResultSet's query.
 I'm not sure of the exact significance of 167, as I have configured the 
 queryFetchSize for the cluster to 1000, and in this merge logic I divide that 
 by the number of partitions, 7, so the fetchSize for each of these parallel 
 queries was set to 142.  I suspect this is being treated as a minimum 
 fetchSize, and the driver or server is rounding this up to fill a 
 transmission block.  When I prime the pump, issuing the query against each of 
 the partitions, the initial contents of the result sets are correct.  The 
 failure appears after we advance two of these queries to the next page.
 Although I had been experimenting with fetchMoreResults() for prefetching, I 
 disabled that to isolate this problem, so that is not a factor.   
 I have not yet tried preparing separate instances of the query, as I already 
 have common logic to cache and reuse already prepared statements.
 I have not proven that it is a server bug and not a Java driver bug, but on 
 first glance it was not obvious how the Java driver might associate the 
 responses with the wrong requests.  Were that happening, one would expect to 
 see the right overall collection of rows, just to the wrong queries, and not 
 duplicates, which is what I saw.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7059) Range query with strict bound on clustering column can return less results than required for compact tables

2014-04-28 Thread Christian Spriegel (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983089#comment-13983089
 ] 

Christian Spriegel commented on CASSANDRA-7059:
---

Is it possible that allow filtering is generally not allowed for compact 
storage tables? (due to this ticket?)

 Range query with strict bound on clustering column can return less results 
 than required for compact tables
 ---

 Key: CASSANDRA-7059
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7059
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne

 What's wrong:
 {noformat}
 CREATE TABLE test (
 k int,
 v int,
 PRIMARY KEY (k, v)
 ) WITH COMPACT STORAGE;
 INSERT INTO test(k, v) VALUES (0, 0);
 INSERT INTO test(k, v) VALUES (0, 1);
 INSERT INTO test(k, v) VALUES (1, 0);
 INSERT INTO test(k, v) VALUES (1, 1);
 INSERT INTO test(k, v) VALUES (2, 0);
 INSERT INTO test(k, v) VALUES (2, 1);
 SELECT * FROM test WHERE v  0 LIMIT 3 ALLOW FILTERING;
  k | v
 ---+---
  1 | 1
  0 | 1
 {noformat}
 That last query should return 3 results.
 The problem lies into how we deal with 'strict greater than' ({{}}) for 
 wide compact storage table. Namely, for those tables, we internally only 
 support inclusive bounds (for CQL3 tables this is not a problem as we deal 
 with this using the 'end-of-component' of the CompositeType encoding). So we 
 compensate by asking one more result than asked by the user, and we trim 
 afterwards if that was unnecessary. This works fine for per-partition 
 queries, but don't for range queries since we potentially would have to ask 
 for {{X}} more results where {{X}} is the number of partition fetched, but we 
 don't know {{X}} beforehand.
 I'll note that:
 * this has always be there
 * this only (potentially) affect compact tables
 * this only affect range queries that have a strict bound on the clustering 
 column (this means only {{ALLOW FILTERING}}) queries in particular.
 * this only matters if a {{LIMIT}} is set on the query.
 As for fixes, it's not entirely trivial. The right fix would probably be to 
 start supporting non-inclusive bound internally, but that's far from a small 
 fix and is at best a 2.1 fix (since we'll have to make a messaging protocol 
 change to ship some additional info for SliceQueryFilter). Also, this might 
 be a lot of work for something that only affect some {{ALLOW FILTERING}} 
 queries on compact tables.
 Another (somewhat simpler) solution might be to detect when we have this kind 
 of queries and use a pager with no limit. We would then query a first page 
 using the user limit (plus some smudge factor to avoid being inefficient too 
 often) and would continue paging unless either we've exhausted all results or 
 we can prove that post-processing we do have enough results to satisfy the 
 user limit.  This does mean in some case we might do 2 or more internal 
 queries, but in practice we can probably make that case very rare, and since 
 the query is an {{ALLOW FILTERING}} one, the user is somewhat warned that the 
 query may not be terribly efficient.
 Lastly, we could always start by disallowing the kind of query that is 
 potentially problematic (until we have a proper fix), knowing that users can 
 work around that by either using non-strict bounds or removing the {{LIMIT}}, 
 whichever makes the most sense in their case. In 1.2 in particular, we don't 
 have the query pagers, so the previous solution I describe would be a bit of 
 a mess to implement.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7098) Corrupt sstable error on restart in 2.1

2014-04-28 Thread Yuki Morishita (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983104#comment-13983104
 ] 

Yuki Morishita commented on CASSANDRA-7098:
---

[~suprajaj] 2.1 is still in development and file format can change. Can you 
wipe out /var/lib/cassandra/data and try again?

 Corrupt sstable error on restart in 2.1
 ---

 Key: CASSANDRA-7098
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7098
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: ubuntu 12.04
Reporter: shobana
Assignee: Yuki Morishita
Priority: Minor
 Fix For: 2.1 beta2


 Came across this error while trying to reproduce CASSANDRA-6858. C* was 
 cloned from git repo which has 2.1 merged into it recently. 
 On restart, following error happens: 
 ERROR 23:14:38 Corrupt sstable 
 /var/lib/cassandra/data/system/local-7ad54392bcdd35a684174e047860b377/system-local-ka-9=[CompressionInfo.db,
  Statistics.db, Filter.db, Data.db, Summary.db, TOC.txt, Index.db, 
 Digest.sha1]; skipped
 java.io.EOFException: null
   at 
 org.apache.cassandra.io.util.AbstractDataInput.readBoolean(AbstractDataInput.java:53)
  ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:265)
  ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.metadata.StatsMetadata$StatsMetadataSerializer.deserialize(StatsMetadata.java:194)
  ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:120)
  ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:92)
  ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:365) 
 ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:293) 
 ~[main/:na]
   at 
 org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:427) 
 ~[main/:na]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 [na:1.7.0_51]
   at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
 [na:1.7.0_51]
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  [na:1.7.0_51]
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  [na:1.7.0_51]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7072) Allow better specification of rack properties file

2014-04-28 Thread Brandon Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983120#comment-13983120
 ] 

Brandon Williams commented on CASSANDRA-7072:
-

Can you post a patch that a) does not reformat the license header, and b) uses 
'cassandra.rackdc.properties' instead of 'cassandra.snitch.properties', since 
the latter is more generic and won't apply to all snitches? Thanks!

 Allow better specification of rack properties file
 --

 Key: CASSANDRA-7072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7072
 Project: Cassandra
  Issue Type: Improvement
  Components: Config, Core
Reporter: Patrick Ziegler
  Labels: properties, rack, snitch
 Fix For: 2.0.7

 Attachments: cassandra-2.0.7-7072.txt

   Original Estimate: 72h
  Remaining Estimate: 72h

 Currently the SnitchProperties class loads the properties file via the 
 classloader, this causes problems in our test environment where we use an 
 embedded cassandra for testing and cannot specify what file/where from to 
 load it.
 We will extend the functionality to support loading like the cassandra.yaml 
 file is supported, via the System.properties map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-6751) Setting -Dcassandra.fd_initial_value_ms Results in NPE

2014-04-28 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6751:


Reviewer: Brandon Williams

+1

 Setting -Dcassandra.fd_initial_value_ms Results in NPE
 --

 Key: CASSANDRA-6751
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6751
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Dave Brosius
Priority: Minor
 Fix For: 1.2.17

 Attachments: 6751.txt


 Start Cassandra with {{-Dcassandra.fd_initial_value_ms=1000}} and you'll get 
 the following stacktrace:
 {noformat}
  INFO [main] 2014-02-21 14:45:57,731 StorageService.java (line 617) Starting 
 up server gossip
 ERROR [main] 2014-02-21 14:45:57,736 CassandraDaemon.java (line 464) 
 Exception encountered during startup
 java.lang.ExceptionInInitializerError
 at org.apache.cassandra.gms.Gossiper.init(Gossiper.java:178)
 at org.apache.cassandra.gms.Gossiper.clinit(Gossiper.java:71)
 at 
 org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:618)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:583)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:480)
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
 Caused by: java.lang.NullPointerException
 at 
 org.apache.cassandra.gms.FailureDetector.getInitialValue(FailureDetector.java:81)
 at 
 org.apache.cassandra.gms.FailureDetector.clinit(FailureDetector.java:48)
 ... 8 more
 ERROR [StorageServiceShutdownHook] 2014-02-21 14:45:57,754 
 CassandraDaemon.java (line 191) Exception in thread 
 Thread[StorageServiceShutdownHook,5,main]
 java.lang.NoClassDefFoundError: Could not initialize class 
 org.apache.cassandra.gms.Gossiper
 at 
 org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:550)
 at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
 at java.lang.Thread.run(Thread.java:724)
 {noformat}
 Glancing at the code, this is because the FailureDetector logger isn't 
 initialized when the static initialization of {{INITIAL_VALUE}} happens.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6572) Workload recording / playback

2014-04-28 Thread Jonathan Ellis (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983190#comment-13983190
 ] 

Jonathan Ellis commented on CASSANDRA-6572:
---

Suggest also storing whatever client or connection id we have; seems like we'd 
want to preserve thread relationships on playback.  

 Workload recording / playback
 -

 Key: CASSANDRA-6572
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6572
 Project: Cassandra
  Issue Type: New Feature
  Components: Core, Tools
Reporter: Jonathan Ellis
Assignee: Lyuben Todorov
 Fix For: 2.1.1

 Attachments: 6572-trunk.diff


 Write sample mode gets us part way to testing new versions against a real 
 world workload, but we need an easy way to test the query side as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[5/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread yukim
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a0f2589
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a0f2589
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a0f2589

Branch: refs/heads/cassandra-2.1
Commit: 9a0f25890d3a7499b29d1e2e659effaf90ec5e24
Parents: c8dcc75 9002e76
Author: Yuki Morishita yu...@apache.org
Authored: Mon Apr 28 11:59:52 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Apr 28 11:59:52 2014 -0500

--
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a0f2589/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--



[2/6] git commit: Avoid NPE when doing CFS.scrubDataDirectories

2014-04-28 Thread yukim
Avoid NPE when doing CFS.scrubDataDirectories


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9002e76d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9002e76d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9002e76d

Branch: refs/heads/cassandra-2.1
Commit: 9002e76d6971c3035a439209a9f2e5b8c2465247
Parents: 95e09f2
Author: Yuki Morishita yu...@apache.org
Authored: Mon Apr 28 11:58:43 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Apr 28 11:58:43 2014 -0500

--
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9002e76d/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 36bc470..bfd3b08 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -434,7 +434,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 for (File dir : directories.getCFDirectories())
 {
 File[] lockfiles = dir.listFiles(filter);
-if (lockfiles.length == 0)
+// lock files can be null if I/O error happens
+if (lockfiles == null || lockfiles.length == 0)
 continue;
 logger.info(Removing SSTables from failed streaming session. 
Found {} files to cleanup., lockfiles.length);
 



[3/6] git commit: Avoid NPE when doing CFS.scrubDataDirectories

2014-04-28 Thread yukim
Avoid NPE when doing CFS.scrubDataDirectories


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9002e76d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9002e76d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9002e76d

Branch: refs/heads/trunk
Commit: 9002e76d6971c3035a439209a9f2e5b8c2465247
Parents: 95e09f2
Author: Yuki Morishita yu...@apache.org
Authored: Mon Apr 28 11:58:43 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Apr 28 11:58:43 2014 -0500

--
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9002e76d/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 36bc470..bfd3b08 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -434,7 +434,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 for (File dir : directories.getCFDirectories())
 {
 File[] lockfiles = dir.listFiles(filter);
-if (lockfiles.length == 0)
+// lock files can be null if I/O error happens
+if (lockfiles == null || lockfiles.length == 0)
 continue;
 logger.info(Removing SSTables from failed streaming session. 
Found {} files to cleanup., lockfiles.length);
 



[1/6] git commit: Avoid NPE when doing CFS.scrubDataDirectories

2014-04-28 Thread yukim
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 95e09f262 - 9002e76d6
  refs/heads/cassandra-2.1 c8dcc7515 - 9a0f25890
  refs/heads/trunk d0ddff1d3 - 2e96779db


Avoid NPE when doing CFS.scrubDataDirectories


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9002e76d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9002e76d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9002e76d

Branch: refs/heads/cassandra-2.0
Commit: 9002e76d6971c3035a439209a9f2e5b8c2465247
Parents: 95e09f2
Author: Yuki Morishita yu...@apache.org
Authored: Mon Apr 28 11:58:43 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Apr 28 11:58:43 2014 -0500

--
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9002e76d/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 36bc470..bfd3b08 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -434,7 +434,8 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 for (File dir : directories.getCFDirectories())
 {
 File[] lockfiles = dir.listFiles(filter);
-if (lockfiles.length == 0)
+// lock files can be null if I/O error happens
+if (lockfiles == null || lockfiles.length == 0)
 continue;
 logger.info(Removing SSTables from failed streaming session. 
Found {} files to cleanup., lockfiles.length);
 



[4/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread yukim
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9a0f2589
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9a0f2589
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9a0f2589

Branch: refs/heads/trunk
Commit: 9a0f25890d3a7499b29d1e2e659effaf90ec5e24
Parents: c8dcc75 9002e76
Author: Yuki Morishita yu...@apache.org
Authored: Mon Apr 28 11:59:52 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Apr 28 11:59:52 2014 -0500

--
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9a0f2589/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--



[6/6] git commit: Merge branch 'cassandra-2.1' into trunk

2014-04-28 Thread yukim
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2e96779d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2e96779d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2e96779d

Branch: refs/heads/trunk
Commit: 2e96779db9135d29c384c485b99caecca62545ec
Parents: d0ddff1 9a0f258
Author: Yuki Morishita yu...@apache.org
Authored: Mon Apr 28 12:00:14 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Mon Apr 28 12:00:14 2014 -0500

--
 src/java/org/apache/cassandra/db/ColumnFamilyStore.java | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
--




[jira] [Resolved] (CASSANDRA-7091) java.lang.NullPointerException at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:437)

2014-04-28 Thread Yuki Morishita (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita resolved CASSANDRA-7091.
---

   Resolution: Fixed
Fix Version/s: 2.1 beta2

Actually, we want to skip when lockfiles is null (it can happen at IO error), 
so I changed the condition to continue on null.

{code}
if (lockfiles == null || logkfiles.length == 0)
continue;
{code}

Committed in 9002e76d6971c3035a439209a9f2e5b8c2465247.

 java.lang.NullPointerExceptionat 
 org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:437)
  
 --

 Key: CASSANDRA-7091
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7091
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Vivek Mishra
Assignee: Vivek Mishra
 Fix For: 2.0.8, 2.1 beta2


 java.lang.NullPointerException
   at 
 org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:437)
   at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:243)
   at 
 org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:414)
   at 
 org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:52)
 Fix { ColumnFamilyStore.java }
 ==
 Change 
 for (File dir : directories.getCFDirectories())
 {
 File[] lockfiles = dir.listFiles(filter);
 if (lockfiles.length == 0)
 continue;
 logger.info(Removing SSTables from failed streaming session. Found {} files 
 to cleanup., Integer.valueOf(lockfiles.length));
 WITH 
 for (File dir : directories.getCFDirectories())
 {
 File[] lockfiles = dir.listFiles(filter);
 if (lockfiles != null  lockfiles.length == 0)
 {
 continue;
 logger.info(Removing SSTables from failed streaming session. Found {} files 
 to cleanup., Integer.valueOf(lockfiles.length));
 }



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7090) Add ability to set/get logging levels to nodetool

2014-04-28 Thread Jackson Chung (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983296#comment-13983296
 ] 

Jackson Chung commented on CASSANDRA-7090:
--

hey [~vijay2...@gmail.com] , the diff file suggested it is modifying based on 
the logback, which is not applicable to 2.0. 

Did you mean to say rebase to 2.1 instead?

 Add ability to set/get logging levels to nodetool 
 --

 Key: CASSANDRA-7090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7090
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Jackson Chung
Assignee: Jackson Chung
Priority: Minor
 Fix For: 2.0.8, 2.1 beta2

 Attachments: 0001-CASSANDRA-7090.patch, logging.diff, patch-7090.v20


 While it is nice to use logback (per #CASSANDRA-5883) and with the autoreload 
 feature, in some cases ops/admin may not have the permission or ability to 
 modify the configuration file(s). Or the files are controlled by puppet/chef 
 so it is not desirable to modify them manually.
 There is already an existing operation for setLoggingLevel in the 
 StorageServuceMBean , so that's easy to expose that to the nodetool
 But what was lacking was ability to see the current log level settings for 
 various loggers. 
 The attached diff aims to do 3 things:
 # add JMX getLoggingLevels -- return a map of current loggers and the 
 corresponding levels
 # expose both getLoggingLevels and setLoggingLevel to nodetool. In 
 particular, the setLoggingLevel behave as follows:
 #* If both classQualifer and level are empty/null, it will reload the 
 configuration to reset.
 #* If classQualifer is not empty but level is empty/null, it will set the 
 level to null for the defined classQualifer
 #* The logback configuration should have  jmxConfigurator / set
 The diff is based on the master branch which uses logback, soit is not 
 applicable to 2.0 or 1.2. (2.1 is ok) Though it would be nice to have the 
 same ability for 2.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7090) Add ability to set/get logging levels to nodetool

2014-04-28 Thread Vijay (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983308#comment-13983308
 ] 

Vijay commented on CASSANDRA-7090:
--

Yeah, the ticket is marked for 2.0 so if we need to do something for 2.0 we 
need a patch to make it work for slf4j. 
Either way works,

 Add ability to set/get logging levels to nodetool 
 --

 Key: CASSANDRA-7090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7090
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Jackson Chung
Assignee: Jackson Chung
Priority: Minor
 Fix For: 2.0.8, 2.1 beta2

 Attachments: 0001-CASSANDRA-7090.patch, logging.diff, patch-7090.v20


 While it is nice to use logback (per #CASSANDRA-5883) and with the autoreload 
 feature, in some cases ops/admin may not have the permission or ability to 
 modify the configuration file(s). Or the files are controlled by puppet/chef 
 so it is not desirable to modify them manually.
 There is already an existing operation for setLoggingLevel in the 
 StorageServuceMBean , so that's easy to expose that to the nodetool
 But what was lacking was ability to see the current log level settings for 
 various loggers. 
 The attached diff aims to do 3 things:
 # add JMX getLoggingLevels -- return a map of current loggers and the 
 corresponding levels
 # expose both getLoggingLevels and setLoggingLevel to nodetool. In 
 particular, the setLoggingLevel behave as follows:
 #* If both classQualifer and level are empty/null, it will reload the 
 configuration to reset.
 #* If classQualifer is not empty but level is empty/null, it will set the 
 level to null for the defined classQualifer
 #* The logback configuration should have  jmxConfigurator / set
 The diff is based on the master branch which uses logback, soit is not 
 applicable to 2.0 or 1.2. (2.1 is ok) Though it would be nice to have the 
 same ability for 2.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7090) Add ability to set/get logging levels to nodetool

2014-04-28 Thread Brandon Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983313#comment-13983313
 ] 

Brandon Williams commented on CASSANDRA-7090:
-

Isn't patch-7090.v20 against 2.0 already?

 Add ability to set/get logging levels to nodetool 
 --

 Key: CASSANDRA-7090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7090
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Jackson Chung
Assignee: Jackson Chung
Priority: Minor
 Fix For: 2.0.8, 2.1 beta2

 Attachments: 0001-CASSANDRA-7090.patch, logging.diff, patch-7090.v20


 While it is nice to use logback (per #CASSANDRA-5883) and with the autoreload 
 feature, in some cases ops/admin may not have the permission or ability to 
 modify the configuration file(s). Or the files are controlled by puppet/chef 
 so it is not desirable to modify them manually.
 There is already an existing operation for setLoggingLevel in the 
 StorageServuceMBean , so that's easy to expose that to the nodetool
 But what was lacking was ability to see the current log level settings for 
 various loggers. 
 The attached diff aims to do 3 things:
 # add JMX getLoggingLevels -- return a map of current loggers and the 
 corresponding levels
 # expose both getLoggingLevels and setLoggingLevel to nodetool. In 
 particular, the setLoggingLevel behave as follows:
 #* If both classQualifer and level are empty/null, it will reload the 
 configuration to reset.
 #* If classQualifer is not empty but level is empty/null, it will set the 
 level to null for the defined classQualifer
 #* The logback configuration should have  jmxConfigurator / set
 The diff is based on the master branch which uses logback, soit is not 
 applicable to 2.0 or 1.2. (2.1 is ok) Though it would be nice to have the 
 same ability for 2.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6831) Updates to COMPACT STORAGE tables via cli drop CQL information

2014-04-28 Thread Aleksey Yeschenko (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983319#comment-13983319
 ] 

Aleksey Yeschenko commented on CASSANDRA-6831:
--

1.2 patch - LGTM (w/ those lines removed, and I'd rather inline 
copyAliasesFrom(), since it's not reused and is tiny).
2.0 patch (v2) LGTM

 Updates to COMPACT STORAGE tables via cli drop CQL information
 --

 Key: CASSANDRA-6831
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6831
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Russell Bradberry
Assignee: Mikhail Stepura
Priority: Minor
 Fix For: 1.2.17, 2.0.8, 2.1 beta2

 Attachments: 6831-2.0-v2.txt, cassandra-1.2-6831.patch, 
 cassandra-2.0-6831.patch


 If a COMPACT STORAGE table is altered using the CLI all information about the 
 column names reverts to the initial key, column1, column2 namings.  
 Additionally, the changes in the columns name will not take effect until the 
 Cassandra service is restarted.  This means that the clients using CQL will 
 continue to work properly until the service is restarted, at which time they 
 will start getting errors about non-existant columns in the table.
 When attempting to rename the columns back using ALTER TABLE an error stating 
 the column already exists will be raised.  The only way to get it back is to 
 ALTER TABLE and change the comment or something, which will bring back all 
 the original column names.
 This seems to be related to CASSANDRA-6676 and CASSANDRA-6370
 In cqlsh
 {code}
 Connected to cluster1 at 127.0.0.3:9160.
 [cqlsh 3.1.8 | Cassandra 1.2.15-SNAPSHOT | CQL spec 3.0.0 | Thrift protocol 
 19.36.2]
 Use HELP for help.
 cqlsh CREATE KEYSPACE test WITH REPLICATION = { 'class' : 'SimpleStrategy', 
 'replication_factor' : 3 };
 cqlsh USE test;
 cqlsh:test CREATE TABLE foo (bar text, baz text, qux text, PRIMARY KEY(bar, 
 baz) ) WITH COMPACT STORAGE;
 cqlsh:test describe table foo;
 CREATE TABLE foo (
   bar text,
   baz text,
   qux text,
   PRIMARY KEY (bar, baz)
 ) WITH COMPACT STORAGE AND
   bloom_filter_fp_chance=0.01 AND
   caching='KEYS_ONLY' AND
   comment='' AND
   dclocal_read_repair_chance=0.00 AND
   gc_grace_seconds=864000 AND
   read_repair_chance=0.10 AND
   replicate_on_write='true' AND
   populate_io_cache_on_flush='false' AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={'sstable_compression': 'SnappyCompressor'};
 {code}
 Now in cli:
 {code}
   Connected to: cluster1 on 127.0.0.3/9160
 Welcome to Cassandra CLI version 1.2.15-SNAPSHOT
 Type 'help;' or '?' for help.
 Type 'quit;' or 'exit;' to quit.
 [default@unknown] use test;
 Authenticated to keyspace: test
 [default@test] UPDATE COLUMN FAMILY foo WITH comment='hey this is a comment';
 3bf5fa49-5d03-34f0-b46c-6745f7740925
 {code}
 Now back in cqlsh:
 {code}
 cqlsh:test describe table foo;
 CREATE TABLE foo (
   bar text,
   column1 text,
   value text,
   PRIMARY KEY (bar, column1)
 ) WITH COMPACT STORAGE AND
   bloom_filter_fp_chance=0.01 AND
   caching='KEYS_ONLY' AND
   comment='hey this is a comment' AND
   dclocal_read_repair_chance=0.00 AND
   gc_grace_seconds=864000 AND
   read_repair_chance=0.10 AND
   replicate_on_write='true' AND
   populate_io_cache_on_flush='false' AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={'sstable_compression': 'SnappyCompressor'};
 cqlsh:test ALTER TABLE foo WITH comment='this is a new comment';
 cqlsh:test describe table foo;
 CREATE TABLE foo (
   bar text,
   baz text,
   qux text,
   PRIMARY KEY (bar, baz)
 ) WITH COMPACT STORAGE AND
   bloom_filter_fp_chance=0.01 AND
   caching='KEYS_ONLY' AND
   comment='this is a new comment' AND
   dclocal_read_repair_chance=0.00 AND
   gc_grace_seconds=864000 AND
   read_repair_chance=0.10 AND
   replicate_on_write='true' AND
   populate_io_cache_on_flush='false' AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={'sstable_compression': 'SnappyCompressor'};
 {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7090) Add ability to set/get logging levels to nodetool

2014-04-28 Thread Jackson Chung (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983316#comment-13983316
 ] 

Jackson Chung commented on CASSANDRA-7090:
--

right, that is why i have made 2 patches:

https://issues.apache.org/jira/secure/attachment/12642001/patch-7090.v20 -- 
based on  2.0
https://issues.apache.org/jira/secure/attachment/12641851/logging.diff -- 
based on master

 Add ability to set/get logging levels to nodetool 
 --

 Key: CASSANDRA-7090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7090
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Jackson Chung
Assignee: Jackson Chung
Priority: Minor
 Fix For: 2.0.8, 2.1 beta2

 Attachments: 0001-CASSANDRA-7090.patch, logging.diff, patch-7090.v20


 While it is nice to use logback (per #CASSANDRA-5883) and with the autoreload 
 feature, in some cases ops/admin may not have the permission or ability to 
 modify the configuration file(s). Or the files are controlled by puppet/chef 
 so it is not desirable to modify them manually.
 There is already an existing operation for setLoggingLevel in the 
 StorageServuceMBean , so that's easy to expose that to the nodetool
 But what was lacking was ability to see the current log level settings for 
 various loggers. 
 The attached diff aims to do 3 things:
 # add JMX getLoggingLevels -- return a map of current loggers and the 
 corresponding levels
 # expose both getLoggingLevels and setLoggingLevel to nodetool. In 
 particular, the setLoggingLevel behave as follows:
 #* If both classQualifer and level are empty/null, it will reload the 
 configuration to reset.
 #* If classQualifer is not empty but level is empty/null, it will set the 
 level to null for the defined classQualifer
 #* The logback configuration should have  jmxConfigurator / set
 The diff is based on the master branch which uses logback, soit is not 
 applicable to 2.0 or 1.2. (2.1 is ok) Though it would be nice to have the 
 same ability for 2.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7090) Add ability to set/get logging levels to nodetool

2014-04-28 Thread Vijay (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983325#comment-13983325
 ] 

Vijay commented on CASSANDRA-7090:
--

Ahaa let me review it. Thanks

 Add ability to set/get logging levels to nodetool 
 --

 Key: CASSANDRA-7090
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7090
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Jackson Chung
Assignee: Jackson Chung
Priority: Minor
 Fix For: 2.0.8, 2.1 beta2

 Attachments: 0001-CASSANDRA-7090.patch, logging.diff, patch-7090.v20


 While it is nice to use logback (per #CASSANDRA-5883) and with the autoreload 
 feature, in some cases ops/admin may not have the permission or ability to 
 modify the configuration file(s). Or the files are controlled by puppet/chef 
 so it is not desirable to modify them manually.
 There is already an existing operation for setLoggingLevel in the 
 StorageServuceMBean , so that's easy to expose that to the nodetool
 But what was lacking was ability to see the current log level settings for 
 various loggers. 
 The attached diff aims to do 3 things:
 # add JMX getLoggingLevels -- return a map of current loggers and the 
 corresponding levels
 # expose both getLoggingLevels and setLoggingLevel to nodetool. In 
 particular, the setLoggingLevel behave as follows:
 #* If both classQualifer and level are empty/null, it will reload the 
 configuration to reset.
 #* If classQualifer is not empty but level is empty/null, it will set the 
 level to null for the defined classQualifer
 #* The logback configuration should have  jmxConfigurator / set
 The diff is based on the master branch which uses logback, soit is not 
 applicable to 2.0 or 1.2. (2.1 is ok) Though it would be nice to have the 
 same ability for 2.0.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Benedict (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983417#comment-13983417
 ] 

Benedict commented on CASSANDRA-6694:
-

I've pushed a completed branch 
[here|https://github.com/belliottsmith/cassandra/tree/6694-reorg2]

I've taken to completion your flattening of the PoolAllocator and DataAllocator 
hierarchies, implemented DecoratedKey, reintroduced the extra unit tests, fixed 
some bugs with the Cell hierarchy, slightly rejigged the data layout for native 
cell to simplify offset calculation and fixed a performance regression and the 
message digest optimisation.

The only thing I haven't done is the refactors I would like to perform before 
we finally commit this, so as to make review easier for others.

Note I'm still running dtests and doing some final vetting, but I wanted to 
post this message now as I reckon this version is most likely ready and this is 
somewhat time critical, and because I want to avoid any duplicated effort in 
getting a final patch together.

I think I've addressed your concern's [~iamaleksey], however with the following 
notes:

bq. getAllocator() doesn’t belong to SecondaryIndex, API-wise. CFS#logFlush() 
and CFS.FLCF#run() should just use 
SecondaryIndexManager#getIndexesNotBackedByCfs() and get their allocators 
directly instead of using SIM#getIndexes() and checking for null.

This was a conscious decision to permit custom 2i use our allocators and count 
towards book keeping for memory utilisation.

bq. Composite/CellName/CellNameType/etc#copy() all now have an extra CFMetaData 
argument, while only NativeCell really uses it. Can we isolate its usage to a 
NativeCell-specific methods and leave the rest alone?

Not sure how we do that when either can be present when you want to perform 
these calls. Possible I'm missing something obvious though, so please do let me 
know :)

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Benedict (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983425#comment-13983425
 ] 

Benedict commented on CASSANDRA-6694:
-

Oh, also, [~iamaleksey]: Your assertion about super columns and sparse 
composites appears to be broken by the CliTest somehow. I haven't investigated, 
but this is why I introduced that branch. I've stripped out the condition and 
just always take that branch if we fail to lookup in cfMetaData, to deal with 
names being dropped whilst we aren't expecting, so it no longer assumes this 
but also copes with it.

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7042) Disk space growth until restart

2014-04-28 Thread Zach Aller (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983482#comment-13983482
 ] 

Zach Aller commented on CASSANDRA-7042:
---

Just thought I would also mention setup a brand new 3 node test cluster 
completely fresh and was able to reproduce.

 Disk space growth until restart
 ---

 Key: CASSANDRA-7042
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7042
 Project: Cassandra
  Issue Type: Bug
 Environment: Ubuntu 12.04
 Sun Java 7
 Cassandra 2.0.6
Reporter: Zach Aller
 Attachments: Screen Shot 2014-04-17 at 11.07.24 AM.png, Screen Shot 
 2014-04-18 at 11.47.30 AM.png, Screen Shot 2014-04-22 at 1.40.41 PM.png, 
 after.log, before.log, tabledump_after_restart.txt, 
 tabledump_before_restart.txt


 Cassandra will constantly eat disk space not sure whats causing it the only 
 thing that seems to fix it is a restart of cassandra this happens about every 
 3-5 hrs we will grow from about 350GB to 650GB with no end in site. Once we 
 restart cassandra it usually all clears itself up and disks return to normal 
 for a while then something triggers its and starts climbing again. Sometimes 
 when we restart compactions pending skyrocket and if we restart a second time 
 the compactions pending drop off back to a normal level. One other thing to 
 note is the space is not free'd until cassandra starts back up and not when 
 shutdown.
 I will get a clean log of before and after restarting next time it happens 
 and post it.
 Here is a common ERROR in our logs that might be related
 ERROR [CompactionExecutor:46] 2014-04-15 09:12:51,040 CassandraDaemon.java 
 (line 196) Exception in thread Thread[CompactionExecutor:46,1,main]
 java.lang.RuntimeException: java.io.FileNotFoundException: 
 /local-project/cassandra_data/data/wxgrid/grid/wxgrid-grid-jb-468677-Data.db 
 (No such file or directory)
 at 
 org.apache.cassandra.io.util.ThrottledReader.open(ThrottledReader.java:53)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.openDataReader(SSTableReader.java:1355)
 at 
 org.apache.cassandra.io.sstable.SSTableScanner.init(SSTableScanner.java:67)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1161)
 at 
 org.apache.cassandra.io.sstable.SSTableReader.getScanner(SSTableReader.java:1173)
 at 
 org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getScanners(LeveledCompactionStrategy.java:194)
 at 
 org.apache.cassandra.db.compaction.AbstractCompactionStrategy.getScanners(AbstractCompactionStrategy.java:258)
 at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:126)
 at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
 at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
 at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
 at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
 at 
 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
 at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
 at java.util.concurrent.FutureTask.run(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
 at java.lang.Thread.run(Unknown Source)
 Caused by: java.io.FileNotFoundException: 
 /local-project/cassandra_data/data/wxgrid/grid/wxgrid-grid-jb-468677-Data.db 
 (No such file or directory)
 at java.io.RandomAccessFile.open(Native Method)
 at java.io.RandomAccessFile.init(Unknown Source)
 at 
 org.apache.cassandra.io.util.RandomAccessReader.init(RandomAccessReader.java:58)
 at 
 org.apache.cassandra.io.util.ThrottledReader.init(ThrottledReader.java:35)
 at 
 org.apache.cassandra.io.util.ThrottledReader.open(ThrottledReader.java:49)
 ... 17 more



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (CASSANDRA-7102) CqlStorage loads incorrect schema

2014-04-28 Thread Robbie Strickland (JIRA)
Robbie Strickland created CASSANDRA-7102:


 Summary: CqlStorage loads incorrect schema
 Key: CASSANDRA-7102
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7102
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
 Environment: OS-X, Hadoop 1.2.1, C* 2.0.0, Pig 0.12
Reporter: Robbie Strickland


When running C* 2.0.0 with Hadoop 1.2.1 and Pig 0.12, attempting to load a 
table using CqlStorage produces an invalid schema.

Given the following table:

CREATE TABLE checkins (
  user text,
  time bigint,
  checkinid text,
  address text,
  geohash text,
  lat double,
  lon double,
  PRIMARY KEY (user, time, checkinid)
)

I load my table in Pig as follows:

checkins = LOAD 'cql://wxcheckin/checkins' USING CqlStorage()

... which produces the following schema:

(user:chararray,time:long,checkinid:chararray,address:chararray,checkinid:chararray,geohash:chararray,lat:double,lon:double,time:long,user:chararray)

As you can see it repeats the fields in the PK, and it throws an error when any 
field is referenced:

Invalid field projection. Projected field [time] does not exist in schema: 
user:chararray,time:long,checkinid:chararray,address:chararray,checkinid:chararray,geohash:chararray,lat:double,lon:double,time:long,user:chararray.

Simply upgrading to C* 2.0.7 fixes the issue.




--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7102) CqlStorage loads incorrect schema

2014-04-28 Thread Brandon Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983490#comment-13983490
 ] 

Brandon Williams commented on CASSANDRA-7102:
-

If upgrading C* fixed it, I'm going to notaproblem this.  Could have been 
CASSANDRA-5201 or something else, but either way it's fixed now.

 CqlStorage loads incorrect schema
 -

 Key: CASSANDRA-7102
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7102
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
 Environment: OS-X, Hadoop 1.2.1, C* 2.0.0, Pig 0.12
Reporter: Robbie Strickland

 When running C* 2.0.0 with Hadoop 1.2.1 and Pig 0.12, attempting to load a 
 table using CqlStorage produces an invalid schema.
 Given the following table:
 CREATE TABLE checkins (
   user text,
   time bigint,
   checkinid text,
   address text,
   geohash text,
   lat double,
   lon double,
   PRIMARY KEY (user, time, checkinid)
 )
 I load my table in Pig as follows:
 checkins = LOAD 'cql://wxcheckin/checkins' USING CqlStorage()
 ... which produces the following schema:
 (user:chararray,time:long,checkinid:chararray,address:chararray,checkinid:chararray,geohash:chararray,lat:double,lon:double,time:long,user:chararray)
 As you can see it repeats the fields in the PK, and it throws an error when 
 any field is referenced:
 Invalid field projection. Projected field [time] does not exist in schema: 
 user:chararray,time:long,checkinid:chararray,address:chararray,checkinid:chararray,geohash:chararray,lat:double,lon:double,time:long,user:chararray.
 Simply upgrading to C* 2.0.7 fixes the issue.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (CASSANDRA-7102) CqlStorage loads incorrect schema

2014-04-28 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-7102.
-

Resolution: Not a Problem

 CqlStorage loads incorrect schema
 -

 Key: CASSANDRA-7102
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7102
 Project: Cassandra
  Issue Type: Bug
  Components: Hadoop
 Environment: OS-X, Hadoop 1.2.1, C* 2.0.0, Pig 0.12
Reporter: Robbie Strickland

 When running C* 2.0.0 with Hadoop 1.2.1 and Pig 0.12, attempting to load a 
 table using CqlStorage produces an invalid schema.
 Given the following table:
 CREATE TABLE checkins (
   user text,
   time bigint,
   checkinid text,
   address text,
   geohash text,
   lat double,
   lon double,
   PRIMARY KEY (user, time, checkinid)
 )
 I load my table in Pig as follows:
 checkins = LOAD 'cql://wxcheckin/checkins' USING CqlStorage()
 ... which produces the following schema:
 (user:chararray,time:long,checkinid:chararray,address:chararray,checkinid:chararray,geohash:chararray,lat:double,lon:double,time:long,user:chararray)
 As you can see it repeats the fields in the PK, and it throws an error when 
 any field is referenced:
 Invalid field projection. Projected field [time] does not exist in schema: 
 user:chararray,time:long,checkinid:chararray,address:chararray,checkinid:chararray,geohash:chararray,lat:double,lon:double,time:long,user:chararray.
 Simply upgrading to C* 2.0.7 fixes the issue.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (CASSANDRA-7103) Very poor performance with simple setup

2014-04-28 Thread Martin Bligh (JIRA)
Martin Bligh created CASSANDRA-7103:
---

 Summary: Very poor performance with simple setup
 Key: CASSANDRA-7103
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7103
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Fedora 19 (also happens on Ubuntu), Cassandra 2.0.7. dsc 
standard install
Reporter: Martin Bligh


Single node (this is just development, 32GB 20 core server), single disk array.

Create the following table:

CREATE TABLE reut (
  time_order bigint,
  time_start bigint,
  ack_us mapint, int,
  gc_strategy maptext, int,
  gc_strategy_symbol maptext, int,
  gc_symbol maptext, int,
  ge_strategy maptext, int,
  ge_strategy_symbol maptext, int,
  ge_symbol maptext, int,
  go_strategy maptext, int,
  go_strategy_symbol maptext, int,
  go_symbol maptext, int,
  message_type maptext, int,
  PRIMARY KEY (time_order, time_start)
) WITH
  bloom_filter_fp_chance=0.01 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.00 AND
  gc_grace_seconds=864000 AND
  index_interval=128 AND
  read_repair_chance=0.10 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  default_time_to_live=0 AND
  speculative_retry='99.0PERCENTILE' AND
  memtable_flush_period_in_ms=0 AND
  compaction={'class': 'SizeTieredCompactionStrategy'} AND
  compression={};

Now I just insert data into it (using python driver, async insert, prepared 
insert statement). Each row only fills out one of the gc_*, go_*, or ge_* 
columns, and there's something like 20-100 entries per map column, occasionally 
1000, but it's nothing huge. 

First run 685 inserts in 1.004860 seconds (681.687053 Operations/s).
OK, not great, but that's fine.

Now throw 50,000 rows at it.

Now run the first run again, and it takes 53s to do the same insert of 685 rows 
- I'm getting about 10 rows per second. 

It's not IO bound - iostat 1 shows quiescent for 9 seconds, then ~640KB 
write, then sleeps again - seems like the fflush sync.

Run nodetool flush and performance goes back to as before

Not sure why this gets so slow - I think it just builds huge commit logs and 
memtables, but never writes out to the data/ directory with sstables because I 
only have one table? That doesn't seem like a good situation. 

Worse ... if you let the python driver just throw stuff at it async (I think 
this allows up to 128 request if I understand the underlying protocol, then it 
gets so slow that a single write takes over 10s and times out). Seems to be 
some sort of synchronization problem in Java ... if I limit the concurrent 
async requests to the left column below, I get the number of seconds elapsed on 
the right:

1: 103 seconds
2: 63 seconds
8: 53 seconds
16: 53 seconds
32: 66 seconds
64: so slow it explodes in timeouts on write (over 10s each).

I guess there's some thundering herd type locking issue in whatever Java 
primitive you are using to lock concurrent access to a single table. I know 
some of the Java concurrent.* stuff has this issue. So for the other tests 
above, I was limiting async writes to 16 pending.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7103) Very poor performance with simple setup

2014-04-28 Thread Martin Bligh (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983567#comment-13983567
 ] 

Martin Bligh commented on CASSANDRA-7103:
-

BTW, I'm aware that this is
(a) a single node
(b) half my time_order entries are zero (which I don't think matters as it's a 
single node anyway), so my partition key doesn't have much variance
(c) disk is not performant (but we're not even trying to write to it from 
iostat, so I don't think this matters).
(d) I'm writing to one table
(e) I'm using a single writer.

So I'm creating a hotspot of some form. But really, 

1. I think it should be able to handle more than 700 writes a second to one 
table.
2. It shouldn't degrade to about 10 writes per second.

Sure, I could throw masses of hardware at it, and make it scale a bit better, 
but ... unless it can perform better than this on a single table, single node, 
I don't see how it'd perform in any reasonable fashion on a larger cluster.


 Very poor performance with simple setup
 ---

 Key: CASSANDRA-7103
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7103
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Fedora 19 (also happens on Ubuntu), Cassandra 2.0.7. dsc 
 standard install
Reporter: Martin Bligh

 Single node (this is just development, 32GB 20 core server), single disk 
 array.
 Create the following table:
 CREATE TABLE reut (
   time_order bigint,
   time_start bigint,
   ack_us mapint, int,
   gc_strategy maptext, int,
   gc_strategy_symbol maptext, int,
   gc_symbol maptext, int,
   ge_strategy maptext, int,
   ge_strategy_symbol maptext, int,
   ge_symbol maptext, int,
   go_strategy maptext, int,
   go_strategy_symbol maptext, int,
   go_symbol maptext, int,
   message_type maptext, int,
   PRIMARY KEY (time_order, time_start)
 ) WITH
   bloom_filter_fp_chance=0.01 AND
   caching='KEYS_ONLY' AND
   comment='' AND
   dclocal_read_repair_chance=0.00 AND
   gc_grace_seconds=864000 AND
   index_interval=128 AND
   read_repair_chance=0.10 AND
   replicate_on_write='true' AND
   populate_io_cache_on_flush='false' AND
   default_time_to_live=0 AND
   speculative_retry='99.0PERCENTILE' AND
   memtable_flush_period_in_ms=0 AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={};
 Now I just insert data into it (using python driver, async insert, prepared 
 insert statement). Each row only fills out one of the gc_*, go_*, or ge_* 
 columns, and there's something like 20-100 entries per map column, 
 occasionally 1000, but it's nothing huge. 
 First run 685 inserts in 1.004860 seconds (681.687053 Operations/s).
 OK, not great, but that's fine.
 Now throw 50,000 rows at it.
 Now run the first run again, and it takes 53s to do the same insert of 685 
 rows - I'm getting about 10 rows per second. 
 It's not IO bound - iostat 1 shows quiescent for 9 seconds, then ~640KB 
 write, then sleeps again - seems like the fflush sync.
 Run nodetool flush and performance goes back to as before
 Not sure why this gets so slow - I think it just builds huge commit logs and 
 memtables, but never writes out to the data/ directory with sstables because 
 I only have one table? That doesn't seem like a good situation. 
 Worse ... if you let the python driver just throw stuff at it async (I think 
 this allows up to 128 request if I understand the underlying protocol, then 
 it gets so slow that a single write takes over 10s and times out). Seems to 
 be some sort of synchronization problem in Java ... if I limit the concurrent 
 async requests to the left column below, I get the number of seconds elapsed 
 on the right:
 1: 103 seconds
 2: 63 seconds
 8: 53 seconds
 16: 53 seconds
 32: 66 seconds
 64: so slow it explodes in timeouts on write (over 10s each).
 I guess there's some thundering herd type locking issue in whatever Java 
 primitive you are using to lock concurrent access to a single table. I know 
 some of the Java concurrent.* stuff has this issue. So for the other tests 
 above, I was limiting async writes to 16 pending.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7093) ConfigHelper.setInputColumnFamily incompatible with upper-cased keyspaces since 2.0.7

2014-04-28 Thread Alex Liu (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983606#comment-13983606
 ] 

Alex Liu commented on CASSANDRA-7093:
-

The attached patch quotes the keyspace name at Hadoop side. Since Driver 
invalidates JAVA_317, we have to fix it at C* Hadoop side.

 ConfigHelper.setInputColumnFamily incompatible with upper-cased keyspaces 
 since 2.0.7
 -

 Key: CASSANDRA-7093
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7093
 Project: Cassandra
  Issue Type: Bug
Reporter: Maxime Nay
Assignee: Alex Liu
 Attachments: 7093.txt


 Hi,
 We have a keyspace starting with an upper-case character: Visitors.
 We are trying to run a map reduce job on one of the column family of this 
 keyspace.
 To specify the keyspace it seems we have to use:
 org.apache.cassandra.hadoop.
 ConfigHelper.setInputColumnFamily(conf, keyspace, columnFamily);
 If we do:
 ConfigHelper.setInputColumnFamily(conf, Visitors, columnFamily); we get:
 com.datastax.driver.core.exceptions.InvalidQueryException: Keyspace 
 'visitors' does not exist
 at 
 com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:35)
 at 
 com.datastax.driver.core.DefaultResultSetFuture.extractCauseFromExecutionException(DefaultResultSetFuture.java:256)
 at 
 com.datastax.driver.core.SessionManager.setKeyspace(SessionManager.java:335)
 ...
 And if we do:
 ConfigHelper.setInputColumnFamily(conf, \Visitors\, columnFamily); we get:
 Exception in thread main java.lang.RuntimeException: 
 InvalidRequestException(why:No such keyspace: Visitors)
 at 
 org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getRangeMap(AbstractColumnFamilyInputFormat.java:339)
 at 
 org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:125)
 at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
 at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
 ...
 This is working just fine if the keyspace is lowercase.
 And it was working just fine with Cassandra 2.0.6. But with Cassandra 2.0.7, 
 and the addition of Datastax's java driver in the dependencies, I am getting 
 this error.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (CASSANDRA-7104) Unable to use cqlsh when client certificate authentication is enabled

2014-04-28 Thread Vishy Kasar (JIRA)
Vishy Kasar created CASSANDRA-7104:
--

 Summary: Unable to use cqlsh when client certificate 
authentication is enabled 
 Key: CASSANDRA-7104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7104
 Project: Cassandra
  Issue Type: New Feature
Reporter: Vishy Kasar


As per, 
http://www.datastax.com/documentation/cassandra/1.2/cassandra/security/secureCqlshSSL_t.html

You cannot use cqlsh when client certificate authentication is enabled 
(require_client_auth=true).

This seems like a major limitation if we can not use cqlsh when client auth is 
enabled.

Can this limitation be resolved?




--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-7100) Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster column is reversed

2014-04-28 Thread Alex Liu (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Liu updated CASSANDRA-7100:


Fix Version/s: (was: 1.2.9)

 Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster 
 column is reversed
 --

 Key: CASSANDRA-7100
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7100
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Rohit Rai
Assignee: Alex Liu

 To reproduce it,
 cqlsh:test  select * from wordfreq;
  title   | occurances | word
 -++---
  alex123 |  4 |  liu3
alex1 |  23456 |  liu2
   alex10 | 10 | liu10
   alex12 | 34 |  liu3
 alex | 123456 |  liu1
 alex |   1000 |   liu
 CREATE TABLE wordfreq ( title text, word text, occurances int, PRIMARY KEY 
 (title,occurances)) WITH CLUSTERING ORDER by (occurances DESC);
 The hadoop job returns 7 rows instead of 6 rows. 
 I will post a patch soon.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Assigned] (CASSANDRA-7104) Unable to use cqlsh when client certificate authentication is enabled

2014-04-28 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-7104:
---

Assignee: Mikhail Stepura

Hmm, is this really a problem or are the docs wrong?  ISTM if the client has 
the certs it should be fine.

 Unable to use cqlsh when client certificate authentication is enabled 
 --

 Key: CASSANDRA-7104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7104
 Project: Cassandra
  Issue Type: New Feature
Reporter: Vishy Kasar
Assignee: Mikhail Stepura

 As per, 
 http://www.datastax.com/documentation/cassandra/1.2/cassandra/security/secureCqlshSSL_t.html
 You cannot use cqlsh when client certificate authentication is enabled 
 (require_client_auth=true).
 This seems like a major limitation if we can not use cqlsh when client auth 
 is enabled.
 Can this limitation be resolved?



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7104) Unable to use cqlsh when client certificate authentication is enabled

2014-04-28 Thread Mikhail Stepura (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983623#comment-13983623
 ] 

Mikhail Stepura commented on CASSANDRA-7104:


Well, we support that in 2.1 (with new the new python driver). I'm not sure if 
it's possible to implement with cassandra-dbapi2.

 Unable to use cqlsh when client certificate authentication is enabled 
 --

 Key: CASSANDRA-7104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7104
 Project: Cassandra
  Issue Type: New Feature
Reporter: Vishy Kasar
Assignee: Mikhail Stepura

 As per, 
 http://www.datastax.com/documentation/cassandra/1.2/cassandra/security/secureCqlshSSL_t.html
 You cannot use cqlsh when client certificate authentication is enabled 
 (require_client_auth=true).
 This seems like a major limitation if we can not use cqlsh when client auth 
 is enabled.
 Can this limitation be resolved?



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7100) Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster column is reversed

2014-04-28 Thread Alex Liu (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983639#comment-13983639
 ] 

Alex Liu commented on CASSANDRA-7100:
-

CASSANDRA-6311 create CqlRecordReader to use native protocol paging. It 
automatically handles the reversed column query.

 Regression in 2.0.x - Cql3 reader returns duplicate rows if the cluster 
 column is reversed
 --

 Key: CASSANDRA-7100
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7100
 Project: Cassandra
  Issue Type: Bug
Affects Versions: 1.2.6
Reporter: Rohit Rai
Assignee: Alex Liu

 To reproduce it,
 cqlsh:test  select * from wordfreq;
  title   | occurances | word
 -++---
  alex123 |  4 |  liu3
alex1 |  23456 |  liu2
   alex10 | 10 | liu10
   alex12 | 34 |  liu3
 alex | 123456 |  liu1
 alex |   1000 |   liu
 CREATE TABLE wordfreq ( title text, word text, occurances int, PRIMARY KEY 
 (title,occurances)) WITH CLUSTERING ORDER by (occurances DESC);
 The hadoop job returns 7 rows instead of 6 rows. 
 I will post a patch soon.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-7103) Very poor performance with simple setup

2014-04-28 Thread Mikhail Stepura (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Stepura updated CASSANDRA-7103:
---

Description: 
Single node (this is just development, 32GB 20 core server), single disk array.

Create the following table:
{code}
CREATE TABLE reut (
  time_order bigint,
  time_start bigint,
  ack_us mapint, int,
  gc_strategy maptext, int,
  gc_strategy_symbol maptext, int,
  gc_symbol maptext, int,
  ge_strategy maptext, int,
  ge_strategy_symbol maptext, int,
  ge_symbol maptext, int,
  go_strategy maptext, int,
  go_strategy_symbol maptext, int,
  go_symbol maptext, int,
  message_type maptext, int,
  PRIMARY KEY (time_order, time_start)
) WITH
  bloom_filter_fp_chance=0.01 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.00 AND
  gc_grace_seconds=864000 AND
  index_interval=128 AND
  read_repair_chance=0.10 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  default_time_to_live=0 AND
  speculative_retry='99.0PERCENTILE' AND
  memtable_flush_period_in_ms=0 AND
  compaction={'class': 'SizeTieredCompactionStrategy'} AND
  compression={};
{code}
Now I just insert data into it (using python driver, async insert, prepared 
insert statement). Each row only fills out one of the gc_*, go_*, or ge_* 
columns, and there's something like 20-100 entries per map column, occasionally 
1000, but it's nothing huge. 

First run 685 inserts in 1.004860 seconds (681.687053 Operations/s).
OK, not great, but that's fine.

Now throw 50,000 rows at it.

Now run the first run again, and it takes 53s to do the same insert of 685 rows 
- I'm getting about 10 rows per second. 

It's not IO bound - iostat 1 shows quiescent for 9 seconds, then ~640KB 
write, then sleeps again - seems like the fflush sync.

Run nodetool flush and performance goes back to as before

Not sure why this gets so slow - I think it just builds huge commit logs and 
memtables, but never writes out to the data/ directory with sstables because I 
only have one table? That doesn't seem like a good situation. 

Worse ... if you let the python driver just throw stuff at it async (I think 
this allows up to 128 request if I understand the underlying protocol, then it 
gets so slow that a single write takes over 10s and times out). Seems to be 
some sort of synchronization problem in Java ... if I limit the concurrent 
async requests to the left column below, I get the number of seconds elapsed on 
the right:

1: 103 seconds
2: 63 seconds
8: 53 seconds
16: 53 seconds
32: 66 seconds
64: so slow it explodes in timeouts on write (over 10s each).

I guess there's some thundering herd type locking issue in whatever Java 
primitive you are using to lock concurrent access to a single table. I know 
some of the Java concurrent.* stuff has this issue. So for the other tests 
above, I was limiting async writes to 16 pending.

  was:
Single node (this is just development, 32GB 20 core server), single disk array.

Create the following table:

CREATE TABLE reut (
  time_order bigint,
  time_start bigint,
  ack_us mapint, int,
  gc_strategy maptext, int,
  gc_strategy_symbol maptext, int,
  gc_symbol maptext, int,
  ge_strategy maptext, int,
  ge_strategy_symbol maptext, int,
  ge_symbol maptext, int,
  go_strategy maptext, int,
  go_strategy_symbol maptext, int,
  go_symbol maptext, int,
  message_type maptext, int,
  PRIMARY KEY (time_order, time_start)
) WITH
  bloom_filter_fp_chance=0.01 AND
  caching='KEYS_ONLY' AND
  comment='' AND
  dclocal_read_repair_chance=0.00 AND
  gc_grace_seconds=864000 AND
  index_interval=128 AND
  read_repair_chance=0.10 AND
  replicate_on_write='true' AND
  populate_io_cache_on_flush='false' AND
  default_time_to_live=0 AND
  speculative_retry='99.0PERCENTILE' AND
  memtable_flush_period_in_ms=0 AND
  compaction={'class': 'SizeTieredCompactionStrategy'} AND
  compression={};

Now I just insert data into it (using python driver, async insert, prepared 
insert statement). Each row only fills out one of the gc_*, go_*, or ge_* 
columns, and there's something like 20-100 entries per map column, occasionally 
1000, but it's nothing huge. 

First run 685 inserts in 1.004860 seconds (681.687053 Operations/s).
OK, not great, but that's fine.

Now throw 50,000 rows at it.

Now run the first run again, and it takes 53s to do the same insert of 685 rows 
- I'm getting about 10 rows per second. 

It's not IO bound - iostat 1 shows quiescent for 9 seconds, then ~640KB 
write, then sleeps again - seems like the fflush sync.

Run nodetool flush and performance goes back to as before

Not sure why this gets so slow - I think it just builds huge commit logs and 
memtables, but never writes out to the data/ directory with sstables because I 
only have one table? That doesn't seem like a good situation. 

Worse ... if you let the python driver just throw 

[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Aleksey Yeschenko (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983666#comment-13983666
 ] 

Aleksey Yeschenko commented on CASSANDRA-6694:
--

bq. This was a conscious decision to permit custom 2i use our allocators and 
count towards book keeping for memory utilisation.

I feel like you are lacking context here, wrt custom 2i actually are and what 
implementations we have. The ones that exist don't need it, so IMO this was a 
wrong decision, even if conscious. Have a look at DSE's Solr implementation if 
curious.

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Benedict (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983670#comment-13983670
 ] 

Benedict commented on CASSANDRA-6694:
-

I don't mind dropping it, but it seems a harmless addition for users who 
implement their own buffered writes for secondary indexes, so that they can 
consider the amount of data they are using for their own state when deciding 
which CFs to flush. The fact that DSE doesn't do this doesn't mean it isn't 
useful.

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Aleksey Yeschenko (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983675#comment-13983675
 ] 

Aleksey Yeschenko commented on CASSANDRA-6694:
--

Sure, but I prefer to add stuff when it's clear that there is someone who 
actually needs it, and not some hypothetical user that doesn't exist.

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (CASSANDRA-7104) Unable to use cqlsh when client certificate authentication is enabled

2014-04-28 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-7104.
-

Resolution: Not a Problem

Well, dbapi2 is officially dead and since we have this in 2.1 there's not much 
to do.

 Unable to use cqlsh when client certificate authentication is enabled 
 --

 Key: CASSANDRA-7104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7104
 Project: Cassandra
  Issue Type: New Feature
Reporter: Vishy Kasar
Assignee: Mikhail Stepura

 As per, 
 http://www.datastax.com/documentation/cassandra/1.2/cassandra/security/secureCqlshSSL_t.html
 You cannot use cqlsh when client certificate authentication is enabled 
 (require_client_auth=true).
 This seems like a major limitation if we can not use cqlsh when client auth 
 is enabled.
 Can this limitation be resolved?



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7104) Unable to use cqlsh when client certificate authentication is enabled

2014-04-28 Thread Mikhail Stepura (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983682#comment-13983682
 ] 

Mikhail Stepura commented on CASSANDRA-7104:


For the record: https://issues.apache.org/jira/browse/THRIFT-1867

 Unable to use cqlsh when client certificate authentication is enabled 
 --

 Key: CASSANDRA-7104
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7104
 Project: Cassandra
  Issue Type: New Feature
Reporter: Vishy Kasar
Assignee: Mikhail Stepura

 As per, 
 http://www.datastax.com/documentation/cassandra/1.2/cassandra/security/secureCqlshSSL_t.html
 You cannot use cqlsh when client certificate authentication is enabled 
 (require_client_auth=true).
 This seems like a major limitation if we can not use cqlsh when client auth 
 is enabled.
 Can this limitation be resolved?



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Resolved] (CASSANDRA-7103) Very poor performance with simple setup

2014-04-28 Thread Benedict (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict resolved CASSANDRA-7103.
-

Resolution: Invalid

[~Martin Bligh] it's helpful if you can take up these sorts of issues on the 
user list first to see what can be done to address the issues you're having 
before resorting to filing a ticket. It's likely that CASSANDRA-5417 (in the 
upcoming 2.1 release) would help dramatically with this, however you can also 
batch updates (with unlogged batches) to a single cql row - that is, all of 
those map inserts to the same (time_order, time_start) pairs; performing 
asynchronous writes to the same (large) cql row is a recipe for many wasted cpu 
cycles, as each row is copy-on-write updated, so with 16 updating the same row, 
15 are simply burning cycles at any given moment. There may be other aspects to 
consider as well.



 Very poor performance with simple setup
 ---

 Key: CASSANDRA-7103
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7103
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Fedora 19 (also happens on Ubuntu), Cassandra 2.0.7. dsc 
 standard install
Reporter: Martin Bligh

 Single node (this is just development, 32GB 20 core server), single disk 
 array.
 Create the following table:
 {code}
 CREATE TABLE reut (
   time_order bigint,
   time_start bigint,
   ack_us mapint, int,
   gc_strategy maptext, int,
   gc_strategy_symbol maptext, int,
   gc_symbol maptext, int,
   ge_strategy maptext, int,
   ge_strategy_symbol maptext, int,
   ge_symbol maptext, int,
   go_strategy maptext, int,
   go_strategy_symbol maptext, int,
   go_symbol maptext, int,
   message_type maptext, int,
   PRIMARY KEY (time_order, time_start)
 ) WITH
   bloom_filter_fp_chance=0.01 AND
   caching='KEYS_ONLY' AND
   comment='' AND
   dclocal_read_repair_chance=0.00 AND
   gc_grace_seconds=864000 AND
   index_interval=128 AND
   read_repair_chance=0.10 AND
   replicate_on_write='true' AND
   populate_io_cache_on_flush='false' AND
   default_time_to_live=0 AND
   speculative_retry='99.0PERCENTILE' AND
   memtable_flush_period_in_ms=0 AND
   compaction={'class': 'SizeTieredCompactionStrategy'} AND
   compression={};
 {code}
 Now I just insert data into it (using python driver, async insert, prepared 
 insert statement). Each row only fills out one of the gc_*, go_*, or ge_* 
 columns, and there's something like 20-100 entries per map column, 
 occasionally 1000, but it's nothing huge. 
 First run 685 inserts in 1.004860 seconds (681.687053 Operations/s).
 OK, not great, but that's fine.
 Now throw 50,000 rows at it.
 Now run the first run again, and it takes 53s to do the same insert of 685 
 rows - I'm getting about 10 rows per second. 
 It's not IO bound - iostat 1 shows quiescent for 9 seconds, then ~640KB 
 write, then sleeps again - seems like the fflush sync.
 Run nodetool flush and performance goes back to as before
 Not sure why this gets so slow - I think it just builds huge commit logs and 
 memtables, but never writes out to the data/ directory with sstables because 
 I only have one table? That doesn't seem like a good situation. 
 Worse ... if you let the python driver just throw stuff at it async (I think 
 this allows up to 128 request if I understand the underlying protocol, then 
 it gets so slow that a single write takes over 10s and times out). Seems to 
 be some sort of synchronization problem in Java ... if I limit the concurrent 
 async requests to the left column below, I get the number of seconds elapsed 
 on the right:
 1: 103 seconds
 2: 63 seconds
 8: 53 seconds
 16: 53 seconds
 32: 66 seconds
 64: so slow it explodes in timeouts on write (over 10s each).
 I guess there's some thundering herd type locking issue in whatever Java 
 primitive you are using to lock concurrent access to a single table. I know 
 some of the Java concurrent.* stuff has this issue. So for the other tests 
 above, I was limiting async writes to 16 pending.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (CASSANDRA-7105) SELECT with IN on final column of composite and compound primary key fails

2014-04-28 Thread Bill Mitchell (JIRA)
Bill Mitchell created CASSANDRA-7105:


 Summary: SELECT with IN on final column of composite and compound 
primary key fails
 Key: CASSANDRA-7105
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7105
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: DataStax Cassandra 2.0.7
Windows dual-core laptop
Reporter: Bill Mitchell


I have a failing sequence where I specify an IN constraint on the final int 
column of the composite primary key and an IN constraint on the final String 
column of the compound primary key and no rows are returned, when rows should 
be returned.  
{noformat}
CREATE TABLE IF NOT EXISTS sr2 (siteID TEXT, partition INT, listID BIGINT, 
emailAddr TEXT, emailCrypt TEXT, createDate TIMESTAMP, removeDate TIMESTAMP, 
removeImportID BIGINT, properties TEXT, PRIMARY KEY ((siteID, listID, 
partition), createDate, emailCrypt) ) WITH CLUSTERING ORDER BY (createDate 
DESC, emailCrypt DESC)  AND compression = {'sstable_compression' : 
'SnappyCompressor'} AND compaction = {'class' : 'SizeTieredCompactionStrategy'};
insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, createDate) 
values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 1, 'xyzzy', 
'5fe7719229092cdde4526afbc65c900c', '2014-04-28T14:05:59.236-0500');
insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, createDate) 
values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 2, 'noname', 
'97bf28af2ca9c498d6e47237bb8680bf', '2014-04-28T14:05:59.236-0500');
select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
'97bf28af2ca9c498d6e47237bb8680bf';

 emailcrypt   | emailaddr
--+---
 97bf28af2ca9c498d6e47237bb8680bf |noname

(1 rows)

select emailCrypt, emailAddr  from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
'5fe7719229092cdde4526afbc65c900c';


 emailcrypt   | emailaddr
--+---
 5fe7719229092cdde4526afbc65c900c | xyzzy

(1 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

cqlsh:test_multiple_in select * from sr2;

 siteid   | listid | partition | createdate 
  | emailcrypt | emailaddr| 
properties | removedate | re
moveimportid
--++---+--++--+++---
-
 4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 2 | 2014-04-28 
14:05:59Central Daylight Time | noname | 97bf28af2ca9c498d6e47237bb8680bf | 
  null |   null |
null
 4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 1 | 2014-04-28 
14:05:59Central Daylight Time |  xyzzy | 5fe7719229092cdde4526afbc65c900c | 
  null |   null |
null

(2 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

cqlsh:test_multiple_in select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf');

 emailcrypt   | emailaddr
--+---
 

[jira] [Updated] (CASSANDRA-7105) SELECT with IN on final column of composite and compound primary key fails

2014-04-28 Thread Bill Mitchell (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bill Mitchell updated CASSANDRA-7105:
-

Description: 
I have a failing sequence where I specify an IN constraint on the final int 
column of the composite primary key and an IN constraint on the final String 
column of the compound primary key and no rows are returned, when rows should 
be returned.  
{noformat}
CREATE TABLE IF NOT EXISTS sr2 (siteID TEXT, partition INT, listID BIGINT, 
emailAddr TEXT, emailCrypt TEXT, createDate TIMESTAMP, removeDate TIMESTAMP, 
removeImportID BIGINT, properties TEXT, PRIMARY KEY ((siteID, listID, 
partition), createDate, emailCrypt) ) WITH CLUSTERING ORDER BY (createDate 
DESC, emailCrypt DESC)  AND compression = {'sstable_compression' : 
'SnappyCompressor'} AND compaction = {'class' : 'SizeTieredCompactionStrategy'};
insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, createDate) 
values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 1, 'xyzzy', 
'5fe7719229092cdde4526afbc65c900c', '2014-04-28T14:05:59.236-0500');
insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, createDate) 
values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 2, 'noname', 
'97bf28af2ca9c498d6e47237bb8680bf', '2014-04-28T14:05:59.236-0500');
select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
'97bf28af2ca9c498d6e47237bb8680bf';

 emailcrypt   | emailaddr
--+---
 97bf28af2ca9c498d6e47237bb8680bf |noname

(1 rows)

select emailCrypt, emailAddr  from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
'5fe7719229092cdde4526afbc65c900c';


 emailcrypt   | emailaddr
--+---
 5fe7719229092cdde4526afbc65c900c | xyzzy

(1 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

cqlsh:test_multiple_in select * from sr2;

 siteid   | listid | partition | createdate 
  | emailcrypt | emailaddr| 
properties | removedate | re
moveimportid
--++---+--++--+++---
-
 4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 2 | 2014-04-28 
14:05:59Central Daylight Time | noname | 97bf28af2ca9c498d6e47237bb8680bf | 
  null |   null |
null
 4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 1 | 2014-04-28 
14:05:59Central Daylight Time |  xyzzy | 5fe7719229092cdde4526afbc65c900c | 
  null |   null |
null

(2 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');

(0 rows)

cqlsh:test_multiple_in select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
('97bf28af2ca9c498d6e47237bb8680bf');

 emailcrypt   | emailaddr
--+---
 97bf28af2ca9c498d6e47237bb8680bf |noname

(1 rows)

cqlsh:test_multiple_in select emailCrypt, emailAddr from sr2 where siteID = 
'4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
and createDate = '2014-04-28T14:05:59.236-0500' and 

[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Benedict (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983708#comment-13983708
 ] 

Benedict commented on CASSANDRA-6694:
-

To make shipping easier, I've pushed a rebased and squashed branch 
[here|https://github.com/belliottsmith/cassandra/tree/6694-reorg2-rebase]

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983709#comment-13983709
 ] 

Pavel Yaskevich commented on CASSANDRA-6694:


Here is a [new branch|https://github.com/xedin/cassandra/compare/6694-final] 
with all of the changes from 6694-reorg2 squashed and added a couple of commits 
to cleanup and remove secondary index getAllocator which is unnecessary right 
now. I was about to push similar refactoring for memtable pools to my branch, 
which made review much faster :)

I'm +1 on combination of the squashed changes and cleanup which is in my 
branch, still not sure about CellName implementation in AbstractNativeCell tho, 
that is not my realm, so it would be nice if Sylvain (or somebody as close to 
that code, if anybody) could take a look...

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Created] (CASSANDRA-7106) Retry multiple times before giving up on fetching splits

2014-04-28 Thread Victor Igumnov (JIRA)
Victor Igumnov created CASSANDRA-7106:
-

 Summary: Retry multiple times before giving up on fetching splits
 Key: CASSANDRA-7106
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7106
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Victor Igumnov
Priority: Minor
 Fix For: 2.1 beta1
 Attachments: retry-splits.patch

See Github for patch:

https://github.com/apache/cassandra/pull/34
https://github.com/victori/cassandra

What the patch does:

While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
get any of the queries to return reliably. Looking further into the issue I 
noticed that if just a single fetch for a split fails the whole job is aborted. 
So instead of giving up and failing out the whole job, retry fetching the data 
a few times before completely giving up.  




--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Comment Edited] (CASSANDRA-6694) Slightly More Off-Heap Memtables

2014-04-28 Thread Pavel Yaskevich (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983709#comment-13983709
 ] 

Pavel Yaskevich edited comment on CASSANDRA-6694 at 4/28/14 11:29 PM:
--

Here is a [new branch|https://github.com/xedin/cassandra/compare/6694-final] 
with all of the changes from 6694-reorg2 squashed and added a couple of commits 
to cleanup and remove secondary index getAllocator which is unnecessary right 
now. I was about to push similar refactoring for memtable pools to my branch, 
which made review much faster :)

I'm +1 on combination of the squashed changes and cleanup which is in my 
branch, still not sure about CellName implementation in AbstractNativeCell tho, 
that is not my realm, so it would be nice if [~slebresne] (or somebody as close 
to that code, if anybody) could take a look...


was (Author: xedin):
Here is a [new branch|https://github.com/xedin/cassandra/compare/6694-final] 
with all of the changes from 6694-reorg2 squashed and added a couple of commits 
to cleanup and remove secondary index getAllocator which is unnecessary right 
now. I was about to push similar refactoring for memtable pools to my branch, 
which made review much faster :)

I'm +1 on combination of the squashed changes and cleanup which is in my 
branch, still not sure about CellName implementation in AbstractNativeCell tho, 
that is not my realm, so it would be nice if Sylvain (or somebody as close to 
that code, if anybody) could take a look...

 Slightly More Off-Heap Memtables
 

 Key: CASSANDRA-6694
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6694
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 2.1 beta2


 The Off Heap memtables introduced in CASSANDRA-6689 don't go far enough, as 
 the on-heap overhead is still very large. It should not be tremendously 
 difficult to extend these changes so that we allocate entire Cells off-heap, 
 instead of multiple BBs per Cell (with all their associated overhead).
 The goal (if possible) is to reach an overhead of 16-bytes per Cell (plus 4-6 
 bytes per cell on average for the btree overhead, for a total overhead of 
 around 20-22 bytes). This translates to 8-byte object overhead, 4-byte 
 address (we will do alignment tricks like the VM to allow us to address a 
 reasonably large memory space, although this trick is unlikely to last us 
 forever, at which point we will have to bite the bullet and accept a 24-byte 
 per cell overhead), and 4-byte object reference for maintaining our internal 
 list of allocations, which is unfortunately necessary since we cannot safely 
 (and cheaply) walk the object graph we allocate otherwise, which is necessary 
 for (allocation-) compaction and pointer rewriting.
 The ugliest thing here is going to be implementing the various CellName 
 instances so that they may be backed by native memory OR heap memory.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


git commit: fix npe when specifying -Dcassandra.fd_initial_value_ms

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-1.2 63ba5886e - 7f019804c


fix npe when specifying -Dcassandra.fd_initial_value_ms

patch by dbrosius reviewed by bwilliams for cassandra 6751


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7f019804
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7f019804
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7f019804

Branch: refs/heads/cassandra-1.2
Commit: 7f019804ca82c727062c2a787fe534690e9dbf6d
Parents: 63ba588
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:51:56 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:51:56 2014 -0400

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index aad3721..e8d6a8d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,7 @@
  * Ensure that batchlog and hint timeouts do not produce hints (CASSANDRA-7058)
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
+ * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/src/java/org/apache/cassandra/gms/FailureDetector.java
--
diff --git a/src/java/org/apache/cassandra/gms/FailureDetector.java 
b/src/java/org/apache/cassandra/gms/FailureDetector.java
index 7a35c34..f5c500c 100644
--- a/src/java/org/apache/cassandra/gms/FailureDetector.java
+++ b/src/java/org/apache/cassandra/gms/FailureDetector.java
@@ -43,12 +43,12 @@ import org.apache.cassandra.utils.FBUtilities;
  */
 public class FailureDetector implements IFailureDetector, FailureDetectorMBean
 {
+private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 public static final String MBEAN_NAME = 
org.apache.cassandra.net:type=FailureDetector;
 private static final int SAMPLE_SIZE = 1000;
 protected static final int INITIAL_VALUE = getInitialValue();
 
 public static final IFailureDetector instance = new FailureDetector();
-private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 
 // this is useless except to provide backwards compatibility in 
phi_convict_threshold,
 // because everyone seems pretty accustomed to the default of 8, and users 
who have



[2/2] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

2014-04-28 Thread dbrosius
Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2b322937
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2b322937
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2b322937

Branch: refs/heads/cassandra-2.0
Commit: 2b3229370bb5f7d0e5271fd5964fc6f374874756
Parents: 9002e76 7f01980
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:56:56 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:56:56 2014 -0400

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b322937/CHANGES.txt
--
diff --cc CHANGES.txt
index 376ad87,e8d6a8d..50d5896
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -21,63 -11,12 +21,64 @@@ Merged from 1.2
   * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Require nodetool rebuild_index to specify index names (CASSANDRA-7038)
   * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
+  * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
  
  
 -1.2.16
 +2.0.7
 + * Put nodes in hibernate when join_ring is false (CASSANDRA-6961)
 + * Avoid early loading of non-system keyspaces before compaction-leftovers 
 +   cleanup at startup (CASSANDRA-6913)
 + * Restrict Windows to parallel repairs (CASSANDRA-6907)
 + * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
 + * Fix NPE in MeteredFlusher (CASSANDRA-6820)
 + * Fix race processing range scan responses (CASSANDRA-6820)
 + * Allow deleting snapshots from dropped keyspaces (CASSANDRA-6821)
 + * Add uuid() function (CASSANDRA-6473)
 + * Omit tombstones from schema digests (CASSANDRA-6862)
 + * Include correct consistencyLevel in LWT timeout (CASSANDRA-6884)
 + * Lower chances for losing new SSTables during nodetool refresh and
 +   ColumnFamilyStore.loadNewSSTables (CASSANDRA-6514)
 + * Add support for DELETE ... IF EXISTS to CQL3 (CASSANDRA-5708)
 + * Update hadoop_cql3_word_count example (CASSANDRA-6793)
 + * Fix handling of RejectedExecution in sync Thrift server (CASSANDRA-6788)
 + * Log more information when exceeding tombstone_warn_threshold 
(CASSANDRA-6865)
 + * Fix truncate to not abort due to unreachable fat clients (CASSANDRA-6864)
 + * Fix schema concurrency exceptions (CASSANDRA-6841)
 + * Fix leaking validator FH in StreamWriter (CASSANDRA-6832)
 + * Fix saving triggers to schema (CASSANDRA-6789)
 + * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
 + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
 + * Fix static counter columns (CASSANDRA-6827)
 + * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
 + * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
 + * Correctly compare BooleanType values other than 0 and 1 (CASSANDRA-6779)
 + * Read message id as string from earlier versions (CASSANDRA-6840)
 + * Properly use the Paxos consistency for (non-protocol) batch 
(CASSANDRA-6837)
 + * Add paranoid disk failure option (CASSANDRA-6646)
 + * Improve PerRowSecondaryIndex performance (CASSANDRA-6876)
 + * Extend triggers to support CAS updates (CASSANDRA-6882)
 + * Static columns with IF NOT EXISTS don't always work as expected 
(CASSANDRA-6873)
 + * Fix paging with SELECT DISTINCT (CASSANDRA-6857)
 + * Fix UnsupportedOperationException on CAS timeout (CASSANDRA-6923)
 + * Improve MeteredFlusher handling of MF-unaffected column families
 +   (CASSANDRA-6867)
 + * Add CqlRecordReader using native pagination (CASSANDRA-6311)
 + * Add QueryHandler interface (CASSANDRA-6659)
 + * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
 + * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
 + * Fix LIMIT with static columns (CASSANDRA-6956)
 + * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 + * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 + * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
 + * Fix unintended update with conditional statement (CASSANDRA-6893)
 + * Fix map element access in IF (CASSANDRA-6914)
 + * Avoid costly range calculations for range queries on system keyspaces
 +   (CASSANDRA-6906)
 + * Fix SSTable not released if stream session fails (CASSANDRA-6818)
 + * 

[1/2] git commit: fix npe when specifying -Dcassandra.fd_initial_value_ms

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 9002e76d6 - 2b3229370


fix npe when specifying -Dcassandra.fd_initial_value_ms

patch by dbrosius reviewed by bwilliams for cassandra 6751


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7f019804
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7f019804
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7f019804

Branch: refs/heads/cassandra-2.0
Commit: 7f019804ca82c727062c2a787fe534690e9dbf6d
Parents: 63ba588
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:51:56 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:51:56 2014 -0400

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index aad3721..e8d6a8d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,7 @@
  * Ensure that batchlog and hint timeouts do not produce hints (CASSANDRA-7058)
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
+ * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/src/java/org/apache/cassandra/gms/FailureDetector.java
--
diff --git a/src/java/org/apache/cassandra/gms/FailureDetector.java 
b/src/java/org/apache/cassandra/gms/FailureDetector.java
index 7a35c34..f5c500c 100644
--- a/src/java/org/apache/cassandra/gms/FailureDetector.java
+++ b/src/java/org/apache/cassandra/gms/FailureDetector.java
@@ -43,12 +43,12 @@ import org.apache.cassandra.utils.FBUtilities;
  */
 public class FailureDetector implements IFailureDetector, FailureDetectorMBean
 {
+private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 public static final String MBEAN_NAME = 
org.apache.cassandra.net:type=FailureDetector;
 private static final int SAMPLE_SIZE = 1000;
 protected static final int INITIAL_VALUE = getInitialValue();
 
 public static final IFailureDetector instance = new FailureDetector();
-private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 
 // this is useless except to provide backwards compatibility in 
phi_convict_threshold,
 // because everyone seems pretty accustomed to the default of 8, and users 
who have



[1/3] git commit: fix npe when specifying -Dcassandra.fd_initial_value_ms

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 9a0f25890 - 7316bb3a6


fix npe when specifying -Dcassandra.fd_initial_value_ms

patch by dbrosius reviewed by bwilliams for cassandra 6751


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7f019804
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7f019804
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7f019804

Branch: refs/heads/cassandra-2.1
Commit: 7f019804ca82c727062c2a787fe534690e9dbf6d
Parents: 63ba588
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:51:56 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:51:56 2014 -0400

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index aad3721..e8d6a8d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,7 @@
  * Ensure that batchlog and hint timeouts do not produce hints (CASSANDRA-7058)
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
+ * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/src/java/org/apache/cassandra/gms/FailureDetector.java
--
diff --git a/src/java/org/apache/cassandra/gms/FailureDetector.java 
b/src/java/org/apache/cassandra/gms/FailureDetector.java
index 7a35c34..f5c500c 100644
--- a/src/java/org/apache/cassandra/gms/FailureDetector.java
+++ b/src/java/org/apache/cassandra/gms/FailureDetector.java
@@ -43,12 +43,12 @@ import org.apache.cassandra.utils.FBUtilities;
  */
 public class FailureDetector implements IFailureDetector, FailureDetectorMBean
 {
+private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 public static final String MBEAN_NAME = 
org.apache.cassandra.net:type=FailureDetector;
 private static final int SAMPLE_SIZE = 1000;
 protected static final int INITIAL_VALUE = getInitialValue();
 
 public static final IFailureDetector instance = new FailureDetector();
-private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 
 // this is useless except to provide backwards compatibility in 
phi_convict_threshold,
 // because everyone seems pretty accustomed to the default of 8, and users 
who have



[2/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

2014-04-28 Thread dbrosius
Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2b322937
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2b322937
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2b322937

Branch: refs/heads/cassandra-2.1
Commit: 2b3229370bb5f7d0e5271fd5964fc6f374874756
Parents: 9002e76 7f01980
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:56:56 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:56:56 2014 -0400

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b322937/CHANGES.txt
--
diff --cc CHANGES.txt
index 376ad87,e8d6a8d..50d5896
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -21,63 -11,12 +21,64 @@@ Merged from 1.2
   * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Require nodetool rebuild_index to specify index names (CASSANDRA-7038)
   * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
+  * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
  
  
 -1.2.16
 +2.0.7
 + * Put nodes in hibernate when join_ring is false (CASSANDRA-6961)
 + * Avoid early loading of non-system keyspaces before compaction-leftovers 
 +   cleanup at startup (CASSANDRA-6913)
 + * Restrict Windows to parallel repairs (CASSANDRA-6907)
 + * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
 + * Fix NPE in MeteredFlusher (CASSANDRA-6820)
 + * Fix race processing range scan responses (CASSANDRA-6820)
 + * Allow deleting snapshots from dropped keyspaces (CASSANDRA-6821)
 + * Add uuid() function (CASSANDRA-6473)
 + * Omit tombstones from schema digests (CASSANDRA-6862)
 + * Include correct consistencyLevel in LWT timeout (CASSANDRA-6884)
 + * Lower chances for losing new SSTables during nodetool refresh and
 +   ColumnFamilyStore.loadNewSSTables (CASSANDRA-6514)
 + * Add support for DELETE ... IF EXISTS to CQL3 (CASSANDRA-5708)
 + * Update hadoop_cql3_word_count example (CASSANDRA-6793)
 + * Fix handling of RejectedExecution in sync Thrift server (CASSANDRA-6788)
 + * Log more information when exceeding tombstone_warn_threshold 
(CASSANDRA-6865)
 + * Fix truncate to not abort due to unreachable fat clients (CASSANDRA-6864)
 + * Fix schema concurrency exceptions (CASSANDRA-6841)
 + * Fix leaking validator FH in StreamWriter (CASSANDRA-6832)
 + * Fix saving triggers to schema (CASSANDRA-6789)
 + * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
 + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
 + * Fix static counter columns (CASSANDRA-6827)
 + * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
 + * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
 + * Correctly compare BooleanType values other than 0 and 1 (CASSANDRA-6779)
 + * Read message id as string from earlier versions (CASSANDRA-6840)
 + * Properly use the Paxos consistency for (non-protocol) batch 
(CASSANDRA-6837)
 + * Add paranoid disk failure option (CASSANDRA-6646)
 + * Improve PerRowSecondaryIndex performance (CASSANDRA-6876)
 + * Extend triggers to support CAS updates (CASSANDRA-6882)
 + * Static columns with IF NOT EXISTS don't always work as expected 
(CASSANDRA-6873)
 + * Fix paging with SELECT DISTINCT (CASSANDRA-6857)
 + * Fix UnsupportedOperationException on CAS timeout (CASSANDRA-6923)
 + * Improve MeteredFlusher handling of MF-unaffected column families
 +   (CASSANDRA-6867)
 + * Add CqlRecordReader using native pagination (CASSANDRA-6311)
 + * Add QueryHandler interface (CASSANDRA-6659)
 + * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
 + * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
 + * Fix LIMIT with static columns (CASSANDRA-6956)
 + * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 + * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 + * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
 + * Fix unintended update with conditional statement (CASSANDRA-6893)
 + * Fix map element access in IF (CASSANDRA-6914)
 + * Avoid costly range calculations for range queries on system keyspaces
 +   (CASSANDRA-6906)
 + * Fix SSTable not released if stream session fails (CASSANDRA-6818)
 + * 

[3/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread dbrosius
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7316bb3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7316bb3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7316bb3a

Branch: refs/heads/cassandra-2.1
Commit: 7316bb3a65ca0e7e543bf16fe2c69f7d88dc7b31
Parents: 9a0f258 2b32293
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:59:53 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:59:53 2014 -0400

--
 CHANGES.txt| 2 +-
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7316bb3a/CHANGES.txt
--
diff --cc CHANGES.txt
index 1b67357,50d5896..de564e9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -126,12 -88,8 +126,12 @@@ Merged from 1.2
   * Schedule schema pulls on change (CASSANDRA-6971)
   * Non-droppable verbs shouldn't be dropped from OTC (CASSANDRA-6980)
   * Shutdown batchlog executor in SS#drain() (CASSANDRA-7025)
 + * Fix batchlog to account for CF truncation records (CASSANDRA-6999)
 + * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Properly load trustore in the native protocol (CASSANDRA-6847)
 -
 + * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
- 
++ * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
  
  2.0.6
   * Avoid race-prone second scrub of system keyspace (CASSANDRA-6797)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7316bb3a/src/java/org/apache/cassandra/gms/FailureDetector.java
--



[1/4] git commit: fix npe when specifying -Dcassandra.fd_initial_value_ms

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/trunk 2e96779db - 00b6a0372


fix npe when specifying -Dcassandra.fd_initial_value_ms

patch by dbrosius reviewed by bwilliams for cassandra 6751


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7f019804
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7f019804
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7f019804

Branch: refs/heads/trunk
Commit: 7f019804ca82c727062c2a787fe534690e9dbf6d
Parents: 63ba588
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:51:56 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:51:56 2014 -0400

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index aad3721..e8d6a8d 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -13,6 +13,7 @@
  * Ensure that batchlog and hint timeouts do not produce hints (CASSANDRA-7058)
  * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
  * Always clean up references in SerializingCache (CASSANDRA-6994)
+ * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
 
 
 1.2.16

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7f019804/src/java/org/apache/cassandra/gms/FailureDetector.java
--
diff --git a/src/java/org/apache/cassandra/gms/FailureDetector.java 
b/src/java/org/apache/cassandra/gms/FailureDetector.java
index 7a35c34..f5c500c 100644
--- a/src/java/org/apache/cassandra/gms/FailureDetector.java
+++ b/src/java/org/apache/cassandra/gms/FailureDetector.java
@@ -43,12 +43,12 @@ import org.apache.cassandra.utils.FBUtilities;
  */
 public class FailureDetector implements IFailureDetector, FailureDetectorMBean
 {
+private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 public static final String MBEAN_NAME = 
org.apache.cassandra.net:type=FailureDetector;
 private static final int SAMPLE_SIZE = 1000;
 protected static final int INITIAL_VALUE = getInitialValue();
 
 public static final IFailureDetector instance = new FailureDetector();
-private static final Logger logger = 
LoggerFactory.getLogger(FailureDetector.class);
 
 // this is useless except to provide backwards compatibility in 
phi_convict_threshold,
 // because everyone seems pretty accustomed to the default of 8, and users 
who have



[4/4] git commit: Merge branch 'cassandra-2.1' into trunk

2014-04-28 Thread dbrosius
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00b6a037
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00b6a037
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00b6a037

Branch: refs/heads/trunk
Commit: 00b6a037293bb930f43180fb9e4fc895d70923ed
Parents: 2e96779 7316bb3
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 20:00:51 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 20:00:51 2014 -0400

--
 CHANGES.txt| 2 +-
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00b6a037/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00b6a037/src/java/org/apache/cassandra/gms/FailureDetector.java
--



[3/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread dbrosius
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7316bb3a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7316bb3a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7316bb3a

Branch: refs/heads/trunk
Commit: 7316bb3a65ca0e7e543bf16fe2c69f7d88dc7b31
Parents: 9a0f258 2b32293
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:59:53 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:59:53 2014 -0400

--
 CHANGES.txt| 2 +-
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7316bb3a/CHANGES.txt
--
diff --cc CHANGES.txt
index 1b67357,50d5896..de564e9
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -126,12 -88,8 +126,12 @@@ Merged from 1.2
   * Schedule schema pulls on change (CASSANDRA-6971)
   * Non-droppable verbs shouldn't be dropped from OTC (CASSANDRA-6980)
   * Shutdown batchlog executor in SS#drain() (CASSANDRA-7025)
 + * Fix batchlog to account for CF truncation records (CASSANDRA-6999)
 + * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Properly load trustore in the native protocol (CASSANDRA-6847)
 -
 + * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
- 
++ * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
  
  2.0.6
   * Avoid race-prone second scrub of system keyspace (CASSANDRA-6797)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7316bb3a/src/java/org/apache/cassandra/gms/FailureDetector.java
--



[2/4] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

2014-04-28 Thread dbrosius
Merge branch 'cassandra-1.2' into cassandra-2.0


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2b322937
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2b322937
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2b322937

Branch: refs/heads/trunk
Commit: 2b3229370bb5f7d0e5271fd5964fc6f374874756
Parents: 9002e76 7f01980
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 19:56:56 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 19:56:56 2014 -0400

--
 CHANGES.txt| 1 +
 src/java/org/apache/cassandra/gms/FailureDetector.java | 2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2b322937/CHANGES.txt
--
diff --cc CHANGES.txt
index 376ad87,e8d6a8d..50d5896
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -21,63 -11,12 +21,64 @@@ Merged from 1.2
   * Fix CQLSH parsing of functions and BLOB literals (CASSANDRA-7018)
   * Require nodetool rebuild_index to specify index names (CASSANDRA-7038)
   * Ensure that batchlog and hint timeouts do not produce hints 
(CASSANDRA-7058)
 - * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
   * Always clean up references in SerializingCache (CASSANDRA-6994)
 + * Don't shut MessagingService down when replacing a node (CASSANDRA-6476)
+  * fix npe when doing -Dcassandra.fd_initial_value_ms (CASSANDRA-6751)
  
  
 -1.2.16
 +2.0.7
 + * Put nodes in hibernate when join_ring is false (CASSANDRA-6961)
 + * Avoid early loading of non-system keyspaces before compaction-leftovers 
 +   cleanup at startup (CASSANDRA-6913)
 + * Restrict Windows to parallel repairs (CASSANDRA-6907)
 + * (Hadoop) Allow manually specifying start/end tokens in CFIF 
(CASSANDRA-6436)
 + * Fix NPE in MeteredFlusher (CASSANDRA-6820)
 + * Fix race processing range scan responses (CASSANDRA-6820)
 + * Allow deleting snapshots from dropped keyspaces (CASSANDRA-6821)
 + * Add uuid() function (CASSANDRA-6473)
 + * Omit tombstones from schema digests (CASSANDRA-6862)
 + * Include correct consistencyLevel in LWT timeout (CASSANDRA-6884)
 + * Lower chances for losing new SSTables during nodetool refresh and
 +   ColumnFamilyStore.loadNewSSTables (CASSANDRA-6514)
 + * Add support for DELETE ... IF EXISTS to CQL3 (CASSANDRA-5708)
 + * Update hadoop_cql3_word_count example (CASSANDRA-6793)
 + * Fix handling of RejectedExecution in sync Thrift server (CASSANDRA-6788)
 + * Log more information when exceeding tombstone_warn_threshold 
(CASSANDRA-6865)
 + * Fix truncate to not abort due to unreachable fat clients (CASSANDRA-6864)
 + * Fix schema concurrency exceptions (CASSANDRA-6841)
 + * Fix leaking validator FH in StreamWriter (CASSANDRA-6832)
 + * Fix saving triggers to schema (CASSANDRA-6789)
 + * Fix trigger mutations when base mutation list is immutable (CASSANDRA-6790)
 + * Fix accounting in FileCacheService to allow re-using RAR (CASSANDRA-6838)
 + * Fix static counter columns (CASSANDRA-6827)
 + * Restore expiring-deleted (cell) compaction optimization (CASSANDRA-6844)
 + * Fix CompactionManager.needsCleanup (CASSANDRA-6845)
 + * Correctly compare BooleanType values other than 0 and 1 (CASSANDRA-6779)
 + * Read message id as string from earlier versions (CASSANDRA-6840)
 + * Properly use the Paxos consistency for (non-protocol) batch 
(CASSANDRA-6837)
 + * Add paranoid disk failure option (CASSANDRA-6646)
 + * Improve PerRowSecondaryIndex performance (CASSANDRA-6876)
 + * Extend triggers to support CAS updates (CASSANDRA-6882)
 + * Static columns with IF NOT EXISTS don't always work as expected 
(CASSANDRA-6873)
 + * Fix paging with SELECT DISTINCT (CASSANDRA-6857)
 + * Fix UnsupportedOperationException on CAS timeout (CASSANDRA-6923)
 + * Improve MeteredFlusher handling of MF-unaffected column families
 +   (CASSANDRA-6867)
 + * Add CqlRecordReader using native pagination (CASSANDRA-6311)
 + * Add QueryHandler interface (CASSANDRA-6659)
 + * Track liveRatio per-memtable, not per-CF (CASSANDRA-6945)
 + * Make sure upgradesstables keeps sstable level (CASSANDRA-6958)
 + * Fix LIMIT with static columns (CASSANDRA-6956)
 + * Fix clash with CQL column name in thrift validation (CASSANDRA-6892)
 + * Fix error with super columns in mixed 1.2-2.0 clusters (CASSANDRA-6966)
 + * Fix bad skip of sstables on slice query with composite start/finish 
(CASSANDRA-6825)
 + * Fix unintended update with conditional statement (CASSANDRA-6893)
 + * Fix map element access in IF (CASSANDRA-6914)
 + * Avoid costly range calculations for range queries on system keyspaces
 +   (CASSANDRA-6906)
 + * Fix SSTable not released if stream session fails (CASSANDRA-6818)
 + * Avoid 

[jira] [Commented] (CASSANDRA-7106) Retry multiple times before giving up on fetching splits

2014-04-28 Thread Brandon Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983754#comment-13983754
 ] 

Brandon Williams commented on CASSANDRA-7106:
-

Please undo all the import re-writing.

 Retry multiple times before giving up on fetching splits
 

 Key: CASSANDRA-7106
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7106
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Victor Igumnov
Priority: Minor
 Fix For: 2.1 beta1

 Attachments: retry-splits.patch


 See Github for patch:
 https://github.com/apache/cassandra/pull/34
 https://github.com/victori/cassandra
 What the patch does:
 While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
 get any of the queries to return reliably. Looking further into the issue I 
 noticed that if just a single fetch for a split fails the whole job is 
 aborted. So instead of giving up and failing out the whole job, retry 
 fetching the data a few times before completely giving up.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-6875) CQL3: select multiple CQL rows in a single partition using IN

2014-04-28 Thread JIRA

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michaël Figuière updated CASSANDRA-6875:


Issue Type: Improvement  (was: Bug)

 CQL3: select multiple CQL rows in a single partition using IN
 -

 Key: CASSANDRA-6875
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6875
 Project: Cassandra
  Issue Type: Improvement
  Components: API
Reporter: Nicolas Favre-Felix
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.8


 In the spirit of CASSANDRA-4851 and to bring CQL to parity with Thrift, it is 
 important to support reading several distinct CQL rows from a given partition 
 using a distinct set of coordinates for these rows within the partition.
 CASSANDRA-4851 introduced a range scan over the multi-dimensional space of 
 clustering keys. We also need to support a multi-get of CQL rows, 
 potentially using the IN keyword to define a set of clustering keys to 
 fetch at once.
 (reusing the same example\:)
 Consider the following table:
 {code}
 CREATE TABLE test (
   k int,
   c1 int,
   c2 int,
   PRIMARY KEY (k, c1, c2)
 );
 {code}
 with the following data:
 {code}
  k | c1 | c2
 ---++
  0 |  0 |  0
  0 |  0 |  1
  0 |  1 |  0
  0 |  1 |  1
 {code}
 We can fetch a single row or a range of rows, but not a set of them:
 {code}
  SELECT * FROM test WHERE k = 0 AND (c1, c2) IN ((0, 0), (1,1)) ;
 Bad Request: line 1:54 missing EOF at ','
 {code}
 Supporting this syntax would return:
 {code}
  k | c1 | c2
 ---++
  0 |  0 |  0
  0 |  1 |  1
 {code}
 Being able to fetch these two CQL rows in a single read is important to 
 maintain partition-level isolation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-7106) Retry multiple times before giving up on fetching splits

2014-04-28 Thread Victor Igumnov (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Victor Igumnov updated CASSANDRA-7106:
--

Attachment: retries-reworked.patch

 Retry multiple times before giving up on fetching splits
 

 Key: CASSANDRA-7106
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7106
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Victor Igumnov
Priority: Minor
 Fix For: 2.1 beta1

 Attachments: retries-reworked.patch, retry-splits.patch


 See Github for patch:
 https://github.com/apache/cassandra/pull/34
 https://github.com/victori/cassandra
 What the patch does:
 While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
 get any of the queries to return reliably. Looking further into the issue I 
 noticed that if just a single fetch for a split fails the whole job is 
 aborted. So instead of giving up and failing out the whole job, retry 
 fetching the data a few times before completely giving up.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7106) Retry multiple times before giving up on fetching splits

2014-04-28 Thread Victor Igumnov (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983856#comment-13983856
 ] 

Victor Igumnov commented on CASSANDRA-7106:
---

Done, I also matched the formatting of the class so it is the same style. 

 Retry multiple times before giving up on fetching splits
 

 Key: CASSANDRA-7106
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7106
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Victor Igumnov
Priority: Minor
 Fix For: 2.1 beta1

 Attachments: retries-reworked.patch, retry-splits.patch


 See Github for patch:
 https://github.com/apache/cassandra/pull/34
 https://github.com/victori/cassandra
 What the patch does:
 While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
 get any of the queries to return reliably. Looking further into the issue I 
 noticed that if just a single fetch for a split fails the whole job is 
 aborted. So instead of giving up and failing out the whole job, retry 
 fetching the data a few times before completely giving up.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-7106) Retry multiple times before giving up on fetching splits

2014-04-28 Thread Victor Igumnov (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Victor Igumnov updated CASSANDRA-7106:
--

Description: 
See Github for patch:

https://github.com/victori/cassandra

What the patch does:

While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
get any of the queries to return reliably. Looking further into the issue I 
noticed that if just a single fetch for a split fails the whole job is aborted. 
So instead of giving up and failing out the whole job, retry fetching the data 
a few times before completely giving up.  


  was:
See Github for patch:

https://github.com/apache/cassandra/pull/34
https://github.com/victori/cassandra

What the patch does:

While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
get any of the queries to return reliably. Looking further into the issue I 
noticed that if just a single fetch for a split fails the whole job is aborted. 
So instead of giving up and failing out the whole job, retry fetching the data 
a few times before completely giving up.  



 Retry multiple times before giving up on fetching splits
 

 Key: CASSANDRA-7106
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7106
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Victor Igumnov
Priority: Minor
 Fix For: 2.1 beta1

 Attachments: retries-reworked.patch, retry-splits.patch


 See Github for patch:
 https://github.com/victori/cassandra
 What the patch does:
 While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
 get any of the queries to return reliably. Looking further into the issue I 
 noticed that if just a single fetch for a split fails the whole job is 
 aborted. So instead of giving up and failing out the whole job, retry 
 fetching the data a few times before completely giving up.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-6982) start_column in get_page_slice has odd behaivor

2014-04-28 Thread Dave Brosius (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius updated CASSANDRA-6982:


Priority: Minor  (was: Critical)

 start_column in get_page_slice has odd behaivor
 ---

 Key: CASSANDRA-6982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6982
 Project: Cassandra
  Issue Type: Bug
Reporter: Edward Capriolo
Assignee: Edward Capriolo
Priority: Minor

 get_paged_slice is described as so:
 {code}
  /**
returns a range of columns, wrapping to the next rows if necessary to 
 collect max_results.
   */
   listKeySlice get_paged_slice(1:required string column_family,
  2:required KeyRange range,
  3:required binary start_column,
  4:required ConsistencyLevel 
 consistency_level=ConsistencyLevel.ONE)
  throws (1:InvalidRequestException ire, 
 2:UnavailableException ue, 3:TimedOutException te),
 {code}
 The term max_results is not defined, I take it to mean key_range.count.
 The larger issue I have found is that start_column seems to be ignored in 
 some cases.
 testNormal() produces this error
 junit.framework.ComparisonFailure: null expected:[c] but was:[a]
 The problem seems to be KeyRanges that use tokens and not keys.
 {code}
 KeyRange kr = new KeyRange();
   kr.setCount(3);
   kr.setStart_token();
   kr.setEnd_token();   
 {code}
 A failing test is here:
 https://github.com/edwardcapriolo/cassandra/compare/pg?expand=1
 Is this a bug? It feels like one, or is this just undefined behaviour. If it 
 is a bug I would like to fix. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-7072) Allow better specification of rack properties file

2014-04-28 Thread Dave Brosius (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius updated CASSANDRA-7072:


Reproduced In: 2.0.7, 2.0.6, 2.0.5, 2.0.4, 2.0.3  (was: 2.0.3, 2.0.4, 
2.0.5, 2.0.6, 2.0.7)
 Priority: Minor  (was: Major)

 Allow better specification of rack properties file
 --

 Key: CASSANDRA-7072
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7072
 Project: Cassandra
  Issue Type: Improvement
  Components: Config, Core
Reporter: Patrick Ziegler
Priority: Minor
  Labels: properties, rack, snitch
 Fix For: 2.0.7

 Attachments: cassandra-2.0.7-7072.txt

   Original Estimate: 72h
  Remaining Estimate: 72h

 Currently the SnitchProperties class loads the properties file via the 
 classloader, this causes problems in our test environment where we use an 
 embedded cassandra for testing and cannot specify what file/where from to 
 load it.
 We will extend the functionality to support loading like the cassandra.yaml 
 file is supported, via the System.properties map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7105) SELECT with IN on final column of composite and compound primary key fails

2014-04-28 Thread Bill Mitchell (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983883#comment-13983883
 ] 

Bill Mitchell commented on CASSANDRA-7105:
--

As was suggested in the query results above, the first IN is not essential to 
this problem.  Issuing the query against each partition separately with an 
equality relation, specifying IN only for the final column, returns no rows.  

 SELECT with IN on final column of composite and compound primary key fails
 --

 Key: CASSANDRA-7105
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7105
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: DataStax Cassandra 2.0.7
 Windows dual-core laptop
Reporter: Bill Mitchell

 I have a failing sequence where I specify an IN constraint on the final int 
 column of the composite primary key and an IN constraint on the final String 
 column of the compound primary key and no rows are returned, when rows should 
 be returned.  
 {noformat}
 CREATE TABLE IF NOT EXISTS sr2 (siteID TEXT, partition INT, listID BIGINT, 
 emailAddr TEXT, emailCrypt TEXT, createDate TIMESTAMP, removeDate TIMESTAMP, 
 removeImportID BIGINT, properties TEXT, PRIMARY KEY ((siteID, listID, 
 partition), createDate, emailCrypt) ) WITH CLUSTERING ORDER BY (createDate 
 DESC, emailCrypt DESC)  AND compression = {'sstable_compression' : 
 'SnappyCompressor'} AND compaction = {'class' : 
 'SizeTieredCompactionStrategy'};
 insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, 
 createDate) values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 1, 'xyzzy', 
 '5fe7719229092cdde4526afbc65c900c', '2014-04-28T14:05:59.236-0500');
 insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, 
 createDate) values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 2, 'noname', 
 '97bf28af2ca9c498d6e47237bb8680bf', '2014-04-28T14:05:59.236-0500');
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
 '97bf28af2ca9c498d6e47237bb8680bf';
  emailcrypt   | emailaddr
 --+---
  97bf28af2ca9c498d6e47237bb8680bf |noname
 (1 rows)
 select emailCrypt, emailAddr  from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
 '5fe7719229092cdde4526afbc65c900c';
  emailcrypt   | emailaddr
 --+---
  5fe7719229092cdde4526afbc65c900c | xyzzy
 (1 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
 and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 cqlsh:test_multiple_in select * from sr2;
  siteid   | listid | partition | createdate   
 | emailcrypt | emailaddr| 
 properties | removedate | re
 moveimportid
 --++---+--++--+++---
 -
  4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 2 | 2014-04-28 
 14:05:59Central Daylight Time | noname | 97bf28af2ca9c498d6e47237bb8680bf 
 |   null |   null |
 null
  4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 1 | 2014-04-28 
 14:05:59Central Daylight Time |  xyzzy | 5fe7719229092cdde4526afbc65c900c 
 |   null |   null |
 null
 (2 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
 and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition 

[jira] [Comment Edited] (CASSANDRA-7105) SELECT with IN on final column of composite and compound primary key fails

2014-04-28 Thread Bill Mitchell (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983883#comment-13983883
 ] 

Bill Mitchell edited comment on CASSANDRA-7105 at 4/29/14 2:25 AM:
---

As was suggested in the query results above, the first IN is not essential to 
this problem.  Issuing the query against each partition separately with an 
equality relation, specifying IN only for the final column, returns no rows.  

When I changed the schema so the CLUSTERING ORDER was ASC instead of DESC, the 
queries worked fine.  So apparently the problem lies with the IN relation 
combined with order DESC on the column.


was (Author: wtmitchell3):
As was suggested in the query results above, the first IN is not essential to 
this problem.  Issuing the query against each partition separately with an 
equality relation, specifying IN only for the final column, returns no rows.  

 SELECT with IN on final column of composite and compound primary key fails
 --

 Key: CASSANDRA-7105
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7105
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: DataStax Cassandra 2.0.7
 Windows dual-core laptop
Reporter: Bill Mitchell

 I have a failing sequence where I specify an IN constraint on the final int 
 column of the composite primary key and an IN constraint on the final String 
 column of the compound primary key and no rows are returned, when rows should 
 be returned.  
 {noformat}
 CREATE TABLE IF NOT EXISTS sr2 (siteID TEXT, partition INT, listID BIGINT, 
 emailAddr TEXT, emailCrypt TEXT, createDate TIMESTAMP, removeDate TIMESTAMP, 
 removeImportID BIGINT, properties TEXT, PRIMARY KEY ((siteID, listID, 
 partition), createDate, emailCrypt) ) WITH CLUSTERING ORDER BY (createDate 
 DESC, emailCrypt DESC)  AND compression = {'sstable_compression' : 
 'SnappyCompressor'} AND compaction = {'class' : 
 'SizeTieredCompactionStrategy'};
 insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, 
 createDate) values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 1, 'xyzzy', 
 '5fe7719229092cdde4526afbc65c900c', '2014-04-28T14:05:59.236-0500');
 insert into sr2 (siteID, listID, partition, emailAddr, emailCrypt, 
 createDate) values ('4ca4f79e-3ab2-41c5-ae42-c7009736f1d5', 34, 2, 'noname', 
 '97bf28af2ca9c498d6e47237bb8680bf', '2014-04-28T14:05:59.236-0500');
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 2 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
 '97bf28af2ca9c498d6e47237bb8680bf';
  emailcrypt   | emailaddr
 --+---
  97bf28af2ca9c498d6e47237bb8680bf |noname
 (1 rows)
 select emailCrypt, emailAddr  from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
 createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt = 
 '5fe7719229092cdde4526afbc65c900c';
  emailcrypt   | emailaddr
 --+---
  5fe7719229092cdde4526afbc65c900c | xyzzy
 (1 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
 and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 cqlsh:test_multiple_in select * from sr2;
  siteid   | listid | partition | createdate   
 | emailcrypt | emailaddr| 
 properties | removedate | re
 moveimportid
 --++---+--++--+++---
 -
  4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 2 | 2014-04-28 
 14:05:59Central Daylight Time | noname | 97bf28af2ca9c498d6e47237bb8680bf 
 |   null |   null |
 null
  4ca4f79e-3ab2-41c5-ae42-c7009736f1d5 | 34 | 1 | 2014-04-28 
 14:05:59Central Daylight Time |  xyzzy | 5fe7719229092cdde4526afbc65c900c 
 |   null |   null |
 null
 (2 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition IN (1,2) 
 and createDate = '2014-04-28T14:05:59.236-0500' and emailCrypt IN 
 ('97bf28af2ca9c498d6e47237bb8680bf','5fe7719229092cdde4526afbc65c900c');
 (0 rows)
 select emailCrypt, emailAddr from sr2 where siteID = 
 '4ca4f79e-3ab2-41c5-ae42-c7009736f1d5' and listID = 34 and partition = 1 and 
 createDate = '2014-04-28T14:05:59.236-0500' 

[jira] [Updated] (CASSANDRA-6942) Parameters Were Removed from Tracing Session

2014-04-28 Thread Dave Brosius (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius updated CASSANDRA-6942:


Attachment: 6942.txt

patch against 2.0

 Parameters Were Removed from Tracing Session
 

 Key: CASSANDRA-6942
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6942
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
 Fix For: 2.0.8

 Attachments: 6942.txt


 CASSANDRA-6133 accidentally stopped inserting the 'parameters' column on 
 tracing sessions.  (The actual removal of that line was not in the attached 
 patch, but in the final commit: 
 [64890d86d87ec527b2f7fac2bc4d80712c290268|https://github.com/apache/cassandra/commit/64890d86d87ec527b2f7fac2bc4d80712c290268].)



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-6874) CQL3 docs doesn't document conditional DELETE

2014-04-28 Thread Dave Brosius (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius updated CASSANDRA-6874:


Labels: documentation  (was: )

 CQL3 docs doesn't document conditional DELETE
 -

 Key: CASSANDRA-6874
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6874
 Project: Cassandra
  Issue Type: Bug
  Components: Documentation  website
Reporter: Mohica Jasha
  Labels: documentation

 http://cassandra.apache.org/doc/cql3/CQL.html#deleteStmt doesn't document 
 conditional {{DELETE}}. But support for if clause for {{DELETE}} was there 
 from C* 2.0



--
This message was sent by Atlassian JIRA
(v6.2#6252)


git commit: fix message spilleng

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 2b3229370 - daf54c5c7


fix message spilleng


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/daf54c5c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/daf54c5c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/daf54c5c

Branch: refs/heads/cassandra-2.0
Commit: daf54c5c73f7d59967eb198ed6eaba952c1087c7
Parents: 2b32293
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 22:50:57 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 22:50:57 2014 -0400

--
 .../apache/cassandra/cql3/statements/CreateTriggerStatement.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/daf54c5c/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
index 329b7bc..760d870 100644
--- a/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
@@ -49,7 +49,7 @@ public class CreateTriggerStatement extends 
SchemaAlteringStatement
 
 public void checkAccess(ClientState state) throws UnauthorizedException
 {
-state.ensureIsSuper(Only superusers are allowed to perfrom CREATE 
TRIGGER queries);
+state.ensureIsSuper(Only superusers are allowed to perform CREATE 
TRIGGER queries);
 }
 
 public void validate(ClientState state) throws RequestValidationException



[1/2] git commit: fix message spilleng

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 7316bb3a6 - 24eeeb91d


fix message spilleng


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/daf54c5c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/daf54c5c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/daf54c5c

Branch: refs/heads/cassandra-2.1
Commit: daf54c5c73f7d59967eb198ed6eaba952c1087c7
Parents: 2b32293
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 22:50:57 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 22:50:57 2014 -0400

--
 .../apache/cassandra/cql3/statements/CreateTriggerStatement.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/daf54c5c/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
index 329b7bc..760d870 100644
--- a/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
@@ -49,7 +49,7 @@ public class CreateTriggerStatement extends 
SchemaAlteringStatement
 
 public void checkAccess(ClientState state) throws UnauthorizedException
 {
-state.ensureIsSuper(Only superusers are allowed to perfrom CREATE 
TRIGGER queries);
+state.ensureIsSuper(Only superusers are allowed to perform CREATE 
TRIGGER queries);
 }
 
 public void validate(ClientState state) throws RequestValidationException



[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread dbrosius
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/24eeeb91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/24eeeb91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/24eeeb91

Branch: refs/heads/cassandra-2.1
Commit: 24eeeb91d2246b5e397a9728798a65445cb847da
Parents: 7316bb3 daf54c5
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 22:51:23 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 22:51:23 2014 -0400

--
 .../apache/cassandra/cql3/statements/CreateTriggerStatement.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/24eeeb91/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
--



[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

2014-04-28 Thread dbrosius
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/55ac89d2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/55ac89d2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/55ac89d2

Branch: refs/heads/trunk
Commit: 55ac89d248d31821bdc9bfafcdb94ce7a785cc7a
Parents: 00b6a03 24eeeb9
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 22:51:52 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 22:51:52 2014 -0400

--
 .../apache/cassandra/cql3/statements/CreateTriggerStatement.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--




[1/3] git commit: fix message spilleng

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/trunk 00b6a0372 - 55ac89d24


fix message spilleng


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/daf54c5c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/daf54c5c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/daf54c5c

Branch: refs/heads/trunk
Commit: daf54c5c73f7d59967eb198ed6eaba952c1087c7
Parents: 2b32293
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 22:50:57 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 22:50:57 2014 -0400

--
 .../apache/cassandra/cql3/statements/CreateTriggerStatement.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/daf54c5c/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
index 329b7bc..760d870 100644
--- a/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
@@ -49,7 +49,7 @@ public class CreateTriggerStatement extends 
SchemaAlteringStatement
 
 public void checkAccess(ClientState state) throws UnauthorizedException
 {
-state.ensureIsSuper(Only superusers are allowed to perfrom CREATE 
TRIGGER queries);
+state.ensureIsSuper(Only superusers are allowed to perform CREATE 
TRIGGER queries);
 }
 
 public void validate(ClientState state) throws RequestValidationException



[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread dbrosius
Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/24eeeb91
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/24eeeb91
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/24eeeb91

Branch: refs/heads/trunk
Commit: 24eeeb91d2246b5e397a9728798a65445cb847da
Parents: 7316bb3 daf54c5
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 22:51:23 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 22:51:23 2014 -0400

--
 .../apache/cassandra/cql3/statements/CreateTriggerStatement.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/24eeeb91/src/java/org/apache/cassandra/cql3/statements/CreateTriggerStatement.java
--



[jira] [Updated] (CASSANDRA-6942) Parameters Were Removed from Tracing Session

2014-04-28 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6942:


Reviewer: Tyler Hobbs
Assignee: Dave Brosius

 Parameters Were Removed from Tracing Session
 

 Key: CASSANDRA-6942
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6942
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Tyler Hobbs
Assignee: Dave Brosius
 Fix For: 2.0.8

 Attachments: 6942.txt


 CASSANDRA-6133 accidentally stopped inserting the 'parameters' column on 
 tracing sessions.  (The actual removal of that line was not in the attached 
 patch, but in the final commit: 
 [64890d86d87ec527b2f7fac2bc4d80712c290268|https://github.com/apache/cassandra/commit/64890d86d87ec527b2f7fac2bc4d80712c290268].)



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Updated] (CASSANDRA-6875) CQL3: select multiple CQL rows in a single partition using IN

2014-04-28 Thread Brandon Williams (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6875:


Issue Type: Bug  (was: Improvement)

 CQL3: select multiple CQL rows in a single partition using IN
 -

 Key: CASSANDRA-6875
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6875
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Nicolas Favre-Felix
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.8


 In the spirit of CASSANDRA-4851 and to bring CQL to parity with Thrift, it is 
 important to support reading several distinct CQL rows from a given partition 
 using a distinct set of coordinates for these rows within the partition.
 CASSANDRA-4851 introduced a range scan over the multi-dimensional space of 
 clustering keys. We also need to support a multi-get of CQL rows, 
 potentially using the IN keyword to define a set of clustering keys to 
 fetch at once.
 (reusing the same example\:)
 Consider the following table:
 {code}
 CREATE TABLE test (
   k int,
   c1 int,
   c2 int,
   PRIMARY KEY (k, c1, c2)
 );
 {code}
 with the following data:
 {code}
  k | c1 | c2
 ---++
  0 |  0 |  0
  0 |  0 |  1
  0 |  1 |  0
  0 |  1 |  1
 {code}
 We can fetch a single row or a range of rows, but not a set of them:
 {code}
  SELECT * FROM test WHERE k = 0 AND (c1, c2) IN ((0, 0), (1,1)) ;
 Bad Request: line 1:54 missing EOF at ','
 {code}
 Supporting this syntax would return:
 {code}
  k | c1 | c2
 ---++
  0 |  0 |  0
  0 |  1 |  1
 {code}
 Being able to fetch these two CQL rows in a single read is important to 
 maintain partition-level isolation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-6875) CQL3: select multiple CQL rows in a single partition using IN

2014-04-28 Thread Brandon Williams (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13983930#comment-13983930
 ] 

Brandon Williams commented on CASSANDRA-6875:
-

Moving this back to 'bug', [~mfiguiere], because anything that thrift can do 
that CQL can't is a bug.

 CQL3: select multiple CQL rows in a single partition using IN
 -

 Key: CASSANDRA-6875
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6875
 Project: Cassandra
  Issue Type: Bug
  Components: API
Reporter: Nicolas Favre-Felix
Assignee: Tyler Hobbs
Priority: Minor
 Fix For: 2.0.8


 In the spirit of CASSANDRA-4851 and to bring CQL to parity with Thrift, it is 
 important to support reading several distinct CQL rows from a given partition 
 using a distinct set of coordinates for these rows within the partition.
 CASSANDRA-4851 introduced a range scan over the multi-dimensional space of 
 clustering keys. We also need to support a multi-get of CQL rows, 
 potentially using the IN keyword to define a set of clustering keys to 
 fetch at once.
 (reusing the same example\:)
 Consider the following table:
 {code}
 CREATE TABLE test (
   k int,
   c1 int,
   c2 int,
   PRIMARY KEY (k, c1, c2)
 );
 {code}
 with the following data:
 {code}
  k | c1 | c2
 ---++
  0 |  0 |  0
  0 |  0 |  1
  0 |  1 |  0
  0 |  1 |  1
 {code}
 We can fetch a single row or a range of rows, but not a set of them:
 {code}
  SELECT * FROM test WHERE k = 0 AND (c1, c2) IN ((0, 0), (1,1)) ;
 Bad Request: line 1:54 missing EOF at ','
 {code}
 Supporting this syntax would return:
 {code}
  k | c1 | c2
 ---++
  0 |  0 |  0
  0 |  1 |  1
 {code}
 Being able to fetch these two CQL rows in a single read is important to 
 maintain partition-level isolation.



--
This message was sent by Atlassian JIRA
(v6.2#6252)


git commit: quell all the 'internal proprietary API' warnings induced by Unsafe and friends

2014-04-28 Thread dbrosius
Repository: cassandra
Updated Branches:
  refs/heads/trunk 55ac89d24 - afe2e1407


quell all the 'internal proprietary API' warnings induced by Unsafe and friends


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afe2e140
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afe2e140
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afe2e140

Branch: refs/heads/trunk
Commit: afe2e140796f8527520a1a702afae0855a90d24e
Parents: 55ac89d
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Mon Apr 28 23:14:05 2014 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Mon Apr 28 23:14:05 2014 -0400

--
 build.xml | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/afe2e140/build.xml
--
diff --git a/build.xml b/build.xml
index 9326424..2a6f6fe 100644
--- a/build.xml
+++ b/build.xml
@@ -672,6 +672,7 @@
destdir=${build.classes.main} includeantruntime=false 
source=${source.version} target=${target.version}
 src path=${build.src.java}/
 src path=${build.src.gen-java}/
+   compilerarg value=-XDignore.symbol.file/
 classpath refid=cassandra.classpath/
 /javac
 antcall target=createVersionPropFile/



git commit: Add ability to set/get logging levels to nodetool patch by Jackson Chung reviewed by Vijay for CASSANDRA-6751

2014-04-28 Thread vijay
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 daf54c5c7 - 0a20f5f17


Add ability to set/get logging levels to nodetool
patch by Jackson Chung reviewed by Vijay for CASSANDRA-6751


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0a20f5f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0a20f5f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0a20f5f1

Branch: refs/heads/cassandra-2.0
Commit: 0a20f5f170b3596e6e74bba7daddebd4c1f5963a
Parents: daf54c5
Author: Vijay vijay2...@gmail.com
Authored: Mon Apr 28 21:24:07 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Mon Apr 28 21:27:58 2014 -0700

--
 .../cassandra/service/StorageService.java   | 41 ++--
 .../cassandra/service/StorageServiceMBean.java  |  2 +
 .../org/apache/cassandra/tools/NodeCmd.java | 29 +-
 .../org/apache/cassandra/tools/NodeProbe.java   | 10 +
 .../apache/cassandra/tools/NodeToolHelp.yaml|  6 +++
 5 files changed, 84 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0a20f5f1/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 75f6427..f44eaed 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -45,6 +45,7 @@ import com.google.common.util.concurrent.Uninterruptibles;
 import org.apache.cassandra.cql3.CQL3Type;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.log4j.Level;
+import org.apache.log4j.LogManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -57,7 +58,6 @@ import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.config.KSMetaData;
 import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.db.*;
-import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.index.SecondaryIndex;
 import org.apache.cassandra.dht.*;
@@ -2795,9 +2795,44 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public void setLog4jLevel(String classQualifier, String rawLevel)
 {
+org.apache.log4j.Logger log4jlogger = 
org.apache.log4j.Logger.getLogger(classQualifier);
+// if both classQualifer and rawLevel are empty, reload from 
configuration
+if (StringUtils.isBlank(classQualifier)  
StringUtils.isBlank(rawLevel))
+{
+LogManager.resetConfiguration();
+CassandraDaemon.initLog4j();
+return;
+}
+// classQualifer is set, but blank level given
+else if (StringUtils.isNotBlank(classQualifier)  
StringUtils.isBlank(rawLevel))
+{
+if (log4jlogger.getLevel() != null || 
log4jlogger.getAllAppenders().hasMoreElements())
+log4jlogger.setLevel(null);
+return;
+}
+
 Level level = Level.toLevel(rawLevel);
-org.apache.log4j.Logger.getLogger(classQualifier).setLevel(level);
-logger.info(set log level to  + level +  for classes under ' + 
classQualifier + ' (if the level doesn't look like ' + rawLevel + ' then 
log4j couldn't parse ' + rawLevel + '));
+log4jlogger.setLevel(level);
+logger.info(set log level to {} for classes under '{}' (if the level 
doesn't look like '{}' then the logger couldn't parse '{}'), level, 
classQualifier, rawLevel, rawLevel);
+}
+
+/**
+ * @return the runtime logging levels for all the configured loggers
+ */
+@Override
+public MapString,String getLoggingLevels()
+{
+MapString, String logLevelMaps = Maps.newLinkedHashMap();
+org.apache.log4j.Logger rootLogger = 
org.apache.log4j.Logger.getRootLogger();
+logLevelMaps.put(rootLogger.getName(), 
rootLogger.getLevel().toString());
+Enumerationorg.apache.log4j.Logger loggers = 
LogManager.getCurrentLoggers();
+while (loggers.hasMoreElements())
+{
+org.apache.log4j.Logger logger= loggers.nextElement();
+if (logger.getLevel() != null)
+logLevelMaps.put(logger.getName(), 
logger.getLevel().toString());
+}
+return logLevelMaps;
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0a20f5f1/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 

[3/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread vijay
Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/service/StorageService.java
src/java/org/apache/cassandra/tools/NodeCmd.java
src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d402cf68
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d402cf68
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d402cf68

Branch: refs/heads/cassandra-2.1
Commit: d402cf687001390cf81307e2d9d45a87b24f5837
Parents: 24eeeb9 0a20f5f
Author: Vijay vijay2...@gmail.com
Authored: Mon Apr 28 21:38:22 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Mon Apr 28 21:38:22 2014 -0700

--
 conf/logback.xml|  2 +-
 .../cassandra/service/StorageService.java   | 49 ++--
 .../cassandra/service/StorageServiceMBean.java  | 21 -
 .../org/apache/cassandra/tools/NodeProbe.java   | 17 +++
 .../org/apache/cassandra/tools/NodeTool.java| 32 -
 5 files changed, 114 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d402cf68/conf/logback.xml
--
diff --cc conf/logback.xml
index 655bcf6,000..2657174
mode 100644,00..100644
--- a/conf/logback.xml
+++ b/conf/logback.xml
@@@ -1,34 -1,0 +1,34 @@@
 +configuration scan=true
-
++  jmxConfigurator /
 +  appender name=FILE 
class=ch.qos.logback.core.rolling.RollingFileAppender
 +file/var/log/cassandra/system.log/file
 +rollingPolicy 
class=ch.qos.logback.core.rolling.FixedWindowRollingPolicy
 +  fileNamePattern/var/log/cassandra/system.log.%i.zip/fileNamePattern
 +  minIndex1/minIndex
 +  maxIndex20/maxIndex
 +/rollingPolicy
 +
 +triggeringPolicy 
class=ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy
 +  maxFileSize20MB/maxFileSize
 +/triggeringPolicy
 +encoder
 +  pattern%-5level [%thread] %date{ISO8601} %F:%L - %msg%n/pattern
 +  !-- old-style log format
 +  pattern%5level [%thread] %date{ISO8601} %F (line %L) %msg%n/pattern
 +  --
 +/encoder
 +  /appender
 +  
 +  appender name=STDOUT class=ch.qos.logback.core.ConsoleAppender
 +encoder
 +  pattern%-5level %date{HH:mm:ss,SSS} %msg%n/pattern
 +/encoder
 +  /appender
 +
 +  root level=INFO
 +appender-ref ref=FILE /
 +appender-ref ref=STDOUT /
 +  /root
 +  
 +  logger name=com.thinkaurelius.thrift level=ERROR/
 +/configuration

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d402cf68/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index a9c0233,f44eaed..a284ab4
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -29,14 -29,11 +29,20 @@@ import java.util.*
  import java.util.concurrent.*;
  import java.util.concurrent.atomic.AtomicInteger;
  import java.util.concurrent.atomic.AtomicLong;
 +
++import javax.management.JMX;
  import javax.management.MBeanServer;
  import javax.management.Notification;
  import javax.management.NotificationBroadcasterSupport;
  import javax.management.ObjectName;
 +import javax.management.openmbean.TabularData;
 +import javax.management.openmbean.TabularDataSupport;
 +
++import ch.qos.logback.classic.LoggerContext;
++import ch.qos.logback.classic.jmx.JMXConfiguratorMBean;
++import ch.qos.logback.classic.spi.ILoggingEvent;
++import ch.qos.logback.core.Appender;
+ 
  import com.google.common.annotations.VisibleForTesting;
  import com.google.common.base.Predicate;
  import com.google.common.collect.*;
@@@ -44,9 -42,10 +50,8 @@@ import com.google.common.util.concurren
  import com.google.common.util.concurrent.Futures;
  import com.google.common.util.concurrent.Uninterruptibles;
  
--import org.apache.cassandra.cql3.CQL3Type;
  import org.apache.commons.lang3.StringUtils;
 -import org.apache.log4j.Level;
 -import org.apache.log4j.LogManager;
 +
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;
  
@@@ -2874,15 -2793,49 +2879,53 @@@ public class StorageService extends Not
  return liveEps;
  }
  
- public void setLoggingLevel(String classQualifier, String rawLevel)
 -public void setLog4jLevel(String classQualifier, String rawLevel)
++public void setLoggingLevel(String classQualifier, String rawLevel) 
throws Exception
  {
 -org.apache.log4j.Logger log4jlogger = 
org.apache.log4j.Logger.getLogger(classQualifier);
 +ch.qos.logback.classic.Logger logBackLogger = 
(ch.qos.logback.classic.Logger) 

[1/3] git commit: Add ability to set/get logging levels to nodetool patch by Jackson Chung reviewed by Vijay for CASSANDRA-6751

2014-04-28 Thread vijay
Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 24eeeb91d - d402cf687


Add ability to set/get logging levels to nodetool
patch by Jackson Chung reviewed by Vijay for CASSANDRA-6751


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0a20f5f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0a20f5f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0a20f5f1

Branch: refs/heads/cassandra-2.1
Commit: 0a20f5f170b3596e6e74bba7daddebd4c1f5963a
Parents: daf54c5
Author: Vijay vijay2...@gmail.com
Authored: Mon Apr 28 21:24:07 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Mon Apr 28 21:27:58 2014 -0700

--
 .../cassandra/service/StorageService.java   | 41 ++--
 .../cassandra/service/StorageServiceMBean.java  |  2 +
 .../org/apache/cassandra/tools/NodeCmd.java | 29 +-
 .../org/apache/cassandra/tools/NodeProbe.java   | 10 +
 .../apache/cassandra/tools/NodeToolHelp.yaml|  6 +++
 5 files changed, 84 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0a20f5f1/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 75f6427..f44eaed 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -45,6 +45,7 @@ import com.google.common.util.concurrent.Uninterruptibles;
 import org.apache.cassandra.cql3.CQL3Type;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.log4j.Level;
+import org.apache.log4j.LogManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -57,7 +58,6 @@ import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.config.KSMetaData;
 import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.db.*;
-import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.index.SecondaryIndex;
 import org.apache.cassandra.dht.*;
@@ -2795,9 +2795,44 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public void setLog4jLevel(String classQualifier, String rawLevel)
 {
+org.apache.log4j.Logger log4jlogger = 
org.apache.log4j.Logger.getLogger(classQualifier);
+// if both classQualifer and rawLevel are empty, reload from 
configuration
+if (StringUtils.isBlank(classQualifier)  
StringUtils.isBlank(rawLevel))
+{
+LogManager.resetConfiguration();
+CassandraDaemon.initLog4j();
+return;
+}
+// classQualifer is set, but blank level given
+else if (StringUtils.isNotBlank(classQualifier)  
StringUtils.isBlank(rawLevel))
+{
+if (log4jlogger.getLevel() != null || 
log4jlogger.getAllAppenders().hasMoreElements())
+log4jlogger.setLevel(null);
+return;
+}
+
 Level level = Level.toLevel(rawLevel);
-org.apache.log4j.Logger.getLogger(classQualifier).setLevel(level);
-logger.info(set log level to  + level +  for classes under ' + 
classQualifier + ' (if the level doesn't look like ' + rawLevel + ' then 
log4j couldn't parse ' + rawLevel + '));
+log4jlogger.setLevel(level);
+logger.info(set log level to {} for classes under '{}' (if the level 
doesn't look like '{}' then the logger couldn't parse '{}'), level, 
classQualifier, rawLevel, rawLevel);
+}
+
+/**
+ * @return the runtime logging levels for all the configured loggers
+ */
+@Override
+public MapString,String getLoggingLevels()
+{
+MapString, String logLevelMaps = Maps.newLinkedHashMap();
+org.apache.log4j.Logger rootLogger = 
org.apache.log4j.Logger.getRootLogger();
+logLevelMaps.put(rootLogger.getName(), 
rootLogger.getLevel().toString());
+Enumerationorg.apache.log4j.Logger loggers = 
LogManager.getCurrentLoggers();
+while (loggers.hasMoreElements())
+{
+org.apache.log4j.Logger logger= loggers.nextElement();
+if (logger.getLevel() != null)
+logLevelMaps.put(logger.getName(), 
logger.getLevel().toString());
+}
+return logLevelMaps;
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0a20f5f1/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 

[1/4] git commit: Add ability to set/get logging levels to nodetool patch by Jackson Chung reviewed by Vijay for CASSANDRA-6751

2014-04-28 Thread vijay
Repository: cassandra
Updated Branches:
  refs/heads/trunk afe2e1407 - b2ef56478


Add ability to set/get logging levels to nodetool
patch by Jackson Chung reviewed by Vijay for CASSANDRA-6751


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0a20f5f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0a20f5f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0a20f5f1

Branch: refs/heads/trunk
Commit: 0a20f5f170b3596e6e74bba7daddebd4c1f5963a
Parents: daf54c5
Author: Vijay vijay2...@gmail.com
Authored: Mon Apr 28 21:24:07 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Mon Apr 28 21:27:58 2014 -0700

--
 .../cassandra/service/StorageService.java   | 41 ++--
 .../cassandra/service/StorageServiceMBean.java  |  2 +
 .../org/apache/cassandra/tools/NodeCmd.java | 29 +-
 .../org/apache/cassandra/tools/NodeProbe.java   | 10 +
 .../apache/cassandra/tools/NodeToolHelp.yaml|  6 +++
 5 files changed, 84 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0a20f5f1/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 75f6427..f44eaed 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -45,6 +45,7 @@ import com.google.common.util.concurrent.Uninterruptibles;
 import org.apache.cassandra.cql3.CQL3Type;
 import org.apache.commons.lang3.StringUtils;
 import org.apache.log4j.Level;
+import org.apache.log4j.LogManager;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -57,7 +58,6 @@ import org.apache.cassandra.config.DatabaseDescriptor;
 import org.apache.cassandra.config.KSMetaData;
 import org.apache.cassandra.config.Schema;
 import org.apache.cassandra.db.*;
-import org.apache.cassandra.db.Keyspace;
 import org.apache.cassandra.db.commitlog.CommitLog;
 import org.apache.cassandra.db.index.SecondaryIndex;
 import org.apache.cassandra.dht.*;
@@ -2795,9 +2795,44 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public void setLog4jLevel(String classQualifier, String rawLevel)
 {
+org.apache.log4j.Logger log4jlogger = 
org.apache.log4j.Logger.getLogger(classQualifier);
+// if both classQualifer and rawLevel are empty, reload from 
configuration
+if (StringUtils.isBlank(classQualifier)  
StringUtils.isBlank(rawLevel))
+{
+LogManager.resetConfiguration();
+CassandraDaemon.initLog4j();
+return;
+}
+// classQualifer is set, but blank level given
+else if (StringUtils.isNotBlank(classQualifier)  
StringUtils.isBlank(rawLevel))
+{
+if (log4jlogger.getLevel() != null || 
log4jlogger.getAllAppenders().hasMoreElements())
+log4jlogger.setLevel(null);
+return;
+}
+
 Level level = Level.toLevel(rawLevel);
-org.apache.log4j.Logger.getLogger(classQualifier).setLevel(level);
-logger.info(set log level to  + level +  for classes under ' + 
classQualifier + ' (if the level doesn't look like ' + rawLevel + ' then 
log4j couldn't parse ' + rawLevel + '));
+log4jlogger.setLevel(level);
+logger.info(set log level to {} for classes under '{}' (if the level 
doesn't look like '{}' then the logger couldn't parse '{}'), level, 
classQualifier, rawLevel, rawLevel);
+}
+
+/**
+ * @return the runtime logging levels for all the configured loggers
+ */
+@Override
+public MapString,String getLoggingLevels()
+{
+MapString, String logLevelMaps = Maps.newLinkedHashMap();
+org.apache.log4j.Logger rootLogger = 
org.apache.log4j.Logger.getRootLogger();
+logLevelMaps.put(rootLogger.getName(), 
rootLogger.getLevel().toString());
+Enumerationorg.apache.log4j.Logger loggers = 
LogManager.getCurrentLoggers();
+while (loggers.hasMoreElements())
+{
+org.apache.log4j.Logger logger= loggers.nextElement();
+if (logger.getLevel() != null)
+logLevelMaps.put(logger.getName(), 
logger.getLevel().toString());
+}
+return logLevelMaps;
 }
 
 /**

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0a20f5f1/src/java/org/apache/cassandra/service/StorageServiceMBean.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageServiceMBean.java 
b/src/java/org/apache/cassandra/service/StorageServiceMBean.java
index 

[4/4] git commit: Merge branch 'cassandra-2.1' into trunk

2014-04-28 Thread vijay
Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2ef5647
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2ef5647
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2ef5647

Branch: refs/heads/trunk
Commit: b2ef56478d5be36d5e20932428698ed930b9b992
Parents: afe2e14 d402cf6
Author: Vijay vijay2...@gmail.com
Authored: Mon Apr 28 21:39:20 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Mon Apr 28 21:39:20 2014 -0700

--
 conf/logback.xml|  2 +-
 .../cassandra/service/StorageService.java   | 49 ++--
 .../cassandra/service/StorageServiceMBean.java  | 21 -
 .../org/apache/cassandra/tools/NodeProbe.java   | 17 +++
 .../org/apache/cassandra/tools/NodeTool.java| 32 -
 5 files changed, 114 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2ef5647/src/java/org/apache/cassandra/service/StorageService.java
--



[3/4] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

2014-04-28 Thread vijay
Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
src/java/org/apache/cassandra/service/StorageService.java
src/java/org/apache/cassandra/tools/NodeCmd.java
src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d402cf68
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d402cf68
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d402cf68

Branch: refs/heads/trunk
Commit: d402cf687001390cf81307e2d9d45a87b24f5837
Parents: 24eeeb9 0a20f5f
Author: Vijay vijay2...@gmail.com
Authored: Mon Apr 28 21:38:22 2014 -0700
Committer: Vijay vijay2...@gmail.com
Committed: Mon Apr 28 21:38:22 2014 -0700

--
 conf/logback.xml|  2 +-
 .../cassandra/service/StorageService.java   | 49 ++--
 .../cassandra/service/StorageServiceMBean.java  | 21 -
 .../org/apache/cassandra/tools/NodeProbe.java   | 17 +++
 .../org/apache/cassandra/tools/NodeTool.java| 32 -
 5 files changed, 114 insertions(+), 7 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/d402cf68/conf/logback.xml
--
diff --cc conf/logback.xml
index 655bcf6,000..2657174
mode 100644,00..100644
--- a/conf/logback.xml
+++ b/conf/logback.xml
@@@ -1,34 -1,0 +1,34 @@@
 +configuration scan=true
-
++  jmxConfigurator /
 +  appender name=FILE 
class=ch.qos.logback.core.rolling.RollingFileAppender
 +file/var/log/cassandra/system.log/file
 +rollingPolicy 
class=ch.qos.logback.core.rolling.FixedWindowRollingPolicy
 +  fileNamePattern/var/log/cassandra/system.log.%i.zip/fileNamePattern
 +  minIndex1/minIndex
 +  maxIndex20/maxIndex
 +/rollingPolicy
 +
 +triggeringPolicy 
class=ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy
 +  maxFileSize20MB/maxFileSize
 +/triggeringPolicy
 +encoder
 +  pattern%-5level [%thread] %date{ISO8601} %F:%L - %msg%n/pattern
 +  !-- old-style log format
 +  pattern%5level [%thread] %date{ISO8601} %F (line %L) %msg%n/pattern
 +  --
 +/encoder
 +  /appender
 +  
 +  appender name=STDOUT class=ch.qos.logback.core.ConsoleAppender
 +encoder
 +  pattern%-5level %date{HH:mm:ss,SSS} %msg%n/pattern
 +/encoder
 +  /appender
 +
 +  root level=INFO
 +appender-ref ref=FILE /
 +appender-ref ref=STDOUT /
 +  /root
 +  
 +  logger name=com.thinkaurelius.thrift level=ERROR/
 +/configuration

http://git-wip-us.apache.org/repos/asf/cassandra/blob/d402cf68/src/java/org/apache/cassandra/service/StorageService.java
--
diff --cc src/java/org/apache/cassandra/service/StorageService.java
index a9c0233,f44eaed..a284ab4
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@@ -29,14 -29,11 +29,20 @@@ import java.util.*
  import java.util.concurrent.*;
  import java.util.concurrent.atomic.AtomicInteger;
  import java.util.concurrent.atomic.AtomicLong;
 +
++import javax.management.JMX;
  import javax.management.MBeanServer;
  import javax.management.Notification;
  import javax.management.NotificationBroadcasterSupport;
  import javax.management.ObjectName;
 +import javax.management.openmbean.TabularData;
 +import javax.management.openmbean.TabularDataSupport;
 +
++import ch.qos.logback.classic.LoggerContext;
++import ch.qos.logback.classic.jmx.JMXConfiguratorMBean;
++import ch.qos.logback.classic.spi.ILoggingEvent;
++import ch.qos.logback.core.Appender;
+ 
  import com.google.common.annotations.VisibleForTesting;
  import com.google.common.base.Predicate;
  import com.google.common.collect.*;
@@@ -44,9 -42,10 +50,8 @@@ import com.google.common.util.concurren
  import com.google.common.util.concurrent.Futures;
  import com.google.common.util.concurrent.Uninterruptibles;
  
--import org.apache.cassandra.cql3.CQL3Type;
  import org.apache.commons.lang3.StringUtils;
 -import org.apache.log4j.Level;
 -import org.apache.log4j.LogManager;
 +
  import org.slf4j.Logger;
  import org.slf4j.LoggerFactory;
  
@@@ -2874,15 -2793,49 +2879,53 @@@ public class StorageService extends Not
  return liveEps;
  }
  
- public void setLoggingLevel(String classQualifier, String rawLevel)
 -public void setLog4jLevel(String classQualifier, String rawLevel)
++public void setLoggingLevel(String classQualifier, String rawLevel) 
throws Exception
  {
 -org.apache.log4j.Logger log4jlogger = 
org.apache.log4j.Logger.getLogger(classQualifier);
 +ch.qos.logback.classic.Logger logBackLogger = 
(ch.qos.logback.classic.Logger) 

[jira] [Updated] (CASSANDRA-7106) Retry multiple times before giving up on fetching splits

2014-04-28 Thread Victor Igumnov (JIRA)

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Victor Igumnov updated CASSANDRA-7106:
--

Attachment: retries-fix-iterator-bug.patch

 Retry multiple times before giving up on fetching splits
 

 Key: CASSANDRA-7106
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7106
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Victor Igumnov
Priority: Minor
 Fix For: 2.1 beta1

 Attachments: retries-fix-iterator-bug.patch, retries-reworked.patch, 
 retry-splits.patch


 See Github for patch:
 https://github.com/victori/cassandra
 What the patch does:
 While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
 get any of the queries to return reliably. Looking further into the issue I 
 noticed that if just a single fetch for a split fails the whole job is 
 aborted. So instead of giving up and failing out the whole job, retry 
 fetching the data a few times before completely giving up.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)


[jira] [Commented] (CASSANDRA-7106) Retry multiple times before giving up on fetching splits

2014-04-28 Thread Victor Igumnov (JIRA)

[ 
https://issues.apache.org/jira/browse/CASSANDRA-7106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13984007#comment-13984007
 ] 

Victor Igumnov commented on CASSANDRA-7106:
---

Fixed an issue with my last patch, I should have made a copy of the key set to 
avoid a concurrent modification exception when modifying the collection. 

retries-fix-iterator-bug.patch should be the patch that is reviewed.

- Victor

 Retry multiple times before giving up on fetching splits
 

 Key: CASSANDRA-7106
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7106
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Victor Igumnov
Priority: Minor
 Fix For: 2.1 beta1

 Attachments: retries-fix-iterator-bug.patch, retries-reworked.patch, 
 retry-splits.patch


 See Github for patch:
 https://github.com/victori/cassandra
 What the patch does:
 While I was evaluating Shark/Hive for analyzing Cassandra's data, I could not 
 get any of the queries to return reliably. Looking further into the issue I 
 noticed that if just a single fetch for a split fails the whole job is 
 aborted. So instead of giving up and failing out the whole job, retry 
 fetching the data a few times before completely giving up.  



--
This message was sent by Atlassian JIRA
(v6.2#6252)