date:20150317


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365266#comment-14365266
 ] 

Jonathan Ellis commented on CASSANDRA-8568:
---

SGTM.

 Impose new API on data tracker modifications that makes correct usage obvious 
 and imposes safety
 

 Key: CASSANDRA-8568
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8568
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict

 DataTracker has become a bit of a quagmire, and not at all obvious to 
 interface with, with many subtly different modifiers. I suspect it is still 
 subtly broken, especially around error recovery.
 I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for 
 those situations where a try/finally block isn't possible) objects that have 
 transactional behaviour, and with few simple declarative methods that can be 
 composed simply to provide all of the functionality we currently need.
 See CASSANDRA-8399 for context



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8851) Uncaught exception on thread Thread[SharedPool-Worker-16,5,main] after upgrade to 2.1.3


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365360#comment-14365360
 ] 

Benedict commented on CASSANDRA-8851:
-

Hi [~Hachmann]. Thanks, could you please provide the schema and your full 
system log (at least as far as a couple of hours prior to the very first 
occurrence of the exception)?

 Uncaught exception on thread Thread[SharedPool-Worker-16,5,main] after 
 upgrade to 2.1.3
 ---

 Key: CASSANDRA-8851
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8851
 Project: Cassandra
  Issue Type: Bug
 Environment: ubuntu 
Reporter: Tobias Schlottke
Assignee: Benedict
Priority: Critical
 Fix For: 2.1.4


 Hi there,
 after upgrading to 2.1.3 we've got the following error every few seconds:
 {code}
 WARN  [SharedPool-Worker-16] 2015-02-23 10:20:36,392 
 AbstractTracingAwareExecutorService.java:169 - Uncaught exception on thread 
 Thread[SharedPool-Worker-16,5,main]: {}
 java.lang.AssertionError: null
   at org.apache.cassandra.io.util.Memory.size(Memory.java:307) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.utils.obs.OffHeapBitSet.capacity(OffHeapBitSet.java:61) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
   at org.apache.cassandra.utils.BloomFilter.indexes(BloomFilter.java:74) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.utils.BloomFilter.isPresent(BloomFilter.java:98) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:1366)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:1350)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.columniterator.SSTableSliceIterator.init(SSTableSliceIterator.java:41)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:185)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:62)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:273)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:62)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1915)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1748)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:342) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:57)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) 
 ~[apache-cassandra-2.1.3.jar:2.1.3]
   at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
 ~[na:1.7.0_45]
   at 
 org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
   at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) 
 [apache-cassandra-2.1.3.jar:2.1.3]
   at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
 {code}
 This seems to crash the compactions and pushes up server load and piles up 
 compactions.
 Any idea / possible workaround?
 Best,
 Tobias



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8977) Potential DOS Exposure from Netty

2015-03-17 Thread Brandon Williams (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365311#comment-14365311
 ] 

Brandon Williams commented on CASSANDRA-8977:
-

I really don't like changing deps in a minor, but I guess for a vulnerability 
we should consider it (but really if you expose the ports to an attacker, 
you're an idiot.)  [~enigmacurry] can your team upgrade and test please?

 Potential DOS Exposure from Netty
 -

 Key: CASSANDRA-8977
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8977
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benjamin Coverston
Assignee: Brandon Williams

 CVE-2014-3488 : The SslHandler in Netty before 3.9.2 allows remote attackers 
 to cause a denial of service (infinite loop and CPU consumption) via a 
 crafted SSLv2Hello message.
 2.0 currently uses Netty 3.6.6 
 Upgrading to 3.9.2+ will fix this issue, and patch a few other known 
 vulnerabilities that we do not currently expose.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: ninja-fix UFTest/CQLTester

Repository: cassandra
Updated Branches:
  refs/heads/trunk c9a6d7ce0 - ac3062188


ninja-fix UFTest/CQLTester


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ac306218
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ac306218
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ac306218

Branch: refs/heads/trunk
Commit: ac306218848f62be71db8cea5ce90c4835c626c6
Parents: c9a6d7c
Author: Robert Stupp sn...@snazy.de
Authored: Tue Mar 17 16:24:57 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Tue Mar 17 16:24:57 2015 +0100

--
 .../org/apache/cassandra/cql3/CQLTester.java| 27 
 test/unit/org/apache/cassandra/cql3/UFTest.java | 20 +++
 2 files changed, 32 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac306218/test/unit/org/apache/cassandra/cql3/CQLTester.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/CQLTester.java 
b/test/unit/org/apache/cassandra/cql3/CQLTester.java
index 831a8d7..d8914a9 100644
--- a/test/unit/org/apache/cassandra/cql3/CQLTester.java
+++ b/test/unit/org/apache/cassandra/cql3/CQLTester.java
@@ -75,11 +75,28 @@ public abstract class CQLTester
 private static org.apache.cassandra.transport.Server server;
 private static final int nativePort;
 private static final InetAddress nativeAddr;
-private static final Cluster cluster[] = new 
Cluster[Server.CURRENT_VERSION];
-private static final Session session[] = new 
Session[Server.CURRENT_VERSION];
+private static final Cluster[] cluster;
+private static final Session[] session;
+
+static int maxProtocolVersion;
+static {
+int version;
+for (version = 1; version = Server.CURRENT_VERSION; version++)
+{
+try
+{
+ProtocolVersion.fromInt(version);
+}
+catch (IllegalArgumentException e)
+{
+version--;
+break;
+}
+}
+maxProtocolVersion = version;
+cluster = new Cluster[maxProtocolVersion];
+session = new Session[maxProtocolVersion];
 
-static
-{
 // Once per-JVM is enough
 SchemaLoader.prepareServer();
 
@@ -210,7 +227,7 @@ public abstract class CQLTester
 server = new org.apache.cassandra.transport.Server(nativeAddr, 
nativePort);
 server.start();
 
-for (int version = 1; version = Server.CURRENT_VERSION; version++)
+for (int version = 1; version = maxProtocolVersion; version++)
 {
 if (cluster[version-1] != null)
 continue;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac306218/test/unit/org/apache/cassandra/cql3/UFTest.java
--
diff --git a/test/unit/org/apache/cassandra/cql3/UFTest.java 
b/test/unit/org/apache/cassandra/cql3/UFTest.java
index 84a7dd9..b3cc545 100644
--- a/test/unit/org/apache/cassandra/cql3/UFTest.java
+++ b/test/unit/org/apache/cassandra/cql3/UFTest.java
@@ -670,7 +670,7 @@ public class UFTest extends CQLTester
row(list, set, map));
 
 // same test - but via native protocol
-for (int version = Server.VERSION_2; version = 
Server.CURRENT_VERSION; version++)
+for (int version = Server.VERSION_2; version = maxProtocolVersion; 
version++)
 assertRowsNet(version,
   executeNet(version, SELECT  + fList + (lst),  + 
fSet + (st),  + fMap + (mp) FROM %s WHERE key = 1),
   row(list, set, map));
@@ -751,7 +751,7 @@ public class UFTest extends CQLTester
 Assert.assertNull(row.getBytes(t));
 Assert.assertNull(row.getBytes(u));
 
-for (int version = Server.VERSION_2; version = 
Server.CURRENT_VERSION; version++)
+for (int version = Server.VERSION_2; version = maxProtocolVersion; 
version++)
 {
 Row r = executeNet(version, SELECT  +
 fList + (lst) as l,  +
@@ -867,7 +867,7 @@ public class UFTest extends CQLTester
DataType.set(DataType.text()),
DataType.map(DataType.cint(), 
DataType.cboolean()));
 TupleValue tup = tType.newValue(1d, list, set, map);
-for (int version = Server.VERSION_2; version = 
Server.CURRENT_VERSION; version++)
+for (int version = Server.VERSION_2; version = maxProtocolVersion; 
version++)
 {
 assertRowsNet(version,
   executeNet(version, SELECT  + fTup0 + (tup) FROM 
%s WHERE key = 1),
@@ -894,7 +894,7 @@ public class

[jira] [Commented] (CASSANDRA-8977) Potential DOS Exposure from Netty

2015-03-17 Thread Ryan McGuire (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365317#comment-14365317
 ] 

Ryan McGuire commented on CASSANDRA-8977:
-

Sure.

 Potential DOS Exposure from Netty
 -

 Key: CASSANDRA-8977
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8977
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benjamin Coverston
Assignee: Brandon Williams

 CVE-2014-3488 : The SslHandler in Netty before 3.9.2 allows remote attackers 
 to cause a denial of service (infinite loop and CPU consumption) via a 
 crafted SSLv2Hello message.
 2.0 currently uses Netty 3.6.6 
 Upgrading to 3.9.2+ will fix this issue, and patch a few other known 
 vulnerabilities that we do not currently expose.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds

[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366586#comment-14366586
]

Jonathan Ellis commented on CASSANDRA-3852:
---

bq. It's worth pointing out this change won't necessarily help tail latencies,
it will just shift the tail

I think the idea is not so much to improve latencies, so much as to allow
*some* work to succeed under overload conditions, vs FIFO where it's possible
for every request to time out as it's processed. Taking the newest to work on
instead of the oldest avoids this scenario.

use LIFO queueing policy when queue size exceeds thresholds
---

Key: CASSANDRA-3852
URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
Project: Cassandra
Issue Type: Improvement
Reporter: Peter Schuller
Labels: performance
Fix For: 3.1

A strict FIFO policy for queueing (between stages) is detrimental to latency
and forward progress. Whenever a node is saturated beyond incoming request
rate, *all* requests become slow. If it is consistently saturated, you start
effectively timing out on *all* requests.
A much better strategy from the point of view of latency is to serve a subset
requests quickly, and letting some time out, rather than letting all either
time out or be slow.
Care must be taken such that:
* We still guarantee that requests are processed reasonably timely (we
couldn't go strict LIFO for example as that would result in requests getting
stuck potentially forever on a loaded node).
* Maybe, depending on the previous point's solution, ensure that some
requests bypass the policy and get prioritized (e.g., schema migrations, or
anything internal to a node).
A possible implementation is to go LIFO whenever there are requests in the
queue that are older than N milliseconds (or a certain queue size, etc).
Benefits:
* All cases where the client is directly, or is indirectly affecting through
other layers, a system which has limited concurrency (e.g., thread pool size
of X to serve some incoming request rate), it is *much* better for a few
requests to time out while most are serviced quickly, than for all requests
to become slow, as it doesn't explode concurrency. Think any random
non-super-advanced php app, ruby web app, java servlet based app, etc.
Essentially, it optimizes very heavily for improved average latencies.
* Systems with strict p95/p99/p999 requirements on latencies should greatly
benefit from such a policy. For example, suppose you have a system at 85% of
capacity, and it takes a write spike (or has a hiccup like GC pause, blocking
on a commit log write, etc). Suppose the hiccup racks up 500 ms worth of
requests. At 15% margin at steady state, that takes 500ms * 100/15 = 3.2
seconds to recover. Instead of *all* requests for an entire 3.2 second window
being slow, we'd serve requests quickly for 2.7 of those seconds, with the
incoming requests during that 500 ms interval being the ones primarily
affected. The flip side though is that once you're at the point where more
than N percent of requests end up having to wait for others to take LIFO
priority, the p(100-N) latencies will actually be *worse* than without this
change (but at this point you have to consider what the root reason for those
pXX requirements are).
* In the case of complete saturation, it allows forward progress. Suppose
you're taking 25% more traffic than you are able to handle. Instead of
getting backed up and ending up essentially timing out *every single
request*, you will succeed in processing up to 75% of them (I say up to
because it depends; for example on a {{QUORUM}} request you need at least two
of the requests from the co-ordinator to succeed so the percentage is brought
down) and allowing clients to make forward progress and get work done, rather
than being stuck.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming

2015-03-17 Thread Sylvain Lebresne (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364788#comment-14364788
]

Sylvain Lebresne commented on CASSANDRA-8838:
-

bq. The first issue is that starting a node with {{no_wait=True}} fails to
detect the process pid and the test stops there, this applies to all four tests.

I've noticed that too and this is not specific to this issue: there is quite a
few dtests that uses this and are currently failing on trunk on my box for the
reason you mention. It appears that on current trunk, on my box, the Cassandra
process takes 10 seconds to even get created (which is definitively something
relatively new). I haven't investigated what made that happen so it could be
nice to bisect it so we at least know, but as you said, removing the
{{no_wait}} flag at least fixes the tests.

Resumable bootstrap streaming
-

Key: CASSANDRA-8838
URL: https://issues.apache.org/jira/browse/CASSANDRA-8838
Project: Cassandra
Issue Type: Sub-task
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Minor
Labels: dense-storage
Fix For: 3.0

This allows the bootstrapping node not to be streamed already received data.
The bootstrapping node records received keyspace/ranges as one stream session
completes. When some sessions with other nodes fail, bootstrapping fails
also, though next time it re-bootstraps, already received keyspace/ranges are
skipped to be streamed.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8358) Bundled tools shouldn't be using Thrift API

[
https://issues.apache.org/jira/browse/CASSANDRA-8358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365885#comment-14365885
]

Alex Liu commented on CASSANDRA-8358:
-

When can this ticket committed to 2.1 branch?

Bundled tools shouldn't be using Thrift API
---

Key: CASSANDRA-8358
URL: https://issues.apache.org/jira/browse/CASSANDRA-8358
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Philip Thompson
Fix For: 3.0

In 2.1, we switched cqlsh to the python-driver.
In 3.0, we got rid of cassandra-cli.
Yet there is still code that's using legacy Thrift API. We want to convert it
all to use the java-driver instead.
1. BulkLoader uses Thrift to query the schema tables. It should be using
java-driver metadata APIs directly instead.
2. o.a.c.hadoop.cql3.CqlRecordWriter is using Thrift
3. o.a.c.hadoop.ColumnFamilyRecordReader is using Thrift
4. o.a.c.hadoop.AbstractCassandraStorage is using Thrift
5. o.a.c.hadoop.pig.CqlStorage is using Thrift
Some of the things listed above use Thrift to get the list of partition key
columns or clustering columns. Those should be converted to use the Metadata
API of the java-driver.
Somewhat related to that, we also have badly ported code from Thrift in
o.a.c.hadoop.cql3.CqlRecordReader (see fetchKeys()) that manually fetches
columns from schema tables instead of properly using the driver's Metadata
API.
We need all of it fixed. One exception, for now, is
o.a.c.hadoop.AbstractColumnFamilyInputFormat - it's using Thrift for its
describe_splits_ex() call that cannot be currently replaced by any
java-driver call (?).
Once this is done, we can stop starting Thrift RPC port by default in
cassandra.yaml.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Issue Comment Deleted] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alex Liu updated CASSANDRA-7410:

Comment: was deleted

(was: Waiting for CASSANDRA-8358)

 Pig support for BulkOutputFormat as a parameter in url
 --

 Key: CASSANDRA-7410
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
Priority: Minor
 Fix For: 2.0.14

 Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, 
 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, 
 CASSANDRA-7410-v2-2.1-branch.txt


 Add BulkOutputFormat support in Pig url



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7712) temporary files need to be cleaned by unit tests


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365859#comment-14365859
 ] 

Michael Shuler edited comment on CASSANDRA-7712 at 3/17/15 7:53 PM:


Attached 7712_workaround.txt patch to set a {{tmp.dir}} property and use that 
as a {{jvmarg}} in testlist.  Tested this with and without passing in the 
option with no problems - -the jvm also created my custom tmp.dir for me- (I 
must have done a run with my tmp.dir already created - we do need to mkdir it). 
 This workaround provides us the ability to set the temporary data dir within 
our git workspace, and since we are using {{git clean -xdf}} at the beginning 
of every unit test run, this data will no longer accumulate.


was (Author: mshuler):
Attached 7712_workaround.txt patch to set a {{tmp.dir}} property and use that 
as a {{jvmarg}} in testlist.  Tested this with and without passing in the 
option with no problems - the jvm also created my custom tmp.dir for me.  This 
workaround provides us the ability to set the temporary data dir within our git 
workspace, and since we are using {{git clean -xdf}} at the beginning of every 
unit test run, this data will no longer accumulate.

 temporary files need to be cleaned by unit tests
 

 Key: CASSANDRA-7712
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Michael Shuler
Assignee: Michael Shuler
Priority: Minor
  Labels: bootcamp, lhf
 Fix For: 2.1.4, 2.0.14

 Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt, 
 7712-v3.txt, 7712_workaround.txt, CASSANDRA-7712_apache_cassandra_2.0.txt


 There are many unit test temporary files left behind after test runs. In the 
 case of CI servers, I have seen 70,000 files accumulate in /tmp over a 
 period of time. Each unit test should make an effort to remove its temporary 
 files when the test is completed.
 My current unit test cleanup block:
 {noformat}
 # clean up after unit tests..
 rm -rf  /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest* 
 /tmp/Keyspace1* \
 /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest* 
 /tmp/SSTableImportTest* \
 /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest* 
 /tmp/ValuesWithQuotes* \
 /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest* 
 /tmp/liblz4-java*.so /tmp/readtest* \
 /tmp/set_length_during_read_mode* /tmp/set_negative_length* 
 /tmp/snappy-*.so
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6178) Consider allowing timestamp at the protocol level ... and deprecating server side timestamps

2015-03-17 Thread Jon Haddad (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366136#comment-14366136
]

Jon Haddad commented on CASSANDRA-6178:
---

I would argue there's a valid use case for being able to disable client side
timestamps and only use server side ones.

Consider allowing timestamp at the protocol level ... and deprecating server
side timestamps

Key: CASSANDRA-6178
URL: https://issues.apache.org/jira/browse/CASSANDRA-6178
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne

Generating timestamps server side by default for CQL has been done for
convenience, so that end-user don't have to provide one with every query.
However, doing it server side has the downside that updates made sequentially
by one single client (thread) are no guaranteed to have sequentially
increasing timestamps. Unless a client thread is always pinned to one
specific server connection that is, but no good client driver out (that is,
including thrit driver) there does that because that's contradictory to
abstracting fault tolerance to the driver user (and goes again most sane load
balancing strategy).
Very concretely, this means that if you write a very trivial test program
that sequentially insert a value and then erase it (or overwrite it), then,
if you let CQL pick timestamp server side, the deletion might not erase the
just inserted value (because the delete might reach a different coordinator
than the insert and thus get a lower timestamp). From the user point of view,
this is a very confusing behavior, and understandably so: if timestamps are
optional, you'd hope that they are least respect the sequentiality of
operation from a single client thread.
Of course we do support client-side assigned timestamps so it's not like the
test above is not fixable. And you could argue that's it's not a bug per-se.
Still, it's a very confusing default behavior for something very simple,
which suggest it's not the best default.
You could also argue that inserting a value and deleting/overwriting right
away (in the same thread) is not something real program often do. And indeed,
it's likely that in practice server-side timestamps work fine for most real
application. Still, it's too easy to get counter-intuitive behavior with
server-side timestamps and I think we should consider moving away from them.
So what I'd suggest is that we push back the job of providing timestamp
client side. But to make it easy for the driver to generate it (rather than
the end user), we should allow providing said timestamp at the protocol level.
As a side note, letting the client provide the timestamp would also have the
advantage of making it easy for the driver to retry failed operations with
their initial timestamp, so that retries are truly idempotent.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-6178) Consider allowing timestamp at the protocol level ... and deprecating server side timestamps

2015-03-17 Thread Jon Haddad (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-6178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366136#comment-14366136
]

Jon Haddad edited comment on CASSANDRA-6178 at 3/17/15 9:39 PM:

I would argue there's a valid use case for being able to disable client side
timestamps and only use server side ones, which would put me in the camp of
being very against this issue.

was (Author: rustyrazorblade):
I would argue there's a valid use case for being able to disable client side
timestamps and only use server side ones.

Consider allowing timestamp at the protocol level ... and deprecating server
side timestamps

Key: CASSANDRA-6178
URL: https://issues.apache.org/jira/browse/CASSANDRA-6178
Project: Cassandra
Issue Type: Improvement
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8940) Inconsistent select count and select distinct

2015-03-17 Thread Frens Jan Rumph (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frens Jan Rumph updated CASSANDRA-8940:
---
  Environment: 2.1.2
Fix Version/s: (was: 2.1.2)

 Inconsistent select count and select distinct
 -

 Key: CASSANDRA-8940
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8940
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: 2.1.2
Reporter: Frens Jan Rumph

 When performing {{select count( * ) from ...}} I expect the results to be 
 consistent over multiple query executions if the table at hand is not written 
 to / deleted from in the mean time. However, in my set-up it is not. The 
 counts returned vary considerable (several percent). The same holds for 
 {{select distinct partition-key-columns from ...}}.
 I have a table in a keyspace with replication_factor = 1 which is something 
 like:
 {code}
 CREATE TABLE tbl (
 id frozenid_type,
 bucket bigint,
 offset int,
 value double,
 PRIMARY KEY ((id, bucket), offset)
 )
 {code}
 The frozen udt is:
 {code}
 CREATE TYPE id_type (
 tags maptext, text
 );
 {code}
 The table contains around 35k rows (I'm not trying to be funny here ...). The 
 consistency level for the queries was ONE.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: DatabaseDescriptor throws NPE when rpc_interface is used

DatabaseDescriptor throws NPE when rpc_interface is used

patch by Ariel Weisberg; reviewed by Carl Yeksigian for CASSANDRA-8839


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89cdfd8e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89cdfd8e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89cdfd8e

Branch: refs/heads/trunk
Commit: 89cdfd8e075d8883d776d7f881735f1c25e3cb54
Parents: c0f159b
Author: Ariel Weisberg ar...@weisberg.ws
Authored: Tue Mar 17 19:27:54 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Tue Mar 17 19:27:54 2015 +0100

--
 CHANGES.txt |   1 +
 conf/cassandra.yaml |  12 ++
 .../org/apache/cassandra/config/Config.java |   6 +-
 .../cassandra/config/DatabaseDescriptor.java| 205 +++
 .../config/YamlConfigurationLoader.java |  16 +-
 .../config/DatabaseDescriptorTest.java  | 139 -
 6 files changed, 277 insertions(+), 102 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7b8e0ad..30bf698 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.4
+ * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
  * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
  * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
  * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index cea12b3..2b43ba7 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -370,8 +370,14 @@ ssl_storage_port: 7001
 # address associated with the hostname (it might not be).
 #
 # Setting listen_address to 0.0.0.0 is always wrong.
+#
+# If you choose to specify the interface by name and the interface has an ipv4 
and an ipv6 address
+# you can specify which should be chosen using listen_interface_prefer_ipv6. 
If false the first ipv4
+# address will be used. If true the first ipv6 address will be used. Defaults 
to false preferring
+# ipv4. If there is only one address it will be selected regardless of 
ipv4/ipv6.
 listen_address: localhost
 # listen_interface: eth0
+# listen_interface_prefer_ipv6: false
 
 # Address to broadcast to other Cassandra nodes
 # Leaving this blank will set it to the same value as listen_address
@@ -422,8 +428,14 @@ start_rpc: true
 # set broadcast_rpc_address to a value other than 0.0.0.0.
 #
 # For security reasons, you should not expose this port to the internet.  
Firewall it if needed.
+#
+# If you choose to specify the interface by name and the interface has an ipv4 
and an ipv6 address
+# you can specify which should be chosen using rpc_interface_prefer_ipv6. If 
false the first ipv4
+# address will be used. If true the first ipv6 address will be used. Defaults 
to false preferring
+# ipv4. If there is only one address it will be selected regardless of 
ipv4/ipv6.
 rpc_address: localhost
 # rpc_interface: eth1
+# rpc_interface_prefer_ipv6: false
 
 # port for Thrift to listen for clients on
 rpc_port: 9160

http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ccd4467..fbbd1dd 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -34,7 +34,7 @@ import org.apache.cassandra.utils.FBUtilities;
 
 /**
  * A class that contains configuration properties for the cassandra node it 
runs within.
- * 
+ *
  * Properties declared as volatile can be mutated via JMX.
  */
 public class Config
@@ -101,12 +101,14 @@ public class Config
 public Integer ssl_storage_port = 7001;
 public String listen_address;
 public String listen_interface;
+public Boolean listen_interface_prefer_ipv6 = false;
 public String broadcast_address;
 public String internode_authenticator;
 
 public Boolean start_rpc = true;
 public String rpc_address;
 public String rpc_interface;
+public Boolean rpc_interface_prefer_ipv6 = false;
 public String broadcast_rpc_address;
 public Integer rpc_port = 9160;
 public Integer rpc_listen_backlog = 50;
@@ -155,7 +157,7 @@ public class Config
 public Double commitlog_sync_batch_window_in_ms;
 public Integer

[jira] [Created] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout

2015-03-17 Thread Tom Alexander (JIRA)

Tom Alexander created CASSANDRA-8982:


 Summary: sstableloader outputs progress information and summary 
stats to stderr instead of stdout
 Key: CASSANDRA-8982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 12.04.3 LTS
Cassandra 2.1.3
Reporter: Tom Alexander


On Cassandra 2.1.0+, the progress information from sstableloader is output into 
the stderr stream instead of stdout. 

This can be reproduced by running an sstableloader and redirecting stderr to a 
file:

{code}
sstableloader -d 127.0.0.1 
/home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
 2 error.log
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8670) Large columns + NIO memory pooling causes excessive direct memory usage

2015-03-17 Thread Ariel Weisberg (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366114#comment-14366114
]

Ariel Weisberg commented on CASSANDRA-8670:
---

I can confirm that compression determines whether you see an issue with memory
used by intracluster messaging.

The memory used by Netty shouldn't scale with cluster size the same way.

Still figuring out how to test for the issue when Netty is dominating direct
bytebuffer usage. I don't see a way to get Netty to report on it's memory usage
nor a way to get the allocations done by NIO tracked.

Monkey patching where art thou?

Large columns + NIO memory pooling causes excessive direct memory usage
---

Key: CASSANDRA-8670
URL: https://issues.apache.org/jira/browse/CASSANDRA-8670
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
Fix For: 3.0

If you provide a large byte array to NIO and ask it to populate the byte
array from a socket it will allocate a thread local byte buffer that is the
size of the requested read no matter how large it is. Old IO wraps new IO for
sockets (but not files) so old IO is effected as well.
Even If you are using Buffered{Input | Output}Stream you can end up passing a
large byte array to NIO. The byte array read method will pass the array to
NIO directly if it is larger than the internal buffer.
Passing large cells between nodes as part of intra-cluster messaging can
cause the NIO pooled buffers to quickly reach a high watermark and stay
there. This ends up costing 2x the largest cell size because there is a
buffer for input and output since they are different threads. This is further
multiplied by the number of nodes in the cluster - 1 since each has a
dedicated thread pair with separate thread locals.
Anecdotally it appears that the cost is doubled beyond that although it isn't
clear why. Possibly the control connections or possibly there is some way in
which multiple
Need a workload in CI that tests the advertised limits of cells on a cluster.
It would be reasonable to ratchet down the max direct memory for the test to
trigger failures if a memory pooling issue is introduced. I don't think we
need to test concurrently pulling in a lot of them, but it should at least
work serially.
The obvious fix to address this issue would be to read in smaller chunks when
dealing with large values. I think small should still be relatively large (4
megabytes) so that code that is reading from a disk can amortize the cost of
a seek. It can be hard to tell what the underlying thing being read from is
going to be in some of the contexts where we might choose to implement
switching to reading chunks.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-6809) Compressed Commit Log

2015-03-17 Thread Alan Boudreault (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Boudreault updated CASSANDRA-6809:
---
Tester: Alan Boudreault

 Compressed Commit Log
 -

 Key: CASSANDRA-6809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
  Labels: docs-impacting, performance
 Fix For: 3.0

 Attachments: ComitLogStress.java, logtest.txt


 It seems an unnecessary oversight that we don't compress the commit log. 
 Doing so should improve throughput, but some care will need to be taken to 
 ensure we use as much of a segment as possible. I propose decoupling the 
 writing of the records from the segments. Basically write into a (queue of) 
 DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X 
 MB written to the CL (where X is ordinarily CLS size), and then pack as many 
 of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS

2015-03-17 Thread Dave Brosius (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius updated CASSANDRA-8976:

Attachment: binary.txt

 Remove BINARY from DROPPABLE_VERBS
 --

 Key: CASSANDRA-8976
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
 Project: Cassandra
  Issue Type: Improvement
Reporter: Chris Lohfink
Assignee: Dave Brosius
Priority: Trivial
  Labels: lhf
 Fix For: 3.0

 Attachments: binary.txt


 The deprecated {{Verb.BINARY}} type has been listed in 
 {{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.  
 This is pretty irrelevant but it shows up in results to {{nodetool tpstats}} 
 which may seem to be related to the binary protocol.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/5] cassandra git commit: Match estimated tasks arithmetic to score in LCS

Match estimated tasks arithmetic to score in LCS

Patch by carlyeks; reviewed by marcuse for CASSANDRA-8904


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08147376
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08147376
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08147376

Branch: refs/heads/trunk
Commit: 08147376aa16570c07a4522931cc8fa330519d3e
Parents: 31a1828
Author: Carl Yeksigian c...@apache.org
Authored: Mon Mar 16 14:54:08 2015 +
Committer: Marcus Eriksson marc...@apache.org
Committed: Tue Mar 17 11:10:14 2015 +0100

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8843908..65f2da4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Improve compaction estimated tasks estimation (CASSANDRA-8904)
  * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
  * Expose commit log archive status via JMX (CASSANDRA-8734)
  * Provide better exceptions for invalid replication strategy parameters

http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index aefd573..6554eb2 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -623,7 +623,8 @@ public class LeveledManifest
 for (int i = generations.length - 1; i = 0; i--)
 {
 ListSSTableReader sstables = generations[i];
-estimated[i] = Math.max(0L, SSTableReader.getTotalBytes(sstables) 
- maxBytesForLevel(i)) / maxSSTableSizeInBytes;
+   // If there is 1 byte over TBL - (MBL * 1.001), there is still a 
task left, so we need to round up.
+   estimated[i] = (long)Math.ceil((double)Math.max(0L, 
SSTableReader.getTotalBytes(sstables) - (long)(maxBytesForLevel(i) * 1.001)) / 
(double)maxSSTableSizeInBytes);
 tasks += estimated[i];
 }

[2/5] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89f2c1de
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89f2c1de
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89f2c1de

Branch: refs/heads/trunk
Commit: 89f2c1de63a75871367aee0f22985c8d2c4f9930
Parents: 1a77a69 31a1828
Author: T Jake Luciani j...@apache.org
Authored: Mon Mar 16 11:49:39 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Mon Mar 16 11:49:39 2015 -0400

--

--

[4/5] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/LeveledManifest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f159b8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f159b8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f159b8

Branch: refs/heads/trunk
Commit: c0f159b82a9d805990818f60735623e2a7d8c517
Parents: 89f2c1d 0814737
Author: Marcus Eriksson marc...@apache.org
Authored: Tue Mar 17 11:13:40 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Tue Mar 17 11:13:40 2015 +0100

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/CHANGES.txt
--
diff --cc CHANGES.txt
index a48787f,65f2da4..7b8e0ad
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,49 -1,5 +1,50 @@@
 -2.0.14:
 +2.1.4
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
 + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
 +Merged from 2.0:
+  * Improve compaction estimated tasks estimation (CASSANDRA-8904)
   * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
   * Expose commit log archive status via JMX (CASSANDRA-8734)
   * Provide better exceptions for invalid replication strategy parameters

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index f2af848,6554eb2..c076a64
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++

[5/5] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0b4cce86
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0b4cce86
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0b4cce86

Branch: refs/heads/trunk
Commit: 0b4cce86a222c4e9d9a0b32567ab16da519d87d3
Parents: d182fa1 c0f159b
Author: Marcus Eriksson marc...@apache.org
Authored: Tue Mar 17 11:14:53 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Tue Mar 17 11:14:53 2015 +0100

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0b4cce86/CHANGES.txt
--
diff --cc CHANGES.txt
index ef2a7c7,7b8e0ad..be57f1b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -113,7 -44,7 +113,8 @@@
   * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
   * Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
  Merged from 2.0:
 +2.0.14:
+  * Improve compaction estimated tasks estimation (CASSANDRA-8904)
   * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
   * Expose commit log archive status via JMX (CASSANDRA-8734)
   * Provide better exceptions for invalid replication strategy parameters

http://git-wip-us.apache.org/repos/asf/cassandra/blob/0b4cce86/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--

[1/5] cassandra git commit: bump versions

Repository: cassandra
Updated Branches:
  refs/heads/trunk d182fa1b4 - 0b4cce86a


bump versions


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/31a18287
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/31a18287
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/31a18287

Branch: refs/heads/trunk
Commit: 31a182876eb5ddf7eef2e40fbbe232c236413671
Parents: 2199a87
Author: T Jake Luciani j...@apache.org
Authored: Thu Mar 12 21:01:11 2015 -0400
Committer: T Jake Luciani j...@apache.org
Committed: Mon Mar 16 11:48:11 2015 -0400

--
 build.xml| 2 +-
 debian/changelog | 6 ++
 2 files changed, 7 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/31a18287/build.xml
--
diff --git a/build.xml b/build.xml
index 78c6db3..bdbd10a 100644
--- a/build.xml
+++ b/build.xml
@@ -25,7 +25,7 @@
 property name=debuglevel value=source,lines,vars/
 
 !-- default version and SCM information --
-property name=base.version value=2.0.12/
+property name=base.version value=2.0.13/
 property name=scm.connection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.developerConnection 
value=scm:git://git.apache.org/cassandra.git/
 property name=scm.url 
value=http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree/

http://git-wip-us.apache.org/repos/asf/cassandra/blob/31a18287/debian/changelog
--
diff --git a/debian/changelog b/debian/changelog
index 53fa20f..2c80800 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+cassandra (2.0.13) unstable; urgency=medium
+
+  * New release
+
+ -- Jake Luciani j...@apache.org  Thu, 12 Mar 2015 21:00:06 -0400
+
 cassandra (2.0.12) unstable; urgency=medium
 
   * New release

[jira] [Comment Edited] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS

2015-03-17 Thread Dave Brosius (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364806#comment-14364806
]

Dave Brosius edited comment on CASSANDRA-8976 at 3/17/15 9:22 AM:
--

Verb.BINARY has not been used at all since 1.1. One assumes you are not allowed
to run mixed 1.1/3.0 cluster, and so it can be removed from DROPPABLE_VERBS but
it would seem the enum itself has to remain otherwise it will mess up 2.1 -
3.0 serialization of Verbs, as serialization is done by ordinal.

so the patch removes it from DROPPABLE_VERBS.

was (Author: dbrosius):
Verb.BINARY has not been used at all since 1.1. One assumes you are not allowed
to run mixed 1.1/3.0 cluster, and so it can be removed from DROPPABLE_VERBS but
it would seem the enum itself has to remain otherwise it will mess up
serialization of 2.1 - 3.0 Verbs, as serialization is done by ordinal.

so the patch removes it from DROPPABLE_VERBS.

Remove BINARY from DROPPABLE_VERBS
--

Key: CASSANDRA-8976
URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
Project: Cassandra
Issue Type: Improvement
Reporter: Chris Lohfink
Assignee: Dave Brosius
Priority: Trivial
Labels: lhf
Fix For: 3.0

Attachments: binary.txt

The deprecated {{Verb.BINARY}} type has been listed in
{{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.
This is pretty irrelevant but it shows up in results to {{nodetool tpstats}}
which may seem to be related to the binary protocol.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/2] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt
src/java/org/apache/cassandra/db/compaction/LeveledManifest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f159b8
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f159b8
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f159b8

Branch: refs/heads/cassandra-2.1
Commit: c0f159b82a9d805990818f60735623e2a7d8c517
Parents: 89f2c1d 0814737
Author: Marcus Eriksson marc...@apache.org
Authored: Tue Mar 17 11:13:40 2015 +0100
Committer: Marcus Eriksson marc...@apache.org
Committed: Tue Mar 17 11:13:40 2015 +0100

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/CHANGES.txt
--
diff --cc CHANGES.txt
index a48787f,65f2da4..7b8e0ad
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,49 -1,5 +1,50 @@@
 -2.0.14:
 +2.1.4
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
 + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
 +Merged from 2.0:
+  * Improve compaction estimated tasks estimation (CASSANDRA-8904)
   * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
   * Expose commit log archive status via JMX (CASSANDRA-8734)
   * Provide better exceptions for invalid replication strategy parameters

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f159b8/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --cc src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index f2af848,6554eb2..c076a64
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++

cassandra git commit: Match estimated tasks arithmetic to score in LCS

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 31a182876 - 08147376a


Match estimated tasks arithmetic to score in LCS

Patch by carlyeks; reviewed by marcuse for CASSANDRA-8904


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08147376
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08147376
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08147376

Branch: refs/heads/cassandra-2.0
Commit: 08147376aa16570c07a4522931cc8fa330519d3e
Parents: 31a1828
Author: Carl Yeksigian c...@apache.org
Authored: Mon Mar 16 14:54:08 2015 +
Committer: Marcus Eriksson marc...@apache.org
Committed: Tue Mar 17 11:10:14 2015 +0100

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8843908..65f2da4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Improve compaction estimated tasks estimation (CASSANDRA-8904)
  * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
  * Expose commit log archive status via JMX (CASSANDRA-8734)
  * Provide better exceptions for invalid replication strategy parameters

http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index aefd573..6554eb2 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -623,7 +623,8 @@ public class LeveledManifest
 for (int i = generations.length - 1; i = 0; i--)
 {
 ListSSTableReader sstables = generations[i];
-estimated[i] = Math.max(0L, SSTableReader.getTotalBytes(sstables) 
- maxBytesForLevel(i)) / maxSSTableSizeInBytes;
+   // If there is 1 byte over TBL - (MBL * 1.001), there is still a 
task left, so we need to round up.
+   estimated[i] = (long)Math.ceil((double)Math.max(0L, 
SSTableReader.getTotalBytes(sstables) - (long)(maxBytesForLevel(i) * 1.001)) / 
(double)maxSSTableSizeInBytes);
 tasks += estimated[i];
 }

[jira] [Commented] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS

2015-03-17 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364835#comment-14364835
 ] 

Aleksey Yeschenko commented on CASSANDRA-8976:
--

+1

 Remove BINARY from DROPPABLE_VERBS
 --

 Key: CASSANDRA-8976
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
 Project: Cassandra
  Issue Type: Improvement
Reporter: Chris Lohfink
Assignee: Dave Brosius
Priority: Trivial
  Labels: lhf
 Fix For: 3.0

 Attachments: binary.txt


 The deprecated {{Verb.BINARY}} type has been listed in 
 {{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.  
 This is pretty irrelevant but it shows up in results to {{nodetool tpstats}} 
 which may seem to be related to the binary protocol.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8976) Remove BINARY from DROPPABLE_VERBS

2015-03-17 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-8976:
-
Reviewer: Aleksey Yeschenko

 Remove BINARY from DROPPABLE_VERBS
 --

 Key: CASSANDRA-8976
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8976
 Project: Cassandra
  Issue Type: Improvement
Reporter: Chris Lohfink
Assignee: Dave Brosius
Priority: Trivial
  Labels: lhf
 Fix For: 3.0

 Attachments: binary.txt


 The deprecated {{Verb.BINARY}} type has been listed in 
 {{MessagingService.DROPPABLE_VERBS}} for awhile even though it has no value.  
 This is pretty irrelevant but it shows up in results to {{nodetool tpstats}} 
 which may seem to be related to the binary protocol.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8561) Tombstone log warning does not log partition key

2015-03-17 Thread Lyuben Todorov (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lyuben Todorov updated CASSANDRA-8561:
--
Attachment: cassandra-2.1-8561.diff

 Tombstone log warning does not log partition key
 

 Key: CASSANDRA-8561
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8561
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
 Environment: Datastax DSE 4.5
Reporter: Jens Rantil
Assignee: Lyuben Todorov
  Labels: logging
 Fix For: 2.1.4

 Attachments: cassandra-2.1-8561.diff


 AFAIK, the tombstone warning in system.log does not contain the primary key. 
 See: https://gist.github.com/JensRantil/44204676f4dbea79ea3a
 Including it would help a lot in diagnosing why the (CQL) row has so many 
 tombstones.
 Let me know if I have misunderstood something.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/2] cassandra git commit: Match estimated tasks arithmetic to score in LCS

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 89f2c1de6 - c0f159b82


Match estimated tasks arithmetic to score in LCS

Patch by carlyeks; reviewed by marcuse for CASSANDRA-8904


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/08147376
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/08147376
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/08147376

Branch: refs/heads/cassandra-2.1
Commit: 08147376aa16570c07a4522931cc8fa330519d3e
Parents: 31a1828
Author: Carl Yeksigian c...@apache.org
Authored: Mon Mar 16 14:54:08 2015 +
Committer: Marcus Eriksson marc...@apache.org
Committed: Tue Mar 17 11:10:14 2015 +0100

--
 CHANGES.txt  | 1 +
 src/java/org/apache/cassandra/db/compaction/LeveledManifest.java | 3 ++-
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 8843908..65f2da4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Improve compaction estimated tasks estimation (CASSANDRA-8904)
  * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
  * Expose commit log archive status via JMX (CASSANDRA-8734)
  * Provide better exceptions for invalid replication strategy parameters

http://git-wip-us.apache.org/repos/asf/cassandra/blob/08147376/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index aefd573..6554eb2 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -623,7 +623,8 @@ public class LeveledManifest
 for (int i = generations.length - 1; i = 0; i--)
 {
 ListSSTableReader sstables = generations[i];
-estimated[i] = Math.max(0L, SSTableReader.getTotalBytes(sstables) 
- maxBytesForLevel(i)) / maxSSTableSizeInBytes;
+   // If there is 1 byte over TBL - (MBL * 1.001), there is still a 
task left, so we need to round up.
+   estimated[i] = (long)Math.ceil((double)Math.max(0L, 
SSTableReader.getTotalBytes(sstables) - (long)(maxBytesForLevel(i) * 1.001)) / 
(double)maxSSTableSizeInBytes);
 tasks += estimated[i];
 }

[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout

2015-03-17 Thread Brandon Williams (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-8982:

Reviewer: Brandon Williams  (was: Joshua McKenzie)

+1

 sstableloader outputs progress information and summary stats to stderr 
 instead of stdout
 

 Key: CASSANDRA-8982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 12.04.3 LTS
 Cassandra 2.1.3
Reporter: Tom Alexander
Assignee: Yuki Morishita
 Fix For: 2.1.4

 Attachments: 0001-use-stdout-for-normal-output.patch


 On Cassandra 2.1.0+, the progress information from sstableloader is output 
 into the stderr stream instead of stdout. 
 This can be reproduced by running an sstableloader and redirecting stderr to 
 a file:
 {code}
 sstableloader -d 127.0.0.1 
 /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
  2 error.log
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: UDF shouldn't log errors at ERROR (change to DEBUG)

Repository: cassandra
Updated Branches:
  refs/heads/trunk 287366752 - ac9ccfdf7


UDF shouldn't log errors at ERROR (change to DEBUG)

Patch by Robert Stupp for CASSANDRA-8980


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ac9ccfdf
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ac9ccfdf
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ac9ccfdf

Branch: refs/heads/trunk
Commit: ac9ccfdf7c1d06cf01eeb84561a7c514674cc3b8
Parents: 2873667
Author: Robert Stupp sn...@snazy.de
Authored: Tue Mar 17 18:15:52 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Tue Mar 17 18:15:52 2015 +0100

--
 .../apache/cassandra/cql3/functions/JavaSourceUDFFactory.java| 4 ++--
 src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac9ccfdf/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java 
b/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
index 3033d98..1abb40b 100644
--- a/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
+++ b/src/java/org/apache/cassandra/cql3/functions/JavaSourceUDFFactory.java
@@ -202,7 +202,7 @@ public final class JavaSourceUDFFactory
  * }
  * catch (Throwable t)
  * {
- * logger.info(Invocation of function '{}' failed, this, t);
+ * logger.debug(Invocation of function '{}' failed, this, t);
  * if (t instanceof VirtualMachineError)
  * throw (VirtualMachineError)t;
  * throw 
org.apache.cassandra.exceptions.FunctionExecutionException.build(this, t);
@@ -244,7 +244,7 @@ public final class JavaSourceUDFFactory
   }\n +
   catch (Throwable t)\n +
   {\n +
-logger.info(\Invocation of function '{}' failed\, 
this, t);\n +
+logger.debug(\Invocation of function '{}' failed\, 
this, t);\n +
 // handle OutOfMemoryError and other fatals not here!
 if (t instanceof VirtualMachineError)\n +
   throw (VirtualMachineError)t;\n +

http://git-wip-us.apache.org/repos/asf/cassandra/blob/ac9ccfdf/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
--
diff --git a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java 
b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
index 06452e6..4fe6ac9 100644
--- a/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
+++ b/src/java/org/apache/cassandra/cql3/functions/ScriptBasedUDF.java
@@ -139,7 +139,7 @@ public class ScriptBasedUDF extends UDFunction
 }
 catch (RuntimeException | ScriptException e)
 {
-logger.info(Execution of UDF '{}' failed, name, e);
+logger.debug(Execution of UDF '{}' failed, name, e);
 throw FunctionExecutionException.create(this, e);
 }
 }

[jira] [Created] (CASSANDRA-8983) Move Hadoop code out of tree

2015-03-17 Thread Jeremiah Jordan (JIRA)

Jeremiah Jordan created CASSANDRA-8983:
--

 Summary: Move Hadoop code out of tree
 Key: CASSANDRA-8983
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8983
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Jeremiah Jordan
Priority: Minor


After CASSANDRA-8358 all the Hadoop stuff should be going through the Java 
driver.
Since there shouldn't be any internal dependancies after that, we should break 
out the Hadoop client driver code into its own repo, like we did for all the 
other client drivers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout

2015-03-17 Thread Yuki Morishita (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita reassigned CASSANDRA-8982:
-

Assignee: Yuki Morishita

 sstableloader outputs progress information and summary stats to stderr 
 instead of stdout
 

 Key: CASSANDRA-8982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 12.04.3 LTS
 Cassandra 2.1.3
Reporter: Tom Alexander
Assignee: Yuki Morishita
 Fix For: 2.1.4

 Attachments: 0001-use-stdout-for-normal-output.patch


 On Cassandra 2.1.0+, the progress information from sstableloader is output 
 into the stderr stream instead of stdout. 
 This can be reproduced by running an sstableloader and redirecting stderr to 
 a file:
 {code}
 sstableloader -d 127.0.0.1 
 /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
  2 error.log
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk

[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365557#comment-14365557
]

Benedict commented on CASSANDRA-8981:
-

Jake wrote this test, but sure. I'll take a look, perhaps tomorrow

IndexSummaryManagerTest.testCompactionsRace failing on trunk

Key: CASSANDRA-8981
URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Joshua McKenzie
Priority: Minor
Fix For: 3.0

Keep running it repeatedly w/showoutput=yes in build.xml on junit and
you'll see it time out with:
{noformat}
[junit] WARN 17:02:56 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
[junit] WARN 17:02:56 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
[junit] WARN 17:02:57 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
[junit] WARN 17:02:57 Unable to cancel in-progress compactions for
StandardRace. Perhaps there is an unusually large row in progress somewhere,
or the system is simply overloaded.
{noformat}
I originally thought this was a Windows specific problem (CASSANDRA-8962) but
can reproduce on linux by just repeatedly running the test.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7712) temporary files need to be cleaned by unit tests


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365690#comment-14365690
 ] 

Michael Shuler commented on CASSANDRA-7712:
---

A couple suggestions were attempted as workarounds, but all the unit test data 
was still written to /tmp/
{noformat}
(trunk)mshuler@hana:~/git/cassandra$ export TMPDIR=`pwd`/testdata
(trunk)mshuler@hana:~/git/cassandra$ ant test -Djava.io.tmpdir=`pwd`/testdata
...
{noformat}

 temporary files need to be cleaned by unit tests
 

 Key: CASSANDRA-7712
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Michael Shuler
Priority: Minor
  Labels: bootcamp, lhf
 Fix For: 2.0.14

 Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt, 
 7712-v3.txt, CASSANDRA-7712_apache_cassandra_2.0.txt


 There are many unit test temporary files left behind after test runs. In the 
 case of CI servers, I have seen 70,000 files accumulate in /tmp over a 
 period of time. Each unit test should make an effort to remove its temporary 
 files when the test is completed.
 My current unit test cleanup block:
 {noformat}
 # clean up after unit tests..
 rm -rf  /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest* 
 /tmp/Keyspace1* \
 /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest* 
 /tmp/SSTableImportTest* \
 /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest* 
 /tmp/ValuesWithQuotes* \
 /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest* 
 /tmp/liblz4-java*.so /tmp/readtest* \
 /tmp/set_length_during_read_mode* /tmp/set_negative_length* 
 /tmp/snappy-*.so
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk

[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365749#comment-14365749
]

Benedict commented on CASSANDRA-8981:
-

Well, I did commit it, so you can't blame git blame for blaming me :)

IndexSummaryManagerTest.testCompactionsRace failing on trunk

Key: CASSANDRA-8981
URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Joshua McKenzie
Priority: Minor
Fix For: 3.0

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout

2015-03-17 Thread Philip Thompson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8982:
---
Since Version: 2.1.0

 sstableloader outputs progress information and summary stats to stderr 
 instead of stdout
 

 Key: CASSANDRA-8982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 12.04.3 LTS
 Cassandra 2.1.3
Reporter: Tom Alexander
 Fix For: 2.1.4


 On Cassandra 2.1.0+, the progress information from sstableloader is output 
 into the stderr stream instead of stdout. 
 This can be reproduced by running an sstableloader and redirecting stderr to 
 a file:
 {code}
 sstableloader -d 127.0.0.1 
 /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
  2 error.log
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout

2015-03-17 Thread Philip Thompson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Philip Thompson updated CASSANDRA-8982:
---
Fix Version/s: 2.1.4

 sstableloader outputs progress information and summary stats to stderr 
 instead of stdout
 

 Key: CASSANDRA-8982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 12.04.3 LTS
 Cassandra 2.1.3
Reporter: Tom Alexander
 Fix For: 2.1.4


 On Cassandra 2.1.0+, the progress information from sstableloader is output 
 into the stderr stream instead of stdout. 
 This can be reproduced by running an sstableloader and redirecting stderr to 
 a file:
 {code}
 sstableloader -d 127.0.0.1 
 /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
  2 error.log
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk

2015-03-17 Thread T Jake Luciani (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365769#comment-14365769
]

T Jake Luciani commented on CASSANDRA-8981:
---

Looks like it's just a timeout. The test is purposely trying to cause a race
condition. Perhaps this should be a long-unit test

IndexSummaryManagerTest.testCompactionsRace failing on trunk

Key: CASSANDRA-8981
URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Joshua McKenzie
Priority: Minor
Fix For: 3.0

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk

2015-03-17 Thread Joshua McKenzie (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Joshua McKenzie updated CASSANDRA-8981:
---
Summary: IndexSummaryManagerTest.testCompactionsRace intermittently timing
out on trunk (was: IndexSummaryManagerTest.testCompactionsRace failing on
trunk)

IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk
--

Key: CASSANDRA-8981
URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Joshua McKenzie
Priority: Minor
Fix For: 3.0

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8982) sstableloader outputs progress information and summary stats to stderr instead of stdout

2015-03-17 Thread Yuki Morishita (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8982?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yuki Morishita updated CASSANDRA-8982:
--
Attachment: 0001-use-stdout-for-normal-output.patch

Patch attached to use stdout for progress and stats reporting.

 sstableloader outputs progress information and summary stats to stderr 
 instead of stdout
 

 Key: CASSANDRA-8982
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8982
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Ubuntu 12.04.3 LTS
 Cassandra 2.1.3
Reporter: Tom Alexander
 Fix For: 2.1.4

 Attachments: 0001-use-stdout-for-normal-output.patch


 On Cassandra 2.1.0+, the progress information from sstableloader is output 
 into the stderr stream instead of stdout. 
 This can be reproduced by running an sstableloader and redirecting stderr to 
 a file:
 {code}
 sstableloader -d 127.0.0.1 
 /home/automaton/cassandra-src/data/data/keyspace1/standard1-abfab2e0ccb811e49d4417363885fa00
  2 error.log
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/8d570fa7
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/8d570fa7
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/8d570fa7

Branch: refs/heads/trunk
Commit: 8d570fa7992f8eaae55e8c13c57e6e84491062ee
Parents: ac9ccfd 89cdfd8
Author: Ariel Weisberg ar...@weisberg.ws
Authored: Tue Mar 17 19:31:54 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Tue Mar 17 19:31:54 2015 +0100

--
 CHANGES.txt |   1 +
 conf/cassandra.yaml |  12 ++
 .../org/apache/cassandra/config/Config.java |   6 +-
 .../cassandra/config/DatabaseDescriptor.java| 202 ++-
 .../config/YamlConfigurationLoader.java |  16 +-
 .../config/DatabaseDescriptorTest.java  | 139 -
 6 files changed, 274 insertions(+), 102 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/8d570fa7/CHANGES.txt
--
diff --cc CHANGES.txt
index be57f1b,30bf698..f813496
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,76 -1,5 +1,77 @@@
 +3.0
 + * Add WriteFailureException to native protocol, notify coordinator of
 +   write failures (CASSANDRA-8592)
 + * Convert SequentialWriter to nio (CASSANDRA-8709)
 + * Add role based access control (CASSANDRA-7653, 8650, 7216, 8760, 8849, 
8761, 8850)
 + * Record client ip address in tracing sessions (CASSANDRA-8162)
 + * Indicate partition key columns in response metadata for prepared
 +   statements (CASSANDRA-7660)
 + * Merge UUIDType and TimeUUIDType parse logic (CASSANDRA-8759)
 + * Avoid memory allocation when searching index summary (CASSANDRA-8793)
 + * Optimise (Time)?UUIDType Comparisons (CASSANDRA-8730)
 + * Make CRC32Ex into a separate maven dependency (CASSANDRA-8836)
 + * Use preloaded jemalloc w/ Unsafe (CASSANDRA-8714)
 + * Avoid accessing partitioner through StorageProxy (CASSANDRA-8244, 8268)
 + * Upgrade Metrics library and remove depricated metrics (CASSANDRA-5657)
 + * Serializing Row cache alternative, fully off heap (CASSANDRA-7438)
 + * Duplicate rows returned when in clause has repeated values (CASSANDRA-6707)
 + * Make CassandraException unchecked, extend RuntimeException (CASSANDRA-8560)
 + * Support direct buffer decompression for reads (CASSANDRA-8464)
 + * DirectByteBuffer compatible LZ4 methods (CASSANDRA-7039)
 + * Group sstables for anticompaction correctly (CASSANDRA-8578)
 + * Add ReadFailureException to native protocol, respond
 +   immediately when replicas encounter errors while handling
 +   a read request (CASSANDRA-7886)
 + * Switch CommitLogSegment from RandomAccessFile to nio (CASSANDRA-8308)
 + * Allow mixing token and partition key restrictions (CASSANDRA-7016)
 + * Support index key/value entries on map collections (CASSANDRA-8473)
 + * Modernize schema tables (CASSANDRA-8261)
 + * Support for user-defined aggregation functions (CASSANDRA-8053)
 + * Fix NPE in SelectStatement with empty IN values (CASSANDRA-8419)
 + * Refactor SelectStatement, return IN results in natural order instead
 +   of IN value list order and ignore duplicate values in partition key IN 
restrictions (CASSANDRA-7981)
 + * Support UDTs, tuples, and collections in user-defined
 +   functions (CASSANDRA-7563)
 + * Fix aggregate fn results on empty selection, result column name,
 +   and cqlsh parsing (CASSANDRA-8229)
 + * Mark sstables as repaired after full repair (CASSANDRA-7586)
 + * Extend Descriptor to include a format value and refactor reader/writer
 +   APIs (CASSANDRA-7443)
 + * Integrate JMH for microbenchmarks (CASSANDRA-8151)
 + * Keep sstable levels when bootstrapping (CASSANDRA-7460)
 + * Add Sigar library and perform basic OS settings check on startup 
(CASSANDRA-7838)
 + * Support for aggregation functions (CASSANDRA-4914)
 + * Remove cassandra-cli (CASSANDRA-7920)
 + * Accept dollar quoted strings in CQL (CASSANDRA-7769)
 + * Make assassinate a first class command (CASSANDRA-7935)
 + * Support IN clause on any partition key column (CASSANDRA-7855)
 + * Support IN clause on any clustering column (CASSANDRA-4762)
 + * Improve compaction logging (CASSANDRA-7818)
 + * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
 + * Do anticompaction in groups (CASSANDRA-6851)
 + * Support user-defined functions (CASSANDRA-7395, 7526, 7562, 7740, 7781, 
7929,
 +   7924, 7812, 8063, 7813, 7708)
 + * Permit configurable timestamps with cassandra-stress (CASSANDRA-7416)
 + * Move sstable RandomAccessReader to nio2, which allows using the
 +   FILE_SHARE_DELETE flag on Windows (CASSANDRA-4050)
 + * Remove CQL2 (CASSANDRA-5918)
 + * Add Thrift get_multi_slice call (CASSANDRA-6757)
 + * Optimize

[jira] [Updated] (CASSANDRA-8839) DatabaseDescriptor throws NPE when rpc_interface is used

2015-03-17 Thread Robert Stupp (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Robert Stupp updated CASSANDRA-8839:

Attachment: 8893-2.1.txt
8839.txt

Committed (attached rebased patches)

 DatabaseDescriptor throws NPE when rpc_interface is used
 

 Key: CASSANDRA-8839
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8839
 Project: Cassandra
  Issue Type: Bug
  Components: Config
 Environment: 2.1.3
Reporter: Jan Kesten
Assignee: Ariel Weisberg
 Fix For: 2.1.4

 Attachments: 8839.txt, 8893-2.1.txt


 Copy from mail to dev mailinglist. 
 When using
 - listen_interface instead of listen_address
 - rpc_interface instead of rpc_address
 starting 2.1.3 throws an NPE:
 {code}
 ERROR [main] 2015-02-20 07:50:09,661 DatabaseDescriptor.java:144 - Fatal 
 error during configuration loading
 java.lang.NullPointerException: null
 at 
 org.apache.cassandra.config.DatabaseDescriptor.applyConfig(DatabaseDescriptor.java:411)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
 at 
 org.apache.cassandra.config.DatabaseDescriptor.clinit(DatabaseDescriptor.java:133)
  ~[apache-cassandra-2.1.3.jar:2.1.3]
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:110) 
 [apache-cassandra-2.1.3.jar:2.1.3]
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:465)
  [apache-cassandra-2.1.3.jar:2.1.3]
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:554) 
 [apache-cassandra-2.1.3.jar:2.1.3]
 {code}
 Occurs on debian package as well as in tar.gz distribution. 
 {code}
 /* Local IP, hostname or interface to bind RPC server to */
 if(conf.rpc_address !=null conf.rpc_interface !=null)
 {
 throw newConfigurationException(Set rpc_address OR rpc_interface, not 
 both);
 }
 else if(conf.rpc_address !=null)
 {
 try
 {
 rpcAddress = InetAddress.getByName(conf.rpc_address);
 }
 catch(UnknownHostException e)
 {
 throw newConfigurationException(Unknown host in rpc_address + 
 conf.rpc_address);
 }
 }
 else if(conf.rpc_interface !=null)
 {
 listenAddress = 
 getNetworkInterfaceAddress(conf.rpc_interface,rpc_interface);
 }
 else
 {
 rpcAddress = FBUtilities.getLocalAddress();
 }
 {code}
 I think that listenAddress in the second else block is an error. In my case 
 rpc_interface is eth0, so listenAddress gets set, and rpcAddress remains 
 unset. The result is NPE in line 411:
 {code}
 if(rpcAddress.isAnyLocalAddress())
 {code}
 After changing rpc_interface to rpc_address everything works as expected.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8984) Introduce Transactional API for behaviours that can corrupt system state


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366454#comment-14366454
 ] 

Benedict commented on CASSANDRA-8984:
-

Patch available 
[here|https://github.com/belliottsmith/cassandra/commits/transactional-writers]

[~krummas] [~JoshuaMcKenzie] WDYT? You've seen the ugliness of these codepaths 
more than anyone else: does this go some way towards sanitising them? The final 
cleanup will have to wait until CASSANDRA-8568 and CASSANDRA-7066, but I think 
this gets us close to the finish line.

The main annoying thing about this patch is that we've always used close() to 
complete a change, and now this will rollback a change without a preceding 
commit, which means it's possible I've missed the introduction of a commit() 
somewhere if it isn't covered by a unit test (I've tried to search exhaustively 
for occurrences, but this is hard to be certain about). Hopefully that would 
become apparent very quickly, though, and I don't see a good alternative for 
safe rollback. 

 Introduce Transactional API for behaviours that can corrupt system state
 

 Key: CASSANDRA-8984
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8984
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1.4


 As a penultimate (and probably final for 2.1, if we agree to introduce it 
 there) round of changes to the internals managing sstable writing, I've 
 introduced a new API called Transactional that I hope will make it much 
 easier to write correct behaviour. As things stand we conflate a lot of 
 behaviours into methods like close - the recent changes unpicked some of 
 these, but didn't go far enough. My proposal here introduces an interface 
 designed to support four actions (on top of their normal function):
 * prepareToCommit
 * commit
 * abort
 * cleanup
 In normal operation, once we have finished constructing a state change we 
 call prepareToCommit; once all such state changes are prepared, we call 
 commit. If at any point everything fails, abort is called. In _either_ case, 
 cleanup is called at the very last.
 These transactional objects are all AutoCloseable, with the behaviour being 
 to rollback any changes unless commit has completed successfully.
 The changes are actually less invasive than it might sound, since we did 
 recently introduce abort in some places, as well as have commit like methods. 
 This simply formalises the behaviour, and makes it consistent between all 
 objects that interact in this way. Much of the code change is boilerplate, 
 such as moving an object into a try-declaration, although the change is still 
 non-trivial. What it _does_ do is eliminate a _lot_ of special casing that we 
 have had since 2.1 was released. The data tracker API changes and compaction 
 leftover cleanups should finish the job with making this much easier to 
 reason about, but this change I think is worthwhile considering for 2.1, 
 since we've just overhauled this entire area (and not released these 
 changes), and this change is essentially just the finishing touches, so the 
 risk is minimal and the potential gains reasonably significant.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: remove dead code

Repository: cassandra
Updated Branches:
  refs/heads/trunk 5a09483df - 45259f095


remove dead code


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45259f09
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45259f09
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45259f09

Branch: refs/heads/trunk
Commit: 45259f0958b5b8b43af0d399bb71c24bf9b53322
Parents: 5a09483
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 22:17:21 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 22:17:21 2015 -0400

--
 .../org/apache/cassandra/db/DataTracker.java | 18 +-
 .../apache/cassandra/service/StorageService.java | 19 ---
 2 files changed, 1 insertion(+), 36 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/45259f09/src/java/org/apache/cassandra/db/DataTracker.java
--
diff --git a/src/java/org/apache/cassandra/db/DataTracker.java 
b/src/java/org/apache/cassandra/db/DataTracker.java
index 19f027c..ca96a38 100644
--- a/src/java/org/apache/cassandra/db/DataTracker.java
+++ b/src/java/org/apache/cassandra/db/DataTracker.java
@@ -205,7 +205,7 @@ public class DataTracker
 if (Iterables.any(sstables, Predicates.in(currentView.compacting)))
 return false;
 
-Predicate live = new PredicateSSTableReader()
+PredicateSSTableReader live = new PredicateSSTableReader()
 {
 public boolean apply(SSTableReader sstable)
 {
@@ -774,22 +774,6 @@ public class DataTracker
 return new View(liveMemtables, flushingMemtables, sstablesMap, 
compactingNew, shadowed, intervalTree);
 }
 
-private SetSSTableReader newSSTables(CollectionSSTableReader 
oldSSTables, IterableSSTableReader replacements)
-{
-ImmutableSetSSTableReader oldSet = 
ImmutableSet.copyOf(oldSSTables);
-int newSSTablesSize = sstables.size() - oldSSTables.size() + 
Iterables.size(replacements);
-assert newSSTablesSize = Iterables.size(replacements) : 
String.format(Incoherent new size %d replacing %s by %s in %s, 
newSSTablesSize, oldSSTables, replacements, this);
-SetSSTableReader newSSTables = new HashSet(newSSTablesSize);
-
-for (SSTableReader sstable : sstables)
-if (!oldSet.contains(sstable))
-newSSTables.add(sstable);
-
-Iterables.addAll(newSSTables, replacements);
-assert newSSTables.size() == newSSTablesSize : 
String.format(Expecting new size of %d, got %d while replacing %s by %s in 
%s, newSSTablesSize, newSSTables.size(), oldSSTables, replacements, this);
-return ImmutableSet.copyOf(newSSTables);
-}
-
 @Override
 public String toString()
 {

http://git-wip-us.apache.org/repos/asf/cassandra/blob/45259f09/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 19e0a5a..b1f352c 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2600,7 +2600,6 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
  * @param endToken end token of the range
  * @return collection of ranges that match ring layout in TokenMetadata
  */
-@SuppressWarnings(unchecked)
 @VisibleForTesting
 CollectionRangeToken createRepairRangeFrom(String beginToken, String 
endToken)
 {
@@ -3631,24 +3630,6 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return finalOwnership;
 }
 
-
-private boolean hasSameReplication(ListString list)
-{
-if (list.isEmpty())
-return false;
-
-for (int i = 0; i  list.size() -1; i++)
-{
-KSMetaData ksm1 = Schema.instance.getKSMetaData(list.get(i));
-KSMetaData ksm2 = Schema.instance.getKSMetaData(list.get(i + 1));
-if (!ksm1.strategyClass.equals(ksm2.strategyClass) ||
-
!Iterators.elementsEqual(ksm1.strategyOptions.entrySet().iterator(),
- 
ksm2.strategyOptions.entrySet().iterator()))
-return false;
-}
-return true;
-}
-
 public ListString getKeyspaces()
 {
 ListString keyspaceNamesList = new 
ArrayList(Schema.instance.getKeyspaces());

[jira] [Commented] (CASSANDRA-8826) Distributed aggregates

[
https://issues.apache.org/jira/browse/CASSANDRA-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366504#comment-14366504
]

Benedict commented on CASSANDRA-8826:
-

I disagree that this is outside of scope for Cassandra. Spark and Hadoop are
not designed for answering realtime aggregation queries over medium sized
timeseries, and nor are coordinator level aggregations. IMO this is something
we should be aiming to do sooner than later, and is probably not _that_ hard
with the repair-aware consistency levels suggested by [~tjake]. Whether or not
we deliver this now, users are likely to abuse aggregations without realising
they are only acceptable for a very constrained kind of workload (very small
slices), and we will find ourselves again behind the user demand curve (trying
to deliver a feature that works as used, rather than as first envisaged).

NB: Pushing aggregations lower into the engine may also permit some very
significant optimisations at a later date, especially with columnar storage, so
even single node queries would be helped.

Distributed aggregates
--

Key: CASSANDRA-8826
URL: https://issues.apache.org/jira/browse/CASSANDRA-8826
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Robert Stupp
Priority: Minor

Aggregations have been implemented in CASSANDRA-4914.
All calculation is performed on the coordinator. This means, that all data is
pulled by the coordinator and processed there.
This ticket's about to distribute aggregates to make them more efficient.
Currently some related tickets (esp. CASSANDRA-8099) are currently in
progress - we should wait for them to land before talking about
implementation.
Another playgrounds (not covered by this ticket), that might be related is
about _distributed filtering_.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8920) Remove IntervalTree from maxPurgeableTimestamp calculation


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-8920:

Attachment: 8920.txt

 Remove IntervalTree from maxPurgeableTimestamp calculation
 --

 Key: CASSANDRA-8920
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8920
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Priority: Minor
 Attachments: 8920.txt


 The IntervalTree only maps partition keys. Since a majority of users deploy a 
 hashed partitioner the work is mostly wasted, since they will be evenly 
 distributed across the full token range owned by the node - and in some cases 
 it is a significant amount of work. We can perform a corroboration against 
 the file bounds if we get a BF match as a sanity check if we like, but 
 performing an IntervalTree search is significantly more expensive (esp. once 
 murmur hash calculation memoization goes mainstream).
 In LCS, the keys are bounded, to it might appear that it would help, but in 
 this scenario we only compact against like bounds, so again it is not helpful.
 With a ByteOrderedPartitioner it could potentially be of use, but this is 
 sufficiently rare to not optimise for IMO.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-8773) cassandra-stress should validate its results in user mode


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict reassigned CASSANDRA-8773:
---

Assignee: Benedict

 cassandra-stress should validate its results in user mode
 ---

 Key: CASSANDRA-8773
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8773
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Benedict
Assignee: Benedict
  Labels: stress





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Suppress warning: 'DirectBuffer is internal proprietary API and may be removed in a future release'

Repository: cassandra
Updated Branches:
  refs/heads/trunk 69f794aca - da8268f15


Suppress warning: 'DirectBuffer is internal proprietary API and may be removed 
in a future release'


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/da8268f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/da8268f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/da8268f1

Branch: refs/heads/trunk
Commit: da8268f1568ac6720144612cd39379de63a08378
Parents: 69f794a
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 23:15:22 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 23:15:22 2015 -0400

--
 build.xml | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/da8268f1/build.xml
--
diff --git a/build.xml b/build.xml
index c99e8f8..09e67d3 100644
--- a/build.xml
+++ b/build.xml
@@ -1084,13 +1084,14 @@
  includeantruntime=false
  source=${source.version} 
  target=${target.version}
-  classpath
+ classpath
 path refid=cassandra.classpath/
-  /classpath
-  src path=${test.unit.src}/
-  src path=${test.long.src}/
-  src path=${test.pig.src}/
-  src path=${test.microbench.src}/
+ /classpath
+compilerarg value=-XDignore.symbol.file/
+ src path=${test.unit.src}/
+ src path=${test.long.src}/
+ src path=${test.pig.src}/
+ src path=${test.microbench.src}/
 /javac
 
 !-- Non-java resources needed by the test suite --

[jira] [Assigned] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk

[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benedict reassigned CASSANDRA-8981:
---

Assignee: Benedict

IndexSummaryManagerTest.testCompactionsRace intermittently timing out on trunk
--

Key: CASSANDRA-8981
URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Joshua McKenzie
Assignee: Benedict
Priority: Minor
Fix For: 3.0

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: simplify - use get(0) over iterator().next() for lists

Repository: cassandra
Updated Branches:
  refs/heads/trunk 201004d9a - 3608bd249


simplify - use get(0) over iterator().next() for lists


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3608bd24
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3608bd24
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3608bd24

Branch: refs/heads/trunk
Commit: 3608bd2494a82ede56471f1cbe5356f90d0a95d1
Parents: 201004d
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 22:34:13 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 22:34:13 2015 -0400

--
 src/java/org/apache/cassandra/tools/StandaloneScrubber.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/3608bd24/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
--
diff --git a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java 
b/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
index 15ff621..133a62e 100644
--- a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneScrubber.java
@@ -150,7 +150,7 @@ public class StandaloneScrubber
 {
 WrappingCompactionStrategy wrappingStrategy = 
(WrappingCompactionStrategy)strategy;
 int maxSizeInMB = 
(int)((cfs.getCompactionStrategy().getMaxSSTableBytes()) / (1024L * 1024L));
-if (wrappingStrategy.getWrappedStrategies().size() == 2  
wrappingStrategy.getWrappedStrategies().iterator().next() instanceof 
LeveledCompactionStrategy)
+if (wrappingStrategy.getWrappedStrategies().size() == 2  
wrappingStrategy.getWrappedStrategies().get(0) instanceof 
LeveledCompactionStrategy)
 {
 System.out.println(Checking leveled manifest);
 PredicateSSTableReader repairedPredicate = new 
PredicateSSTableReader()

[jira] [Updated] (CASSANDRA-8920) Remove IntervalTree from maxPurgeableTimestamp calculation


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8920:
--
Reviewer: Marcus Eriksson

 Remove IntervalTree from maxPurgeableTimestamp calculation
 --

 Key: CASSANDRA-8920
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8920
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
Priority: Minor
 Fix For: 2.1.4

 Attachments: 8920.txt


 The IntervalTree only maps partition keys. Since a majority of users deploy a 
 hashed partitioner the work is mostly wasted, since they will be evenly 
 distributed across the full token range owned by the node - and in some cases 
 it is a significant amount of work. We can perform a corroboration against 
 the file bounds if we get a BF match as a sanity check if we like, but 
 performing an IntervalTree search is significantly more expensive (esp. once 
 murmur hash calculation memoization goes mainstream).
 In LCS, the keys are bounded, to it might appear that it would help, but in 
 this scenario we only compact against like bounds, so again it is not helpful.
 With a ByteOrderedPartitioner it could potentially be of use, but this is 
 sufficiently rare to not optimise for IMO.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8984) Introduce Transactional API for behaviours that can corrupt system state


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8984:
--
Reviewer: Joshua McKenzie

 Introduce Transactional API for behaviours that can corrupt system state
 

 Key: CASSANDRA-8984
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8984
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1.4


 As a penultimate (and probably final for 2.1, if we agree to introduce it 
 there) round of changes to the internals managing sstable writing, I've 
 introduced a new API called Transactional that I hope will make it much 
 easier to write correct behaviour. As things stand we conflate a lot of 
 behaviours into methods like close - the recent changes unpicked some of 
 these, but didn't go far enough. My proposal here introduces an interface 
 designed to support four actions (on top of their normal function):
 * prepareToCommit
 * commit
 * abort
 * cleanup
 In normal operation, once we have finished constructing a state change we 
 call prepareToCommit; once all such state changes are prepared, we call 
 commit. If at any point everything fails, abort is called. In _either_ case, 
 cleanup is called at the very last.
 These transactional objects are all AutoCloseable, with the behaviour being 
 to rollback any changes unless commit has completed successfully.
 The changes are actually less invasive than it might sound, since we did 
 recently introduce abort in some places, as well as have commit like methods. 
 This simply formalises the behaviour, and makes it consistent between all 
 objects that interact in this way. Much of the code change is boilerplate, 
 such as moving an object into a try-declaration, although the change is still 
 non-trivial. What it _does_ do is eliminate a _lot_ of special casing that we 
 have had since 2.1 was released. The data tracker API changes and compaction 
 leftover cleanups should finish the job with making this much easier to 
 reason about, but this change I think is worthwhile considering for 2.1, 
 since we've just overhauled this entire area (and not released these 
 changes), and this change is essentially just the finishing touches, so the 
 risk is minimal and the potential gains reasonably significant.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8568:
--
Reviewer: Marcus Eriksson

(I assigned Josh to review 8984 and Marcus this one, under the assumption that 
they can be done in parallel.  If not, let's give both to Marcus.)

 Impose new API on data tracker modifications that makes correct usage obvious 
 and imposes safety
 

 Key: CASSANDRA-8568
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8568
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.0


 DataTracker has become a bit of a quagmire, and not at all obvious to 
 interface with, with many subtly different modifiers. I suspect it is still 
 subtly broken, especially around error recovery.
 I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for 
 those situations where a try/finally block isn't possible) objects that have 
 transactional behaviour, and with few simple declarative methods that can be 
 composed simply to provide all of the functionality we currently need.
 See CASSANDRA-8399 for context



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds

[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366547#comment-14366547
]

Benedict commented on CASSANDRA-3852:
-

I think this is passing the window of viability for 3.0, so bumping to 3.1+
until discussion can happen on strategy, and/or the introduction of dropped
message notification to MessageService.

use LIFO queueing policy when queue size exceeds thresholds
---

Key: CASSANDRA-3852
URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
Project: Cassandra
Issue Type: Improvement
Reporter: Peter Schuller
Labels: performance
Fix For: 3.1

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-3852) use LIFO queueing policy when queue size exceeds thresholds

[
https://issues.apache.org/jira/browse/CASSANDRA-3852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Benedict updated CASSANDRA-3852:

Fix Version/s: (was: 3.0)
3.1

use LIFO queueing policy when queue size exceeds thresholds
---

Key: CASSANDRA-3852
URL: https://issues.apache.org/jira/browse/CASSANDRA-3852
Project: Cassandra
Issue Type: Improvement
Reporter: Peter Schuller
Labels: performance
Fix For: 3.1

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: better exception messages

Repository: cassandra
Updated Branches:
  refs/heads/trunk 593886a12 - 69f794aca


better exception messages


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/69f794ac
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/69f794ac
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/69f794ac

Branch: refs/heads/trunk
Commit: 69f794aca680cff079c1419f46f04316d4262a23
Parents: 593886a
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 23:07:20 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 23:07:20 2015 -0400

--
 src/java/org/apache/cassandra/cache/CachingOptions.java   |  6 +++---
 src/java/org/apache/cassandra/config/CFMetaData.java  | 10 +-
 .../org/apache/cassandra/config/ColumnDefinition.java |  2 +-
 .../apache/cassandra/config/YamlConfigurationLoader.java  |  2 +-
 src/java/org/apache/cassandra/cql3/Attributes.java|  6 +++---
 src/java/org/apache/cassandra/db/BufferExpiringCell.java  |  4 ++--
 6 files changed, 15 insertions(+), 15 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/69f794ac/src/java/org/apache/cassandra/cache/CachingOptions.java
--
diff --git a/src/java/org/apache/cassandra/cache/CachingOptions.java 
b/src/java/org/apache/cassandra/cache/CachingOptions.java
index c2605f5..1c82f55 100644
--- a/src/java/org/apache/cassandra/cache/CachingOptions.java
+++ b/src/java/org/apache/cassandra/cache/CachingOptions.java
@@ -76,18 +76,18 @@ public class CachingOptions
 {
 if (!(value.equals(ALL) || value.equals(NONE)))
 {
-throw new ConfigurationException('keys' can only have 
values 'ALL' or 'NONE');
+throw new ConfigurationException('keys' can only have 
values 'ALL' or 'NONE', but was ' + value + ');
 }
 }
 else if (entry.getKey().equals(rows_per_partition))
 {
 if (!(value.equals(ALL) || value.equals(NONE) || 
StringUtils.isNumeric(value)))
 {
-throw new ConfigurationException('rows_per_partition' can 
only have values 'ALL', 'NONE' or be numeric.);
+throw new ConfigurationException('rows_per_partition' can 
only have values 'ALL', 'NONE' or be numeric, but was ' + value + '.);
 }
 }
 else
-throw new ConfigurationException(Only supported 
CachingOptions parameters are 'keys' and 'rows_per_partition');
+throw new ConfigurationException(Only supported 
CachingOptions parameters are 'keys' and 'rows_per_partition', but was ' + 
entry.getKey() + ');
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/69f794ac/src/java/org/apache/cassandra/config/CFMetaData.java
--
diff --git a/src/java/org/apache/cassandra/config/CFMetaData.java 
b/src/java/org/apache/cassandra/config/CFMetaData.java
index 138acac..d8eeaf2 100644
--- a/src/java/org/apache/cassandra/config/CFMetaData.java
+++ b/src/java/org/apache/cassandra/config/CFMetaData.java
@@ -112,7 +112,7 @@ public final class CFMetaData
 {
 double value = Double.parseDouble(name.substring(0, 
name.length() - 10));
 if (value  100 || value  0)
-throw new ConfigurationException(PERCENTILE should be 
between 0 and 100);
+throw new ConfigurationException(PERCENTILE should be 
between 0 and 100, but was  + value);
 return new SpeculativeRetry(RetryType.PERCENTILE, (value / 
100));
 }
 else if (name.endsWith(MS))
@@ -813,10 +813,10 @@ public final class CFMetaData
cfm.cfId, cfId));
 
 if (cfm.cfType != cfType)
-throw new ConfigurationException(types do not match.);
+throw new ConfigurationException(String.format(Column family 
types do not match (found %s; expected %s)., cfm.cfType, cfType));
 
 if (!cfm.comparator.isCompatibleWith(comparator))
-throw new ConfigurationException(comparators do not match or are 
not compatible.);
+throw new ConfigurationException(String.format(Column family 
comparators do not match or are not compatible (found %s; expected %s)., 
cfm.comparator.getClass().getSimpleName(), 
comparator.getClass().getSimpleName()));
 }
 
 public static void validateCompactionOptions(Class? extends 
AbstractCompactionStrategy strategyClass, MapString, String options) throws 
ConfigurationException
@@ -838,7 +838,7 @@

cassandra git commit: use long math for long results

Repository: cassandra
Updated Branches:
  refs/heads/trunk 4a4f83334 - 5a09483df


use long math for long results


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a09483d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a09483d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a09483d

Branch: refs/heads/trunk
Commit: 5a09483df399b9e498e30624d0e3227fb60d7fa1
Parents: 4a4f833
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 21:16:49 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 21:16:49 2015 -0400

--
 .../org/apache/cassandra/io/compress/CompressionMetadata.java| 4 ++--
 src/java/org/apache/cassandra/utils/StreamingHistogram.java  | 2 +-
 2 files changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a09483d/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
--
diff --git a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java 
b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
index 182cdd2..928541a 100644
--- a/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
+++ b/src/java/org/apache/cassandra/io/compress/CompressionMetadata.java
@@ -188,7 +188,7 @@ public class CompressionMetadata
 {
 try
 {
-offsets.setLong(i * 8, input.readLong());
+offsets.setLong(i * 8L, input.readLong());
 }
 catch (EOFException e)
 {
@@ -290,7 +290,7 @@ public class CompressionMetadata
 {
 if (count == maxCount)
 {
-SafeMemory newOffsets = offsets.copy((maxCount *= 2L) * 8);
+SafeMemory newOffsets = offsets.copy((maxCount *= 2L) * 8L);
 offsets.close();
 offsets = newOffsets;
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a09483d/src/java/org/apache/cassandra/utils/StreamingHistogram.java
--
diff --git a/src/java/org/apache/cassandra/utils/StreamingHistogram.java 
b/src/java/org/apache/cassandra/utils/StreamingHistogram.java
index 3f5a715..eb884be 100644
--- a/src/java/org/apache/cassandra/utils/StreamingHistogram.java
+++ b/src/java/org/apache/cassandra/utils/StreamingHistogram.java
@@ -201,7 +201,7 @@ public class StreamingHistogram
 MapDouble, Long entries = histogram.getAsMap();
 size += typeSizes.sizeof(entries.size());
 // size of entries = size * (8(double) + 8(long))
-size += entries.size() * (8 + 8);
+size += entries.size() * (8L + 8L);
 return size;
 }
 }

[jira] [Commented] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366487#comment-14366487
 ] 

Benedict commented on CASSANDRA-8568:
-

Improvements to Byteman testing can be a follow up ticket, which I'll get to in 
a week or two. It's ready for review as stands, although I would prefer 
CASSANDRA-8984 take priority.

 Impose new API on data tracker modifications that makes correct usage obvious 
 and imposes safety
 

 Key: CASSANDRA-8568
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8568
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.0


 DataTracker has become a bit of a quagmire, and not at all obvious to 
 interface with, with many subtly different modifiers. I suspect it is still 
 subtly broken, especially around error recovery.
 I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for 
 those situations where a try/finally block isn't possible) objects that have 
 transactional behaviour, and with few simple declarative methods that can be 
 composed simply to provide all of the functionality we currently need.
 See CASSANDRA-8399 for context



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-8568:

Fix Version/s: 3.0

 Impose new API on data tracker modifications that makes correct usage obvious 
 and imposes safety
 

 Key: CASSANDRA-8568
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8568
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.0


 DataTracker has become a bit of a quagmire, and not at all obvious to 
 interface with, with many subtly different modifiers. I suspect it is still 
 subtly broken, especially around error recovery.
 I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for 
 those situations where a try/finally block isn't possible) objects that have 
 transactional behaviour, and with few simple declarative methods that can be 
 composed simply to provide all of the functionality we currently need.
 See CASSANDRA-8399 for context



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: FBUtilities.getProtectedField never returns null

Repository: cassandra
Updated Branches:
  refs/heads/trunk 45259f095 - 201004d9a


FBUtilities.getProtectedField never returns null


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/201004d9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/201004d9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/201004d9

Branch: refs/heads/trunk
Commit: 201004d9a62307af1a0a419caadbfe99d8b72c66
Parents: 45259f0
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 22:24:33 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 22:24:33 2015 -0400

--
 src/java/org/apache/cassandra/utils/CLibrary.java| 6 --
 src/java/org/apache/cassandra/utils/FBUtilities.java | 7 ++-
 test/unit/org/apache/cassandra/auth/RoleOptionsTest.java | 1 -
 3 files changed, 2 insertions(+), 12 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/201004d9/src/java/org/apache/cassandra/utils/CLibrary.java
--
diff --git a/src/java/org/apache/cassandra/utils/CLibrary.java 
b/src/java/org/apache/cassandra/utils/CLibrary.java
index 4dbd904..25f7e5a 100644
--- a/src/java/org/apache/cassandra/utils/CLibrary.java
+++ b/src/java/org/apache/cassandra/utils/CLibrary.java
@@ -281,9 +281,6 @@ public final class CLibrary
 {
 Field field = FBUtilities.getProtectedField(channel.getClass(), fd);
 
-if (field == null)
-return -1;
-
 try
 {
 return getfd((FileDescriptor)field.get(channel));
@@ -304,9 +301,6 @@ public final class CLibrary
 {
 Field field = FBUtilities.getProtectedField(descriptor.getClass(), 
fd);
 
-if (field == null)
-return -1;
-
 try
 {
 return field.getInt(descriptor);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/201004d9/src/java/org/apache/cassandra/utils/FBUtilities.java
--
diff --git a/src/java/org/apache/cassandra/utils/FBUtilities.java 
b/src/java/org/apache/cassandra/utils/FBUtilities.java
index 2263e46..be7cbd8 100644
--- a/src/java/org/apache/cassandra/utils/FBUtilities.java
+++ b/src/java/org/apache/cassandra/utils/FBUtilities.java
@@ -532,19 +532,16 @@ public class FBUtilities
  */
 public static Field getProtectedField(Class klass, String fieldName)
 {
-Field field;
-
 try
 {
-field = klass.getDeclaredField(fieldName);
+Field field = klass.getDeclaredField(fieldName);
 field.setAccessible(true);
+return field;
 }
 catch (Exception e)
 {
 throw new AssertionError(e);
 }
-
-return field;
 }
 
 public static T CloseableIteratorT closeableIterator(IteratorT 
iterator)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/201004d9/test/unit/org/apache/cassandra/auth/RoleOptionsTest.java
--
diff --git a/test/unit/org/apache/cassandra/auth/RoleOptionsTest.java 
b/test/unit/org/apache/cassandra/auth/RoleOptionsTest.java
index 71f0c97..6dea2b5 100644
--- a/test/unit/org/apache/cassandra/auth/RoleOptionsTest.java
+++ b/test/unit/org/apache/cassandra/auth/RoleOptionsTest.java
@@ -118,7 +118,6 @@ public class RoleOptionsTest
 private void setupRoleManager(IRoleManager manager)
 {
 Field field = FBUtilities.getProtectedField(DatabaseDescriptor.class, 
roleManager);
-field.setAccessible(true);
 try
 {
 field.set(null, manager);

cassandra git commit: CompositeType.extractComponent is static, call it as such

Repository: cassandra
Updated Branches:
  refs/heads/trunk 3608bd249 - 593886a12


CompositeType.extractComponent is static, call it as such


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/593886a1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/593886a1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/593886a1

Branch: refs/heads/trunk
Commit: 593886a12062429d752e58a27baba4fb6ba7ac76
Parents: 3608bd2
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 22:39:26 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 22:39:26 2015 -0400

--
 .../index/composites/CompositesIndexOnCollectionKeyAndValue.java   | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/593886a1/src/java/org/apache/cassandra/db/index/composites/CompositesIndexOnCollectionKeyAndValue.java
--
diff --git 
a/src/java/org/apache/cassandra/db/index/composites/CompositesIndexOnCollectionKeyAndValue.java
 
b/src/java/org/apache/cassandra/db/index/composites/CompositesIndexOnCollectionKeyAndValue.java
index e18ea4e..0b7f579 100644
--- 
a/src/java/org/apache/cassandra/db/index/composites/CompositesIndexOnCollectionKeyAndValue.java
+++ 
b/src/java/org/apache/cassandra/db/index/composites/CompositesIndexOnCollectionKeyAndValue.java
@@ -72,7 +72,7 @@ public class CompositesIndexOnCollectionKeyAndValue extends 
CompositesIndexInclu
 
 private ByteBuffer extractIndexKeyComponent(IndexedEntry entry, int 
component)
 {
-return 
((CompositeType)getIndexKeyComparator()).extractComponent(entry.indexValue.getKey(),
 component);
+return CompositeType.extractComponent(entry.indexValue.getKey(), 
component);
 }
 
 private ByteBuffer extractCollectionValue(IndexedEntry entry)

[jira] [Commented] (CASSANDRA-8568) Impose new API on data tracker modifications that makes correct usage obvious and imposes safety


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366532#comment-14366532
 ] 

Benedict commented on CASSANDRA-8568:
-

They can. They touch related code, and I will have to rebase this against the 
result of 8984 before commit which will result in a small follow up 
corroborative review, but the main body of work is independent.

 Impose new API on data tracker modifications that makes correct usage obvious 
 and imposes safety
 

 Key: CASSANDRA-8568
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8568
 Project: Cassandra
  Issue Type: Bug
Reporter: Benedict
Assignee: Benedict
 Fix For: 3.0


 DataTracker has become a bit of a quagmire, and not at all obvious to 
 interface with, with many subtly different modifiers. I suspect it is still 
 subtly broken, especially around error recovery.
 I propose piggy-backing on CASSANDRA-7705 to offer RAII (and GC-enforced, for 
 those situations where a try/finally block isn't possible) objects that have 
 transactional behaviour, and with few simple declarative methods that can be 
 composed simply to provide all of the functionality we currently need.
 See CASSANDRA-8399 for context



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: use parameterized logging

Repository: cassandra
Updated Branches:
  refs/heads/trunk 642546aba - 4a4f83334


use parameterized logging


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4a4f8333
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4a4f8333
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4a4f8333

Branch: refs/heads/trunk
Commit: 4a4f83334265b1c2200e818b3439c146e77d46be
Parents: 642546a
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 21:10:23 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 21:10:23 2015 -0400

--
 .../org/apache/cassandra/io/sstable/format/big/BigTableWriter.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/4a4f8333/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
--
diff --git 
a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java 
b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
index ea2549d..ec5c165 100644
--- a/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/big/BigTableWriter.java
@@ -124,7 +124,7 @@ public class BigTableWriter extends SSTableWriter
 first = lastWrittenKey;
 
 if (logger.isTraceEnabled())
-logger.trace(wrote  + decoratedKey +  at  + dataEnd);
+logger.trace(wrote {} at {}, decoratedKey, dataEnd);
 iwriter.append(decoratedKey, index, dataEnd);
 dbuilder.addPotentialBoundary(dataEnd);
 }

[jira] [Created] (CASSANDRA-8984) Introduce Transactional API for behaviours that can corrupt system state

Benedict created CASSANDRA-8984:
---

 Summary: Introduce Transactional API for behaviours that can 
corrupt system state
 Key: CASSANDRA-8984
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8984
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1.4


As a penultimate (and probably final for 2.1, if we agree to introduce it 
there) round of changes to the internals managing sstable writing, I've 
introduced a new API called Transactional that I hope will make it much 
easier to write correct behaviour. As things stand we conflate a lot of 
behaviours into methods like close - the recent changes unpicked some of 
these, but didn't go far enough. My proposal here introduces an interface 
designed to support four actions (on top of their normal function):

* prepareToCommit
* commit
* abort
* cleanup

In normal operation, once we have finished constructing a state change we call 
prepareToCommit; once all such state changes are prepared, we call commit. If 
at any point everything fails, abort is called. In _either_ case, cleanup is 
called at the very last.

These transactional objects are all AutoCloseable, with the behaviour being to 
rollback any changes unless commit has completed successfully.

The changes are actually less invasive than it might sound, since we did 
recently introduce abort in some places, as well as have commit like methods. 
This simply formalises the behaviour, and makes it consistent between all 
objects that interact in this way. Much of the code change is boilerplate, such 
as moving an object into a try-declaration, although the change is still 
non-trivial. What it _does_ do is eliminate a _lot_ of special casing that we 
have had since 2.1 was released. The data tracker API changes and compaction 
leftover cleanups should finish the job with making this much easier to reason 
about, but this change I think is worthwhile considering for 2.1, since we've 
just overhauled this entire area (and not released these changes), and this 
change is essentially just the finishing touches, so the risk is minimal and 
the potential gains reasonably significant.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-8939) Stack overflow when reading data ingested through SSTableLoader


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict resolved CASSANDRA-8939.
-
   Resolution: Duplicate
Fix Version/s: 2.1.4

 Stack overflow when reading data ingested through SSTableLoader
 ---

 Key: CASSANDRA-8939
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8939
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Single C* node
 Linux Mint 17.1, kernel 3.16.0-30-generic
 Oracle Java 7u75.
Reporter: Piotr Kołaczkowski
Assignee: Benedict
 Fix For: 2.1.4

 Attachments: 8939.txt


 I created an empty table:
 {noformat}
 CREATE TABLE test.kv (
 key int PRIMARY KEY,
 value text
 ) WITH bloom_filter_fp_chance = 0.01
 AND caching = '{keys:ALL, rows_per_partition:NONE}'
 AND comment = ''
 AND compaction = {'min_threshold': '4', 'class': 
 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy', 
 'max_threshold': '32'}
 AND compression = {'sstable_compression': 
 'org.apache.cassandra.io.compress.LZ4Compressor'}
 AND dclocal_read_repair_chance = 0.1
 AND default_time_to_live = 0
 AND gc_grace_seconds = 864000
 AND max_index_interval = 2048
 AND memtable_flush_period_in_ms = 0
 AND min_index_interval = 128
 AND read_repair_chance = 0.0
 AND speculative_retry = '99.0PERCENTILE';
 {noformat}
 Then I loaded some rows into it using CqlSSTableWriter and SSTableLoader 
 (programmatically, doing it the same way as BulkLoader is doing it). The 
 streaming finished with no errors. 
 I can even read all the data back with cqlsh:
 {noformat}
 cqlsh SELECT key, value FROM test.kv;
 
  3405 | foo3405
  5504 | foo5504
  3476 | foo3476
  2542 | foo2542
  6931 | foo6931
 ---MORE---
 (1 rows)
 {noformat}
 However, filtering by token fails:
 {noformat}
 cqlsh SELECT key, value FROM test.kv WHERE token(key)  854443789258213092;
 OperationTimedOut: errors={}, last_host=127.0.0.1
 cqlsh 
 {noformat}
 Server log repors a StackOverflowException:
 {noformat}
 WARN  15:10:05  Uncaught exception on thread 
 Thread[SharedPool-Worker-2,5,main]: {}
 java.lang.StackOverflowError: null
   at 
 java.nio.charset.CharsetDecoder.implReplaceWith(CharsetDecoder.java:302) 
 ~[na:1.7.0_75]
   at java.nio.charset.CharsetDecoder.replaceWith(CharsetDecoder.java:288) 
 ~[na:1.7.0_75]
   at java.nio.charset.CharsetDecoder.init(CharsetDecoder.java:203) 
 ~[na:1.7.0_75]
   at java.nio.charset.CharsetDecoder.init(CharsetDecoder.java:226) 
 ~[na:1.7.0_75]
   at sun.nio.cs.UTF_8$Decoder.init(UTF_8.java:84) ~[na:1.7.0_75]
   at sun.nio.cs.UTF_8$Decoder.init(UTF_8.java:81) ~[na:1.7.0_75]
   at sun.nio.cs.UTF_8.newDecoder(UTF_8.java:68) ~[na:1.7.0_75]
   at 
 org.apache.cassandra.utils.ByteBufferUtil.string(ByteBufferUtil.java:152) 
 ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:39)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.serializers.AbstractTextSerializer.deserialize(AbstractTextSerializer.java:26)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:82) 
 ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.cql3.ColumnIdentifier.init(ColumnIdentifier.java:54) 
 ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.composites.CompoundSparseCellNameType.idFor(CompoundSparseCellNameType.java:169)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.composites.CompoundSparseCellNameType.makeWith(CompoundSparseCellNameType.java:177)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.composites.AbstractCompoundCellNameType.fromByteBuffer(AbstractCompoundCellNameType.java:106)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:397)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.composites.AbstractCType$Serializer.deserialize(AbstractCType.java:381)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.OnDiskAtom$Serializer.deserializeFromSSTable(OnDiskAtom.java:75)
  ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:52) 
 ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 org.apache.cassandra.db.AbstractCell$1.computeNext(AbstractCell.java:46) 
 ~[cassandra-all-2.1.3.248.jar:2.1.3.248]
   at 
 com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
  ~[guava-16.0.1.jar:na]

[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming

2015-03-17 Thread Stefania (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366406#comment-14366406
 ] 

Stefania commented on CASSANDRA-8838:
-

+1 on everything now, tests look great and pass without problems on my box.

I cannot commit myself so I did not resolve the ticket just yet but it can be 
resolved and committed at any time.

 Resumable bootstrap streaming
 -

 Key: CASSANDRA-8838
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8838
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Minor
  Labels: dense-storage
 Fix For: 3.0


 This allows the bootstrapping node not to be streamed already received data.
 The bootstrapping node records received keyspace/ranges as one stream session 
 completes. When some sessions with other nodes fail, bootstrapping fails 
 also, though next time it re-bootstraps, already received keyspace/ranges are 
 skipped to be streamed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8966) SequentialWriter should be Ref counted


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14366253#comment-14366253
 ] 

Benedict commented on CASSANDRA-8966:
-

Another FYI, that I've left out the headline aspect of this ticket from the 
change I'm filing, so if you're still interested in making this addition you're 
welcome to have a crack at it!

CASSANDRA-8984 refactors the APIs to make them safer, but does not introduce 
Ref counting.

 SequentialWriter should be Ref counted
 --

 Key: CASSANDRA-8966
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8966
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Priority: Minor
 Fix For: 3.0


 A LHF to introduce some more resource safety



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[5/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77d58f90
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77d58f90
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77d58f90

Branch: refs/heads/cassandra-2.1
Commit: 77d58f900efac62c7c8dcc63c3de5ecb0fe4e343
Parents: 89cdfd8 b2aa67e
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Mar 17 14:34:26 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Mar 17 14:34:26 2015 -0500

--
 CHANGES.txt | 1 +
 build.xml   | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/CHANGES.txt
--
diff --cc CHANGES.txt
index 30bf698,f10a89a..6b8618b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,50 -1,5 +1,51 @@@
 -2.0.14:
 +2.1.4
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
 + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
 +Merged from 2.0:
+  * Allow specifying the tmp dir (CASSANDRA-7712)
   * Improve compaction estimated tasks estimation (CASSANDRA-8904)
   * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
   * Expose commit log archive status via JMX (CASSANDRA-8734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/build.xml
--
diff --cc build.xml
index 4ced4e2,f5e14ba..bdb3646
--- a/build.xml
+++ b/build.xml
@@@ -1182,21 -1099,8 +1183,22 @@@
jvmarg 
value=-Dmigration-sstable-root=${test.data}/migration-sstables/
jvmarg value=-Dcassandra.ring_delay_ms=1000/
jvmarg value=-Dcassandra.tolerate_sstable_size=true/
+   jvmarg value=-Djava.io.tmpdir=${tmp.dir}/
  /testmacro
 +fileset

[2/6] cassandra git commit: Allow specifying the tmp dir

Allow specifying the tmp dir

Patch by mshuler, reviewed by brandonwilliams for CASSANDRA-7712


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2aa67ea
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2aa67ea
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2aa67ea

Branch: refs/heads/cassandra-2.1
Commit: b2aa67ea298044f4c4ad905e54963f0c03643b68
Parents: 0814737
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Mar 17 14:33:27 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Mar 17 14:33:27 2015 -0500

--
 CHANGES.txt | 1 +
 build.xml   | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 65f2da4..f10a89a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Allow specifying the tmp dir (CASSANDRA-7712)
  * Improve compaction estimated tasks estimation (CASSANDRA-8904)
  * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
  * Expose commit log archive status via JMX (CASSANDRA-8734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/build.xml
--
diff --git a/build.xml b/build.xml
index bdbd10a..f5e14ba 100644
--- a/build.xml
+++ b/build.xml
@@ -60,6 +60,7 @@
 property name=test.long.src value=${test.dir}/long/
 property name=test.pig.src value=${test.dir}/pig/
 property name=dist.dir value=${build.dir}/dist/
+property name=tmp.dir value=${java.io.tmpdir}/

property name=source.version value=1.7/
property name=target.version value=1.7/
@@ -1098,6 +1099,7 @@
   jvmarg 
value=-Dmigration-sstable-root=${test.data}/migration-sstables/
   jvmarg value=-Dcassandra.ring_delay_ms=1000/
   jvmarg value=-Dcassandra.tolerate_sstable_size=true/
+  jvmarg value=-Djava.io.tmpdir=${tmp.dir}/
 /testmacro
   /target

[6/6] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db1cf3e0
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db1cf3e0
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db1cf3e0

Branch: refs/heads/trunk
Commit: db1cf3e084c2b36d66610e78ef89351df5d5d76a
Parents: 8d570fa 77d58f9
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Mar 17 14:34:59 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Mar 17 14:34:59 2015 -0500

--
 CHANGES.txt | 2 +-
 build.xml   | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1cf3e0/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1cf3e0/build.xml
--
diff --cc build.xml
index 3fb4240,bdb3646..c99e8f8
--- a/build.xml
+++ b/build.xml
@@@ -62,9 -60,9 +62,10 @@@
  property name=test.runners value=1/
  property name=test.unit.src value=${test.dir}/unit/
  property name=test.long.src value=${test.dir}/long/
 +property name=test.microbench.src value=${test.dir}/microbench/
  property name=test.pig.src value=${test.dir}/pig/
  property name=dist.dir value=${build.dir}/dist/
+ property name=tmp.dir value=${java.io.tmpdir}/

property name=source.version value=1.7/
property name=target.version value=1.7/

[jira] [Commented] (CASSANDRA-8394) Cassandra 3.0 Auth changes

2015-03-17 Thread Jake Farrell (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365648#comment-14365648
]

Jake Farrell commented on CASSANDRA-8394:
-

Really like the direction this is heading, would love to also see auth be auto
replicated to all nodes without having to update the keyspace and run repair on
all nodes like the current implementation

Cassandra 3.0 Auth changes
--

Key: CASSANDRA-8394
URL: https://issues.apache.org/jira/browse/CASSANDRA-8394
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Sam Tunnicliffe
Labels: docs-impacting
Fix For: 3.0

This will be the task that will group all the 3.0 Auth changes, of which we
need quite some:
1. Implement Roles (CASSANDRA-7653)
2. Merge ISaslAwareAuthenticator and IAuthenticator, making IAuthenticator
SASL-only, and support full SASL capabilities (incl. proxy authentication -
see CASSANDRA-7686, CASSANDRA-8068)
3. Remove the current simplistic permissions cache with
implementation-specific user/credentials/permissions caches, at least in the
implementation we ship with C* (CASSANDRA-7715, CASSANDRA-8194)
4. Consider adding a way to split superuser-ness and the rights to manage
users (CASSANDRA-7216, CASSANDRA-8650)
5. Add permissions for types and functions (CASSANDRA-7557)
6. Consider re-introducing the TRUNCATE permission (CASSANDRA-8082)
7. Re-introduce the DESCRIBE permission (CASSANDRA-8163)

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] cassandra git commit: DatabaseDescriptor throws NPE when rpc_interface is used

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 c0f159b82 - 89cdfd8e0
  refs/heads/trunk ac9ccfdf7 - 8d570fa79


DatabaseDescriptor throws NPE when rpc_interface is used

patch by Ariel Weisberg; reviewed by Carl Yeksigian for CASSANDRA-8839


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/89cdfd8e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/89cdfd8e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/89cdfd8e

Branch: refs/heads/cassandra-2.1
Commit: 89cdfd8e075d8883d776d7f881735f1c25e3cb54
Parents: c0f159b
Author: Ariel Weisberg ar...@weisberg.ws
Authored: Tue Mar 17 19:27:54 2015 +0100
Committer: Robert Stupp sn...@snazy.de
Committed: Tue Mar 17 19:27:54 2015 +0100

--
 CHANGES.txt |   1 +
 conf/cassandra.yaml |  12 ++
 .../org/apache/cassandra/config/Config.java |   6 +-
 .../cassandra/config/DatabaseDescriptor.java| 205 +++
 .../config/YamlConfigurationLoader.java |  16 +-
 .../config/DatabaseDescriptorTest.java  | 139 -
 6 files changed, 277 insertions(+), 102 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7b8e0ad..30bf698 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.1.4
+ * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
  * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
  * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
  * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index cea12b3..2b43ba7 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -370,8 +370,14 @@ ssl_storage_port: 7001
 # address associated with the hostname (it might not be).
 #
 # Setting listen_address to 0.0.0.0 is always wrong.
+#
+# If you choose to specify the interface by name and the interface has an ipv4 
and an ipv6 address
+# you can specify which should be chosen using listen_interface_prefer_ipv6. 
If false the first ipv4
+# address will be used. If true the first ipv6 address will be used. Defaults 
to false preferring
+# ipv4. If there is only one address it will be selected regardless of 
ipv4/ipv6.
 listen_address: localhost
 # listen_interface: eth0
+# listen_interface_prefer_ipv6: false
 
 # Address to broadcast to other Cassandra nodes
 # Leaving this blank will set it to the same value as listen_address
@@ -422,8 +428,14 @@ start_rpc: true
 # set broadcast_rpc_address to a value other than 0.0.0.0.
 #
 # For security reasons, you should not expose this port to the internet.  
Firewall it if needed.
+#
+# If you choose to specify the interface by name and the interface has an ipv4 
and an ipv6 address
+# you can specify which should be chosen using rpc_interface_prefer_ipv6. If 
false the first ipv4
+# address will be used. If true the first ipv6 address will be used. Defaults 
to false preferring
+# ipv4. If there is only one address it will be selected regardless of 
ipv4/ipv6.
 rpc_address: localhost
 # rpc_interface: eth1
+# rpc_interface_prefer_ipv6: false
 
 # port for Thrift to listen for clients on
 rpc_port: 9160

http://git-wip-us.apache.org/repos/asf/cassandra/blob/89cdfd8e/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index ccd4467..fbbd1dd 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -34,7 +34,7 @@ import org.apache.cassandra.utils.FBUtilities;
 
 /**
  * A class that contains configuration properties for the cassandra node it 
runs within.
- * 
+ *
  * Properties declared as volatile can be mutated via JMX.
  */
 public class Config
@@ -101,12 +101,14 @@ public class Config
 public Integer ssl_storage_port = 7001;
 public String listen_address;
 public String listen_interface;
+public Boolean listen_interface_prefer_ipv6 = false;
 public String broadcast_address;
 public String internode_authenticator;
 
 public Boolean start_rpc = true;
 public String rpc_address;
 public String rpc_interface;
+public Boolean rpc_interface_prefer_ipv6 = false;
 public String broadcast_rpc_address;
 public Integer rpc_port = 9160;
 public Integer rpc_listen_backlog = 50;

[jira] [Commented] (CASSANDRA-8981) IndexSummaryManagerTest.testCompactionsRace failing on trunk

2015-03-17 Thread Joshua McKenzie (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365742#comment-14365742
]

Joshua McKenzie commented on CASSANDRA-8981:

That's what I get for trusting the output of git blame. ;)

IndexSummaryManagerTest.testCompactionsRace failing on trunk

Key: CASSANDRA-8981
URL: https://issues.apache.org/jira/browse/CASSANDRA-8981
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Joshua McKenzie
Priority: Minor
Fix For: 3.0

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming

2015-03-17 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365869#comment-14365869
 ] 

Yuki Morishita commented on CASSANDRA-8838:
---

Alright, I force pushed updates to dtests. This version is at least reliable on 
my machine.

https://github.com/yukim/cassandra-dtest/tree/CASSANDRA-8838

bq. With no_wait=True it waits for 2 seconds

That's not what I expected, I removed them all.

bq. I encountered another issue, in that the streaming completes before we kill 
node 1

{{node#start()}} sometimes goes too far before we want to kill the node. This 
version spawns a thread to watch log and kill node 1 so it reacts as expected.

I also fixed my new tests in {{replace_address_test.py}} that does not properly 
replacing.

 Resumable bootstrap streaming
 -

 Key: CASSANDRA-8838
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8838
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Minor
  Labels: dense-storage
 Fix For: 3.0


 This allows the bootstrapping node not to be streamed already received data.
 The bootstrapping node records received keyspace/ranges as one stream session 
 completes. When some sessions with other nodes fail, bootstrapping fails 
 also, though next time it re-bootstraps, already received keyspace/ranges are 
 skipped to be streamed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log

2015-03-17 Thread Ariel Weisberg (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365867#comment-14365867
 ] 

Ariel Weisberg commented on CASSANDRA-6809:
---

I think what Benedict is describing where we only deallocate if the queue has 
more than one available (and  3) is good idea. If the knob is set too low it 
might give better behavior at steady state?

I would still be +1 since I think it won't matter once we multi-thread 
compression (and I intend to do that for 3.1).

 Compressed Commit Log
 -

 Key: CASSANDRA-6809
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Assignee: Branimir Lambov
Priority: Minor
  Labels: docs-impacting, performance
 Fix For: 3.0

 Attachments: ComitLogStress.java, logtest.txt


 It seems an unnecessary oversight that we don't compress the commit log. 
 Doing so should improve throughput, but some care will need to be taken to 
 ensure we use as much of a segment as possible. I propose decoupling the 
 writing of the records from the segments. Basically write into a (queue of) 
 DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X 
 MB written to the CL (where X is ordinarily CLS size), and then pack as many 
 of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] cassandra git commit: Use stdout for progress and stats in sstableloader

2015-03-17 Thread yukim

Use stdout for progress and stats in sstableloader

patch by yukim; reviewed by Brandon Williams for CASSANDRA-8982


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db1a741e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db1a741e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db1a741e

Branch: refs/heads/trunk
Commit: db1a741ea2f30e992345bd06deab843313718775
Parents: 77d58f9
Author: Yuki Morishita yu...@apache.org
Authored: Tue Mar 17 14:00:54 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue Mar 17 14:37:55 2015 -0500

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/tools/BulkLoader.java | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6b8618b..56a7164 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -44,6 +44,7 @@
  * cassandra-stress support for varint (CASSANDRA-8882)
  * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
  * Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
+ * Use stdout for progress and stats in sstableloader (CASSANDRA-8982)
 Merged from 2.0:
  * Allow specifying the tmp dir (CASSANDRA-7712)
  * Improve compaction estimated tasks estimation (CASSANDRA-8904)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java 
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index 73f4ec5..88a4404 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -227,7 +227,7 @@ public class BulkLoader
 peak = average;
 sb.append((avg: ).append(average).append( MB/s));
 
-System.err.print(sb.toString());
+System.out.print(sb.toString());
 }
 }
 
@@ -250,7 +250,7 @@ public class BulkLoader
 sb.append(String.format(   %-30s: %-10d%n, Total duration (ms): 
, durationMS));
 sb.append(String.format(   %-30s: %-10d%n, Average transfer 
rate (MB/s): , + average));
 sb.append(String.format(   %-30s: %-10d%n, Peak transfer rate 
(MB/s): , + peak));
-System.err.println(sb.toString());
+System.out.println(sb.toString());
 }
 }

[1/3] cassandra git commit: Use stdout for progress and stats in sstableloader

2015-03-17 Thread yukim

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 77d58f900 - db1a741ea
  refs/heads/trunk db1cf3e08 - 50ea0182e


Use stdout for progress and stats in sstableloader

patch by yukim; reviewed by Brandon Williams for CASSANDRA-8982


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/db1a741e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/db1a741e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/db1a741e

Branch: refs/heads/cassandra-2.1
Commit: db1a741ea2f30e992345bd06deab843313718775
Parents: 77d58f9
Author: Yuki Morishita yu...@apache.org
Authored: Tue Mar 17 14:00:54 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue Mar 17 14:37:55 2015 -0500

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/tools/BulkLoader.java | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6b8618b..56a7164 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -44,6 +44,7 @@
  * cassandra-stress support for varint (CASSANDRA-8882)
  * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
  * Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
+ * Use stdout for progress and stats in sstableloader (CASSANDRA-8982)
 Merged from 2.0:
  * Allow specifying the tmp dir (CASSANDRA-7712)
  * Improve compaction estimated tasks estimation (CASSANDRA-8904)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/db1a741e/src/java/org/apache/cassandra/tools/BulkLoader.java
--
diff --git a/src/java/org/apache/cassandra/tools/BulkLoader.java 
b/src/java/org/apache/cassandra/tools/BulkLoader.java
index 73f4ec5..88a4404 100644
--- a/src/java/org/apache/cassandra/tools/BulkLoader.java
+++ b/src/java/org/apache/cassandra/tools/BulkLoader.java
@@ -227,7 +227,7 @@ public class BulkLoader
 peak = average;
 sb.append((avg: ).append(average).append( MB/s));
 
-System.err.print(sb.toString());
+System.out.print(sb.toString());
 }
 }
 
@@ -250,7 +250,7 @@ public class BulkLoader
 sb.append(String.format(   %-30s: %-10d%n, Total duration (ms): 
, durationMS));
 sb.append(String.format(   %-30s: %-10d%n, Average transfer 
rate (MB/s): , + average));
 sb.append(String.format(   %-30s: %-10d%n, Peak transfer rate 
(MB/s): , + peak));
-System.err.println(sb.toString());
+System.out.println(sb.toString());
 }
 }

[jira] [Commented] (CASSANDRA-8936) Remove IRequestSink, clean up IMessageSink use

2015-03-17 Thread Jeremiah Jordan (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365912#comment-14365912
 ] 

Jeremiah Jordan commented on CASSANDRA-8936:


LGTM +1

 Remove IRequestSink, clean up IMessageSink use
 --

 Key: CASSANDRA-8936
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8936
 Project: Cassandra
  Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Priority: Trivial
 Fix For: 3.0

 Attachments: 8936-v2.txt, 8936.txt


 Nothing's using IRequestSink anymore, and IMessageSink interface does not 
 ideally represent its current uses.
 A tirival cleanup patch that deals with both issues attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url

2015-03-17 Thread Brandon Williams (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365925#comment-14365925
 ] 

Brandon Williams commented on CASSANDRA-7410:
-

Seems like waiting on CASSANDRA-8358 is the right move, at least for trunk, so 
we don't compound a problem.

 Pig support for BulkOutputFormat as a parameter in url
 --

 Key: CASSANDRA-7410
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
Priority: Minor
 Fix For: 2.0.14

 Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, 
 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, 
 CASSANDRA-7410-v2-2.1-branch.txt


 Add BulkOutputFormat support in Pig url



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: Clean up I*Sink testing hooks

2015-03-17 Thread aleksey

Repository: cassandra
Updated Branches:
  refs/heads/trunk 50ea0182e - 642546aba


Clean up I*Sink testing hooks

patch by Aleksey Yeschenko; reviewed by Jeremiah Jordan for
CASSANDRA-8936


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/642546ab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/642546ab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/642546ab

Branch: refs/heads/trunk
Commit: 642546abac7527b640e228cdeb1993c53fc71582
Parents: 50ea018
Author: Aleksey Yeschenko alek...@apache.org
Authored: Tue Mar 17 13:10:21 2015 -0700
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Tue Mar 17 13:10:21 2015 -0700

--
 .../org/apache/cassandra/net/IMessageSink.java  |  37 +++
 .../apache/cassandra/net/MessagingService.java  |  54 +-
 .../apache/cassandra/service/StorageProxy.java  |  32 ++
 .../org/apache/cassandra/sink/IMessageSink.java |  42 
 .../org/apache/cassandra/sink/IRequestSink.java |  32 --
 .../org/apache/cassandra/sink/SinkManager.java  | 104 ---
 .../apache/cassandra/repair/ValidatorTest.java  |  31 +++---
 .../apache/cassandra/service/RemoveTest.java|   3 +-
 8 files changed, 88 insertions(+), 247 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/642546ab/src/java/org/apache/cassandra/net/IMessageSink.java
--
diff --git a/src/java/org/apache/cassandra/net/IMessageSink.java 
b/src/java/org/apache/cassandra/net/IMessageSink.java
new file mode 100644
index 000..5150901
--- /dev/null
+++ b/src/java/org/apache/cassandra/net/IMessageSink.java
@@ -0,0 +1,37 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * License); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an AS IS BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.net;
+
+import java.net.InetAddress;
+
+public interface IMessageSink
+{
+/**
+ * Allow or drop an outgoing message
+ *
+ * @return true if the message is allowed, false if it should be dropped
+ */
+boolean allowOutgoingMessage(MessageOut message, int id, InetAddress to);
+
+/**
+ * Allow or drop an incoming message
+ *
+ * @return true if the message is allowed, false if it should be dropped
+ */
+boolean allowIncomingMessage(MessageIn message, int id);
+}

http://git-wip-us.apache.org/repos/asf/cassandra/blob/642546ab/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index 1f952a3..fb699e4 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -25,6 +25,7 @@ import java.nio.channels.ClosedChannelException;
 import java.nio.channels.ServerSocketChannel;
 import java.util.*;
 import java.util.concurrent.ConcurrentMap;
+import java.util.concurrent.CopyOnWriteArraySet;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicInteger;
 
@@ -60,7 +61,6 @@ import org.apache.cassandra.io.util.FileUtils;
 import org.apache.cassandra.locator.ILatencySubscriber;
 import org.apache.cassandra.metrics.ConnectionMetrics;
 import org.apache.cassandra.metrics.DroppedMessageMetrics;
-import org.apache.cassandra.sink.SinkManager;
 import org.apache.cassandra.repair.messages.RepairMessage;
 import org.apache.cassandra.security.SSLFactory;
 import org.apache.cassandra.service.*;
@@ -308,10 +308,24 @@ public final class MessagingService implements 
MessagingServiceMBean
 // protocol versions of the other nodes in the cluster
 private final ConcurrentMapInetAddress, Integer versions = new 
NonBlockingHashMapInetAddress, Integer();
 
+// message sinks are a testing hook
+private final SetIMessageSink messageSinks = new CopyOnWriteArraySet();
+
+public void addMessageSink(IMessageSink sink)
+{
+messageSinks.add(sink);
+}
+
+public void clearMessageSinks()
+

[3/3] cassandra git commit: Merge branch 'cassandra-2.1' into trunk

2015-03-17 Thread yukim

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/50ea0182
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/50ea0182
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/50ea0182

Branch: refs/heads/trunk
Commit: 50ea0182e784413e2fba89a97bf7d0c6b8d4570b
Parents: db1cf3e db1a741
Author: Yuki Morishita yu...@apache.org
Authored: Tue Mar 17 14:38:33 2015 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Tue Mar 17 14:38:33 2015 -0500

--
 CHANGES.txt | 1 +
 src/java/org/apache/cassandra/tools/BulkLoader.java | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/50ea0182/CHANGES.txt
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/50ea0182/src/java/org/apache/cassandra/tools/BulkLoader.java
--

[jira] [Commented] (CASSANDRA-8358) Bundled tools shouldn't be using Thrift API

2015-03-17 Thread Aleksey Yeschenko (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-8358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365890#comment-14365890
]

Aleksey Yeschenko commented on CASSANDRA-8358:
--

This ticket is headed into 3.0, not 2.1. Will be committed as soon as I review.

Bundled tools shouldn't be using Thrift API
---

Key: CASSANDRA-8358
URL: https://issues.apache.org/jira/browse/CASSANDRA-8358
Project: Cassandra
Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Philip Thompson
Fix For: 3.0

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365910#comment-14365910
 ] 

Alex Liu commented on CASSANDRA-7410:
-

[~brandon.williams] Do you have time to review it?

 Pig support for BulkOutputFormat as a parameter in url
 --

 Key: CASSANDRA-7410
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
Priority: Minor
 Fix For: 2.0.14

 Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, 
 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, 
 CASSANDRA-7410-v2-2.1-branch.txt


 Add BulkOutputFormat support in Pig url



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7712) temporary files need to be cleaned by unit tests


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler updated CASSANDRA-7712:
--
Attachment: 7712_workaround.txt

Attached 7712_workaround.txt patch to set a {{tmp.dir}} property and use that 
as a {{jvmarg}} in testlist.  Tested this with and without passing in the 
option with no problems - the jvm also created my custom tmp.dir for me.  This 
workaround provides us the ability to set the temporary data dir within our git 
workspace, and since we are using {{git clean -xdf}} at the beginning of every 
unit test run, this data will no longer accumulate.

 temporary files need to be cleaned by unit tests
 

 Key: CASSANDRA-7712
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Michael Shuler
Priority: Minor
  Labels: bootcamp, lhf
 Fix For: 2.0.14

 Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt, 
 7712-v3.txt, 7712_workaround.txt, CASSANDRA-7712_apache_cassandra_2.0.txt


 There are many unit test temporary files left behind after test runs. In the 
 case of CI servers, I have seen 70,000 files accumulate in /tmp over a 
 period of time. Each unit test should make an effort to remove its temporary 
 files when the test is completed.
 My current unit test cleanup block:
 {noformat}
 # clean up after unit tests..
 rm -rf  /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest* 
 /tmp/Keyspace1* \
 /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest* 
 /tmp/SSTableImportTest* \
 /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest* 
 /tmp/ValuesWithQuotes* \
 /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest* 
 /tmp/liblz4-java*.so /tmp/readtest* \
 /tmp/set_length_during_read_mode* /tmp/set_negative_length* 
 /tmp/snappy-*.so
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (CASSANDRA-7712) temporary files need to be cleaned by unit tests


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Shuler reassigned CASSANDRA-7712:
-

Assignee: Michael Shuler

 temporary files need to be cleaned by unit tests
 

 Key: CASSANDRA-7712
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7712
 Project: Cassandra
  Issue Type: Test
  Components: Tests
Reporter: Michael Shuler
Assignee: Michael Shuler
Priority: Minor
  Labels: bootcamp, lhf
 Fix For: 2.0.14

 Attachments: 7712-hung-CliTest_system.log.gz, 7712-v2.txt, 
 7712-v3.txt, 7712_workaround.txt, CASSANDRA-7712_apache_cassandra_2.0.txt


 There are many unit test temporary files left behind after test runs. In the 
 case of CI servers, I have seen 70,000 files accumulate in /tmp over a 
 period of time. Each unit test should make an effort to remove its temporary 
 files when the test is completed.
 My current unit test cleanup block:
 {noformat}
 # clean up after unit tests..
 rm -rf  /tmp/140*-0 /tmp/CFWith* /tmp/Counter1* /tmp/DescriptorTest* 
 /tmp/Keyspace1* \
 /tmp/KeyStreamingTransferTestSpace* /tmp/SSTableExportTest* 
 /tmp/SSTableImportTest* \
 /tmp/Standard1* /tmp/Statistics.db* /tmp/StreamingTransferTest* 
 /tmp/ValuesWithQuotes* \
 /tmp/cassandra* /tmp/jna-* /tmp/ks-cf-ib-1-* /tmp/lengthtest* 
 /tmp/liblz4-java*.so /tmp/readtest* \
 /tmp/set_length_during_read_mode* /tmp/set_negative_length* 
 /tmp/snappy-*.so
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[3/6] cassandra git commit: Allow specifying the tmp dir

Allow specifying the tmp dir

Patch by mshuler, reviewed by brandonwilliams for CASSANDRA-7712


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2aa67ea
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2aa67ea
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2aa67ea

Branch: refs/heads/trunk
Commit: b2aa67ea298044f4c4ad905e54963f0c03643b68
Parents: 0814737
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Mar 17 14:33:27 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Mar 17 14:33:27 2015 -0500

--
 CHANGES.txt | 1 +
 build.xml   | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 65f2da4..f10a89a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Allow specifying the tmp dir (CASSANDRA-7712)
  * Improve compaction estimated tasks estimation (CASSANDRA-8904)
  * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
  * Expose commit log archive status via JMX (CASSANDRA-8734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/build.xml
--
diff --git a/build.xml b/build.xml
index bdbd10a..f5e14ba 100644
--- a/build.xml
+++ b/build.xml
@@ -60,6 +60,7 @@
 property name=test.long.src value=${test.dir}/long/
 property name=test.pig.src value=${test.dir}/pig/
 property name=dist.dir value=${build.dir}/dist/
+property name=tmp.dir value=${java.io.tmpdir}/

property name=source.version value=1.7/
property name=target.version value=1.7/
@@ -1098,6 +1099,7 @@
   jvmarg 
value=-Dmigration-sstable-root=${test.data}/migration-sstables/
   jvmarg value=-Dcassandra.ring_delay_ms=1000/
   jvmarg value=-Dcassandra.tolerate_sstable_size=true/
+  jvmarg value=-Djava.io.tmpdir=${tmp.dir}/
 /testmacro
   /target

[4/6] cassandra git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/77d58f90
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/77d58f90
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/77d58f90

Branch: refs/heads/trunk
Commit: 77d58f900efac62c7c8dcc63c3de5ecb0fe4e343
Parents: 89cdfd8 b2aa67e
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Mar 17 14:34:26 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Mar 17 14:34:26 2015 -0500

--
 CHANGES.txt | 1 +
 build.xml   | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/CHANGES.txt
--
diff --cc CHANGES.txt
index 30bf698,f10a89a..6b8618b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,50 -1,5 +1,51 @@@
 -2.0.14:
 +2.1.4
 + * DatabaseDescriptor throws NPE when rpc_interface is used (CASSANDRA-8839)
 + * Don't check if an sstable is live for offline compactions (CASSANDRA-8841)
 + * Don't set clientMode in SSTableLoader (CASSANDRA-8238)
 + * Fix SSTableRewriter with disabled early open (CASSANDRA-8535)
 + * Allow invalidating permissions and cache time (CASSANDRA-8722)
 + * Log warning when queries that will require ALLOW FILTERING in Cassandra 3.0
 +   are executed (CASSANDRA-8418)
 + * Fix cassandra-stress so it respects the CL passed in user mode 
(CASSANDRA-8948)
 + * Fix rare NPE in ColumnDefinition#hasIndexOption() (CASSANDRA-8786)
 + * cassandra-stress reports per-operation statistics, plus misc 
(CASSANDRA-8769)
 + * Add SimpleDate (cql date) and Time (cql time) types (CASSANDRA-7523)
 + * Use long for key count in cfstats (CASSANDRA-8913)
 + * Make SSTableRewriter.abort() more robust to failure (CASSANDRA-8832)
 + * Remove cold_reads_to_omit from STCS (CASSANDRA-8860)
 + * Make EstimatedHistogram#percentile() use ceil instead of floor 
(CASSANDRA-8883)
 + * Fix top partitions reporting wrong cardinality (CASSANDRA-8834)
 + * Fix rare NPE in KeyCacheSerializer (CASSANDRA-8067)
 + * Pick sstables for validation as late as possible inc repairs 
(CASSANDRA-8366)
 + * Fix commitlog getPendingTasks to not increment (CASSANDRA-8856)
 + * Fix parallelism adjustment in range and secondary index queries
 +   when the first fetch does not satisfy the limit (CASSANDRA-8856)
 + * Check if the filtered sstables is non-empty in STCS (CASSANDRA-8843)
 + * Upgrade java-driver used for cassandra-stress (CASSANDRA-8842)
 + * Fix CommitLog.forceRecycleAllSegments() memory access error 
(CASSANDRA-8812)
 + * Improve assertions in Memory (CASSANDRA-8792)
 + * Fix SSTableRewriter cleanup (CASSANDRA-8802)
 + * Introduce SafeMemory for CompressionMetadata.Writer (CASSANDRA-8758)
 + * 'nodetool info' prints exception against older node (CASSANDRA-8796)
 + * Ensure SSTableReader.last corresponds exactly with the file end 
(CASSANDRA-8750)
 + * Make SSTableWriter.openEarly more robust and obvious (CASSANDRA-8747)
 + * Enforce SSTableReader.first/last (CASSANDRA-8744)
 + * Cleanup SegmentedFile API (CASSANDRA-8749)
 + * Avoid overlap with early compaction replacement (CASSANDRA-8683)
 + * Safer Resource Management++ (CASSANDRA-8707)
 + * Write partition size estimates into a system table (CASSANDRA-7688)
 + * cqlsh: Fix keys() and full() collection indexes in DESCRIBE output
 +   (CASSANDRA-8154)
 + * Show progress of streaming in nodetool netstats (CASSANDRA-8886)
 + * IndexSummaryBuilder utilises offheap memory, and shares data between
 +   each IndexSummary opened from it (CASSANDRA-8757)
 + * markCompacting only succeeds if the exact SSTableReader instances being 
 +   marked are in the live set (CASSANDRA-8689)
 + * cassandra-stress support for varint (CASSANDRA-8882)
 + * Fix Adler32 digest for compressed sstables (CASSANDRA-8778)
 + * Add nodetool statushandoff/statusbackup (CASSANDRA-8912)
 +Merged from 2.0:
+  * Allow specifying the tmp dir (CASSANDRA-7712)
   * Improve compaction estimated tasks estimation (CASSANDRA-8904)
   * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
   * Expose commit log archive status via JMX (CASSANDRA-8734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/77d58f90/build.xml
--
diff --cc build.xml
index 4ced4e2,f5e14ba..bdb3646
--- a/build.xml
+++ b/build.xml
@@@ -1182,21 -1099,8 +1183,22 @@@
jvmarg 
value=-Dmigration-sstable-root=${test.data}/migration-sstables/
jvmarg value=-Dcassandra.ring_delay_ms=1000/
jvmarg value=-Dcassandra.tolerate_sstable_size=true/
+   jvmarg value=-Djava.io.tmpdir=${tmp.dir}/
  /testmacro
 +fileset dir=${test.unit.src}
 +

[1/6] cassandra git commit: Allow specifying the tmp dir

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 08147376a - b2aa67ea2
  refs/heads/cassandra-2.1 89cdfd8e0 - 77d58f900
  refs/heads/trunk 8d570fa79 - db1cf3e08


Allow specifying the tmp dir

Patch by mshuler, reviewed by brandonwilliams for CASSANDRA-7712


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2aa67ea
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2aa67ea
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2aa67ea

Branch: refs/heads/cassandra-2.0
Commit: b2aa67ea298044f4c4ad905e54963f0c03643b68
Parents: 0814737
Author: Brandon Williams brandonwilli...@apache.org
Authored: Tue Mar 17 14:33:27 2015 -0500
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Tue Mar 17 14:33:27 2015 -0500

--
 CHANGES.txt | 1 +
 build.xml   | 2 ++
 2 files changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 65f2da4..f10a89a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.14:
+ * Allow specifying the tmp dir (CASSANDRA-7712)
  * Improve compaction estimated tasks estimation (CASSANDRA-8904)
  * Fix duplicate up/down messages sent to native clients (CASSANDRA-7816)
  * Expose commit log archive status via JMX (CASSANDRA-8734)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2aa67ea/build.xml
--
diff --git a/build.xml b/build.xml
index bdbd10a..f5e14ba 100644
--- a/build.xml
+++ b/build.xml
@@ -60,6 +60,7 @@
 property name=test.long.src value=${test.dir}/long/
 property name=test.pig.src value=${test.dir}/pig/
 property name=dist.dir value=${build.dir}/dist/
+property name=tmp.dir value=${java.io.tmpdir}/

property name=source.version value=1.7/
property name=target.version value=1.7/
@@ -1098,6 +1099,7 @@
   jvmarg 
value=-Dmigration-sstable-root=${test.data}/migration-sstables/
   jvmarg value=-Dcassandra.ring_delay_ms=1000/
   jvmarg value=-Dcassandra.tolerate_sstable_size=true/
+  jvmarg value=-Djava.io.tmpdir=${tmp.dir}/
 /testmacro
   /target

[jira] [Commented] (CASSANDRA-7410) Pig support for BulkOutputFormat as a parameter in url


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365884#comment-14365884
 ] 

Alex Liu commented on CASSANDRA-7410:
-

Waiting for CASSANDRA-8358

 Pig support for BulkOutputFormat as a parameter in url
 --

 Key: CASSANDRA-7410
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7410
 Project: Cassandra
  Issue Type: Improvement
  Components: Hadoop
Reporter: Alex Liu
Assignee: Alex Liu
Priority: Minor
 Fix For: 2.0.14

 Attachments: 7410-2.0-branch.txt, 7410-2.1-branch.txt, 
 7410-v2-2.0-branch.txt, 7410-v3-2.0-branch.txt, 
 CASSANDRA-7410-v2-2.1-branch.txt


 Add BulkOutputFormat support in Pig url



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-5791) A nodetool command to validate all sstables in a node

2015-03-17 Thread Branimir Lambov (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14364963#comment-14364963
 ] 

Branimir Lambov commented on CASSANDRA-5791:


+1

[~benedict], I think the patch is ready. Could you commit it?

 A nodetool command to validate all sstables in a node
 -

 Key: CASSANDRA-5791
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5791
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: sankalp kohli
Assignee: Jeff Jirsa
Priority: Minor
 Attachments: cassandra-5791-patch-3.diff, cassandra-5791.patch-2


 CUrrently there is no nodetool command to validate all sstables on disk. The 
 only way to do this is to run a repair and see if it succeeds. But we cannot 
 repair the system keyspace. 
 Also we can run upgrade sstables but that re writes all the sstables. 
 This command should check the hash of all sstables and return whether all 
 data is readable all not. This should NOT care about consistency. 
 The compressed sstables do not have hash so not sure how it will work there.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-8966) SequentialWriter should be Ref counted


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365069#comment-14365069
 ] 

Benedict commented on CASSANDRA-8966:
-

Just an FYI that I'm looking at this now also, and plan to commit a patch 
shortly including it, along with some final tidying up of the contracts in 
SSTableRewriter, SSTableWriter et al, to improve safety under failure

 SequentialWriter should be Ref counted
 --

 Key: CASSANDRA-8966
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8966
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Priority: Minor
 Fix For: 3.0


 A LHF to introduce some more resource safety



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

cassandra git commit: remove BINARY from DROPPABLE_VERBS patch by dbrosius reviewed by ayeschenko for CASSANDRA-8976

Repository: cassandra
Updated Branches:
  refs/heads/trunk 0b4cce86a - c9a6d7ce0


remove BINARY from DROPPABLE_VERBS
patch by dbrosius reviewed by ayeschenko for CASSANDRA-8976


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c9a6d7ce
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c9a6d7ce
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c9a6d7ce

Branch: refs/heads/trunk
Commit: c9a6d7ce05ff75412d5f40c740b33b33a7e813a0
Parents: 0b4cce8
Author: Dave Brosius dbros...@mebigfatguy.com
Authored: Tue Mar 17 09:39:33 2015 -0400
Committer: Dave Brosius dbros...@mebigfatguy.com
Committed: Tue Mar 17 09:40:54 2015 -0400

--
 src/java/org/apache/cassandra/net/MessagingService.java | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c9a6d7ce/src/java/org/apache/cassandra/net/MessagingService.java
--
diff --git a/src/java/org/apache/cassandra/net/MessagingService.java 
b/src/java/org/apache/cassandra/net/MessagingService.java
index a5cbfa7..1f952a3 100644
--- a/src/java/org/apache/cassandra/net/MessagingService.java
+++ b/src/java/org/apache/cassandra/net/MessagingService.java
@@ -289,8 +289,7 @@ public final class MessagingService implements 
MessagingServiceMBean
  * all correspond to client requests or something triggered by them; we 
don't want to
  * drop internal messages like bootstrap or repair notifications.
  */
-public static final EnumSetVerb DROPPABLE_VERBS = EnumSet.of(Verb.BINARY,
-   Verb._TRACE,
+public static final EnumSetVerb DROPPABLE_VERBS = EnumSet.of(Verb._TRACE,

Verb.MUTATION,

Verb.COUNTER_MUTATION,

Verb.READ_REPAIR,

[jira] [Commented] (CASSANDRA-8838) Resumable bootstrap streaming

2015-03-17 Thread Stefania (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14365081#comment-14365081
 ] 

Stefania commented on CASSANDRA-8838:
-

On my box it takes just under 4 seconds with both cassandra-2.1 and trunk. With 
cassandra-2.0 however, it takes just over 2 seconds. I'll try to spend more 
time investigating this.

 Resumable bootstrap streaming
 -

 Key: CASSANDRA-8838
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8838
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Priority: Minor
  Labels: dense-storage
 Fix For: 3.0


 This allows the bootstrapping node not to be streamed already received data.
 The bootstrapping node records received keyspace/ranges as one stream session 
 completes. When some sessions with other nodes fail, bootstrapping fails 
 also, though next time it re-bootstraps, already received keyspace/ranges are 
 skipped to be streamed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Borg Salling updated CASSANDRA-8978:
---
Description: 
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk 
load via sstableloader, occassionally I get the sporadic error shown below.

I can run the exact same job again - and it will succeed or fail with the same 
error at another location in the input stream. The error is appears to occur 
randomly - with the same input it may occur never, early or late in the run 
with no apparent logic or system.

I use five instances of CQLSSTableWriter in the application (to write 
redundantly to five different tables). But these instances do not exist at the 
same time; and thus never used concurrently.

{code}
09:26:33.582 [main] INFO  d.dma.ais.store.FileSSTableConverter - Finished 
processing directory, 369582175 packets was converted from /nas1/
Exception in thread main java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at 
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
at 
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
at 
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
at 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}

So far I overcome this problem by simply retrying with another run of the 
application in attempt to generate the sstables. But this is a rather time 
consuming and shaky approach - and I feel a bit uneasy relying on the produced 
sstables, though their contents appear to be correct when I sample them with 
cqlsh 'select' after load into Cassandra.



  was:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk 
load via sstableloader, occassionally I get the sporadic error shown below.

I can run the exact same job again - and it will succeed or fail with the same 
error at another location in the input stream. The error is appears to occur 
randomly - with the same input it may occur never, early or late in the run 
with no apparent logic or system.

I use five instances of CQLSSTableWriter in the application (to write 
redundantly to five different tables). But these instances do not exist at the 
same time; and thus never used concurrently.

{code}
09:26:33.582 [main] INFO  d.dma.ais.store.FileSSTableConverter - Finished 
processing directory, 369582175 packets was converted from /nas1/
Exception in thread main java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at

[jira] [Updated] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Borg Salling updated CASSANDRA-8978:
---
Description: 
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk 
load via sstableloader, occassionally I get the sporadic error shown below.

I can run the exact same job again - and it will succeed or fail with the same 
error at another location in the input stream. The error is appears to occur 
randomly - with the same input it may occur never, early or late in the run 
with no apparent logic or system.

I use five instances of CQLSSTableWriter in the application (to write 
redundantly to five different tables). But these instances do not exist at the 
same time; and thus never used concurrently.

{code}
09:26:33.582 [main] INFO  d.dma.ais.store.FileSSTableConverter - Finished 
processing directory, 369582175 packets was converted from /nas1/
Exception in thread main java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at 
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
at 
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
at 
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
at 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}



  was:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk 
load via sstableloader, occassionally get this sporadic error.

I can run the exact same job again - and it will succeed or fail with the same 
error at another location in the input stream. The error is appears to occur 
randomly - with the same input it may occur never, early or late in the run 
with no apparent logic or system.

I use five instances of CQLSSTableWriter in the application (to write 
redundantly to five different tables). But these instances do not exist at the 
same time; and thus never used concurrently.

{code}
09:26:33.582 [main] INFO  d.dma.ais.store.FileSSTableConverter - Finished 
processing directory, 369582175 packets was converted from /nas1/
Exception in thread main java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at

[jira] [Updated] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas Borg Salling updated CASSANDRA-8978:
---
Description: 
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk 
load via sstableloader, occassionally I get the sporadic error shown below.

I can run the exact same job again - and it will succeed or fail with the same 
error at another location in the input stream. The error is appears to occur 
randomly - with the same input it may occur never, early or late in the run 
with no apparent logic or system.

I use five instances of CQLSSTableWriter in the application (to write 
redundantly to five different tables). But these instances do not exist at the 
same time; and thus never used concurrently.

{code}
09:26:33.582 [main] INFO  d.dma.ais.store.FileSSTableConverter - Finished 
processing directory, 369582175 packets was converted from /nas1/
Exception in thread main java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.maybeSortCells(ArrayBackedSortedColumns.java:125)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.access$1100(ArrayBackedSortedColumns.java:44)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns$CellCollection.iterator(ArrayBackedSortedColumns.java:622)
at org.apache.cassandra.db.ColumnFamily.iterator(ColumnFamily.java:476)
at 
org.apache.cassandra.db.ColumnIndex$Builder.build(ColumnIndex.java:129)
at 
org.apache.cassandra.io.sstable.SSTableWriter.rawAppend(SSTableWriter.java:233)
at 
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:218)
at 
org.apache.cassandra.io.sstable.SSTableSimpleUnsortedWriter$DiskWriter.run(SSTableSimpleUnsortedWriter.java:215){code}

So far I overcome this problem by simply retrying with another run of the 
application in attempt to generate the sstables. But this is a rather time 
consuming and shaky approach - and I feel a bit uneasy relying on the produced 
sstables, though their contents appear to by correct with I sample them with 
cqlsh 'select' after load into Cassandra.



  was:
On long-running jobs with CQLSSTableWriter preparing sstables for later bulk 
load via sstableloader, occassionally I get the sporadic error shown below.

I can run the exact same job again - and it will succeed or fail with the same 
error at another location in the input stream. The error is appears to occur 
randomly - with the same input it may occur never, early or late in the run 
with no apparent logic or system.

I use five instances of CQLSSTableWriter in the application (to write 
redundantly to five different tables). But these instances do not exist at the 
same time; and thus never used concurrently.

{code}
09:26:33.582 [main] INFO  d.dma.ais.store.FileSSTableConverter - Finished 
processing directory, 369582175 packets was converted from /nas1/
Exception in thread main java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at dk.dma.commons.app.CliCommandList$1.execute(CliCommandList.java:50)
at dk.dma.commons.app.CliCommandList.invoke(CliCommandList.java:80)
at dk.dma.ais.store.Main.main(Main.java:34)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 297868
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.append(ArrayBackedSortedColumns.java:196)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.appendOrReconcile(ArrayBackedSortedColumns.java:191)
at 
org.apache.cassandra.db.ArrayBackedSortedColumns.sortCells(ArrayBackedSortedColumns.java:176)
at

[jira] [Updated] (CASSANDRA-8977) Potential DOS Exposure from Netty


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-8977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-8977:
--
Assignee: Brandon Williams

 Potential DOS Exposure from Netty
 -

 Key: CASSANDRA-8977
 URL: https://issues.apache.org/jira/browse/CASSANDRA-8977
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benjamin Coverston
Assignee: Brandon Williams

 CVE-2014-3488 : The SslHandler in Netty before 3.9.2 allows remote attackers 
 to cause a denial of service (infinite loop and CPU consumption) via a 
 crafted SSLv2Hello message.
 2.0 currently uses Netty 3.6.6 
 Upgrading to 3.9.2+ will fix this issue, and patch a few other known 
 vulnerabilities that we do not currently expose.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-8978) CQLSSTableWriter causes ArrayIndexOutOfBoundsException