date:20140917

[
https://issues.apache.org/jira/browse/CASSANDRA-7937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136794#comment-14136794
]

Benedict edited comment on CASSANDRA-7937 at 9/17/14 6:03 AM:
--

Under these workload conditions, CASSANDRA-3852 is unlikely to help; it is more
for reads. CASSANDRA-6812 will help for writes. It in no way affects the
concept of load shedding or backpressure, just write latency spikes and hence
timeouts.

There are some issues with load shedding by itself, though, at least as it
stands in Cassandra. Take your example cluster: A..E, with RF=3. Let's say B
_and_ C are now load shedding. The result is that we see 1/5th of requests
queue up and block in A,D and E, which within a short window results in all
requests to those servers being blocked, as all rpc threads are waiting for
results that will never come. Now, with explicit back pressure (whether it be
blocking consumption of network messages or signalling the coordinator we are
load shedding) we could detect this problem and start dropping those requests
on the floor in A, D and E and continue to serve other requests.

FTR, we do already have an accidental back pressure system built into
Incoming/OutboundTcpConnection, with only negative consequences. If the remote
server is blocked due to GC (or some other event), it will not consume its
IncomingTcpConnection which will cause our coordinator's queue to that node to
back up. This is something we should deal with, and introducing explicit back
pressure of the same kind when a node cannot cope with its load would allow us
to deal with both situations using the same mechanism. (We do sort of try to
deal with this, but only as more messages arrive, which is unlikely to happen
when our client rpc threads get filled up waiting for these responses)

was (Author: benedict):
Under these workload conditions, CASSANDRA-3852 is unlikely to help; it is more
for reads. CASSANDRA-6812 will help for writes. It in no way affects the
concept of load shedding or backpressure, just write latency spikes and hence
timeouts.

Apply backpressure gently when overloaded with writes
-

Key: CASSANDRA-7937
URL: https://issues.apache.org/jira/browse/CASSANDRA-7937
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: Cassandra 2.0
Reporter: Piotr Kołaczkowski
Labels: performance

When writing huge amounts of data into C* cluster from analytic tools like
Hadoop or Apache Spark, we can see that often C* can't keep up with the load.
This is because analytic tools typically write data as fast as they can in
parallel, from many nodes and they are not artificially rate-limited, so C*
is the bottleneck here. Also, increasing the number of nodes doesn't really
help, because in a collocated setup this also increases number of
Hadoop/Spark nodes (writers) and although possible write performance is
higher, the problem still remains.
We observe the following behavior:
1. data is ingested at an extreme fast pace into memtables and flush queue
fills up
2. the available memory limit for memtables is reached and writes are no
longer accepted
3. the application gets hit by write timeout, and retries repeatedly, in
vain
4. after several failed attempts to write, the job gets aborted
Desired behaviour:
1. data is ingested at an extreme fast pace into memtables and flush queue
fills up
2. after exceeding some memtable fill threshold, C* applies adaptive rate
limiting to writes - the more the buffers are

[jira] [Commented] (CASSANDRA-7282) Faster Memtable map


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136884#comment-14136884
 ] 

Benedict commented on CASSANDRA-7282:
-

To add some more data to this discussion, I decided quickly to isolate just the 
CSLM and NBHOM for comparison. Code available 
[here|https://github.com/belliottsmith/bench]. 

Using JMH for writes to an append-only collection is tricky, since any 
increased throughput rate is naturally throttling and incurs greater GC and 
algorithmic overhead, so these numbers are likely to be understating the yield, 
but the NBHOM is anywhere from 1.75-2.75x the throughput of CSLM, depending on 
if the workload is read-heavy or write-heavy. It is likely the real yield for 
writes is inbetween these two numbers.

{noformat}
b.b.c.HashOrderedCollections.test  4000   0.9CSLM  
thrpt   10  3593.788  196.459  ops/ms
b.b.c.HashOrderedCollections.test  4000   0.9   NBHOM  
thrpt   10  9299.991  179.408  ops/ms
b.b.c.HashOrderedCollections.test  4000   0.5CSLM  
thrpt   10  2402.021   79.321  ops/ms
b.b.c.HashOrderedCollections.test  4000   0.5   NBHOM  
thrpt   10  5648.160  294.727  ops/ms
b.b.c.HashOrderedCollections.test  4000   0.1CSLM  
thrpt   10  2089.597   71.228  ops/ms
b.b.c.HashOrderedCollections.test  4000   0.1   NBHOM  
thrpt   10  3665.048  138.384  ops/ms
b.b.c.HashOrderedCollections.test  4000 0CSLM  
thrpt   10  2008.595   46.308  ops/ms
b.b.c.HashOrderedCollections.test  4000 0   NBHOM  
thrpt   10  3515.328  214.304  ops/ms
{noformat}

So, as we reduce other overheads in the application and increase memtable 
capacity through further offheap improvements, any yield we are seeing 
application side can only go up.

 Faster Memtable map
 ---

 Key: CASSANDRA-7282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 3.0

 Attachments: profile.yaml, reads.svg, run1.svg, writes.svg


 Currently we maintain a ConcurrentSkipLastMap of DecoratedKey - Partition in 
 our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
 majority of users use a hash partitioner, it occurs to me we could maintain a 
 hybrid ordered list / hash map. The list would impose the normal order on the 
 collection, but a hash index would live alongside as part of the same data 
 structure, simply mapping into the list and permitting O(1) lookups and 
 inserts.
 I've chosen to implement this initial version as a linked-list node per item, 
 but we can optimise this in future by storing fatter nodes that permit a 
 cache-line's worth of hashes to be checked at once,  further reducing the 
 constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7282) Faster Memtable map


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136908#comment-14136908
 ] 

Benedict commented on CASSANDRA-7282:
-

Some more numbers, with a warmup dataset to populate the map so that 
variability due to throughput rate is reduced. These numbers show the NBHOM 
consistently around 3x+ faster, although it introduces per-run variability due 
to GC.

{noformat}
Benchmark(readWriteRatio)  (type)  (warmup)   Mode  
Samples Score  Score error   Units
b.b.c.HashOrderedCollections.test 0.9CSLM   500  thrpt  
  5  1392.273 2918.717  ops/ms
b.b.c.HashOrderedCollections.test 0.9   NBHOM   500  thrpt  
  5  5088.408 8964.885  ops/ms
b.b.c.HashOrderedCollections.test 0.5CSLM   500  thrpt  
  5  1128.637 2589.679  ops/ms
b.b.c.HashOrderedCollections.test 0.5   NBHOM   500  thrpt  
  5  3406.299 5606.085  ops/ms
b.b.c.HashOrderedCollections.test 0.1CSLM   500  thrpt  
  5   924.642 1802.045  ops/ms
b.b.c.HashOrderedCollections.test 0.1   NBHOM   500  thrpt  
  5  3311.107  999.896  ops/ms
b.b.c.HashOrderedCollections.test   0CSLM   500  thrpt  
  5   939.757 1776.641  ops/ms
b.b.c.HashOrderedCollections.test   0   NBHOM   500  thrpt  
  5  2781.503 4723.844  ops/ms
{noformat}

 Faster Memtable map
 ---

 Key: CASSANDRA-7282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 3.0

 Attachments: profile.yaml, reads.svg, run1.svg, writes.svg


 Currently we maintain a ConcurrentSkipLastMap of DecoratedKey - Partition in 
 our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
 majority of users use a hash partitioner, it occurs to me we could maintain a 
 hybrid ordered list / hash map. The list would impose the normal order on the 
 collection, but a hash index would live alongside as part of the same data 
 structure, simply mapping into the list and permitting O(1) lookups and 
 inserts.
 I've chosen to implement this initial version as a linked-list node per item, 
 but we can optimise this in future by storing fatter nodes that permit a 
 cache-line's worth of hashes to be checked at once,  further reducing the 
 constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7951) Add a Keep Alive operation to the Native Protocol

2014-09-17 Thread JIRA

Michaël Figuière created CASSANDRA-7951:
---

 Summary: Add a Keep Alive operation to the Native Protocol
 Key: CASSANDRA-7951
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7951
 Project: Cassandra
  Issue Type: New Feature
Reporter: Michaël Figuière
Priority: Minor


Socket level keep-alive option doesn't seem to bring a strong enough guarantee 
that the connection won't be dropped as some OS or Firewall level settings 
might force some different behavior.

To simplify things and bring the strong guarantee on this point that users 
expect, it would be interesting to add a {{NOOP}} or {{KEEPALIVE}}.

Note that we could also turn it into a {{HEARTBEAT}} operation that would 
expect a response from the server to allow client to regularly probe for 
Cassandra node health in situation where the queries traffic is too low to 
bring this probing implicitly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7951) Add a Keep Alive operation to the Native Protocol

2014-09-17 Thread JIRA


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137029#comment-14137029
 ] 

Michaël Figuière commented on CASSANDRA-7951:
-

A way to do it with the current specification of the protocol is to send an 
{{OPTIONS}} message, but this force the server to answer with a more complex 
response and it seems that we have enough room to spend an opcode on this 
specific requirement...

 Add a Keep Alive operation to the Native Protocol
 -

 Key: CASSANDRA-7951
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7951
 Project: Cassandra
  Issue Type: New Feature
Reporter: Michaël Figuière
Priority: Minor
  Labels: protocolv4

 Socket level keep-alive option doesn't seem to bring a strong enough 
 guarantee that the connection won't be dropped as some OS or Firewall level 
 settings might force some different behavior.
 To simplify things and bring the strong guarantee on this point that users 
 expect, it would be interesting to add a {{NOOP}} or {{KEEPALIVE}}.
 Note that we could also turn it into a {{HEARTBEAT}} operation that would 
 expect a response from the server to allow client to regularly probe for 
 Cassandra node health in situation where the queries traffic is too low to 
 bring this probing implicitly.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7949) LCS compaction low performance, many pending compactions, nodes are almost idle

[
https://issues.apache.org/jira/browse/CASSANDRA-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137054#comment-14137054
]

Nikolai Grigoriev commented on CASSANDRA-7949:
--

I see. I could try to switch to STCS now and see what happens.

My concern is that the issue seems to be permanent. Even after last night none
of the nodes (being vritually idle - the load was over) was able to eat through
the pending compactions. And, to my surprise, half of the nodes in the cluster
do not even compact fast enough - look at the graphs attached.

LCS compaction low performance, many pending compactions, nodes are almost
idle
---

Key: CASSANDRA-7949
URL: https://issues.apache.org/jira/browse/CASSANDRA-7949
Project: Cassandra
Issue Type: Bug
Components: Core
Environment: DSE 4.5.1-1, Cassandra 2.0.8
Reporter: Nikolai Grigoriev
Attachments: iostats.txt, nodetool_compactionstats.txt,
nodetool_tpstats.txt, pending compactions 2day, system.log.gz, vmstat.txt

I've been evaluating new cluster of 15 nodes (32 core, 6x800Gb SSD disks +
2x600Gb SAS, 128Gb RAM, OEL 6.5) and I've built a simulator that creates the
load similar to the load in our future product. Before running the simulator
I had to pre-generate enough data. This was done using Java code and DataStax
Java driver. To avoid going deep into details, two tables have been
generated. Each table currently has about 55M rows and between few dozens and
few thousands of columns in each row.
This data generation process was generating massive amount of non-overlapping
data. Thus, the activity was write-only and highly parallel. This is not the
type of the traffic that the system will have ultimately to deal with, it
will be mix of reads and updates to the existing data in the future. This is
just to explain the choice of LCS, not mentioning the expensive SSD disk
space.
At some point while generating the data I have noticed that the compactions
started to pile up. I knew that I was overloading the cluster but I still
wanted the genration test to complete. I was expecting to give the cluster
enough time to finish the pending compactions and get ready for real traffic.
However, after the storm of write requests have been stopped I have noticed
that the number of pending compactions remained constant (and even climbed up
a little bit) on all nodes. After trying to tune some parameters (like
setting the compaction bandwidth cap to 0) I have noticed a strange pattern:
the nodes were compacting one of the CFs in a single stream using virtually
no CPU and no disk I/O. This process was taking hours. After that it would be
followed by a short burst of few dozens of compactions running in parallel
(CPU at 2000%, some disk I/O - up to 10-20%) and then getting stuck again for
many hours doing one compaction at time. So it looks like this:
# nodetool compactionstats
pending tasks: 3351
compaction typekeyspace table completed
total unit progress
Compaction myks table_list1 66499295588
1910515889913 bytes 3.48%
Active compaction remaining time :n/a
# df -h
...
/dev/sdb1.5T 637G 854G 43% /cassandra-data/disk1
/dev/sdc1.5T 425G 1.1T 29% /cassandra-data/disk2
/dev/sdd1.5T 429G 1.1T 29% /cassandra-data/disk3
# find . -name **table_list1**Data** | grep -v snapshot | wc -l
1310
Among these files I see:
1043 files of 161Mb (my sstable size is 160Mb)
9 large files - 3 between 1 and 2Gb, 3 of 5-8Gb, 55Gb, 70Gb and 370Gb
263 files of various sized - between few dozens of Kb and 160Mb
I've been running the heavy load for about 1,5days and it's been close to 3
days after that and the number of pending compactions does not go down.
I have applied one of the not-so-obvious recommendations to disable
multithreaded compactions and that seems to be helping a bit - I see some
nodes started to have fewer pending compactions. About half of the cluster,
in fact. But even there I see they are sitting idle most of the time lazily
compacting in one stream with CPU at ~140% and occasionally doing the bursts
of compaction work for few minutes.
I am wondering if this is really a bug or something in the LCS logic that
would manifest itself only in such an edge case scenario where I have loaded
lots of unique data quickly.
By the way, I see this pattern only for one of two tables - the one that has
about 4 times more data than another (space-wise, number of rows is the
same). Looks like all these pending compactions are really only for that
larger table.
I'll be attaching the relevant logs shortly.

--
This message was

[jira] [Updated] (CASSANDRA-7949) LCS compaction low performance, many pending compactions, nodes are almost idle

[
https://issues.apache.org/jira/browse/CASSANDRA-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nikolai Grigoriev updated CASSANDRA-7949:
-
Attachment: pending compactions 2day

LCS compaction low performance, many pending compactions, nodes are almost
idle
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7949) LCS compaction low performance, many pending compactions, nodes are almost idle

[
https://issues.apache.org/jira/browse/CASSANDRA-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nikolai Grigoriev updated CASSANDRA-7949:
-
Attachment: (was: pending compactions 2day)

LCS compaction low performance, many pending compactions, nodes are almost
idle
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7949) LCS compaction low performance, many pending compactions, nodes are almost idle

[
https://issues.apache.org/jira/browse/CASSANDRA-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nikolai Grigoriev updated CASSANDRA-7949:
-
Attachment: pending compactions 2day.png

LCS compaction low performance, many pending compactions, nodes are almost
idle
---

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (CASSANDRA-7911) APT repo broken due to empty Packages file

2014-09-17 Thread Sylvain Lebresne (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sylvain Lebresne resolved CASSANDRA-7911.
-
Resolution: Fixed

Should be all fixed now

 APT repo broken due to empty Packages file
 --

 Key: CASSANDRA-7911
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7911
 Project: Cassandra
  Issue Type: Bug
Reporter: Ted Strzalkowski
Assignee: Sylvain Lebresne

 https://archive.apache.org/dist/cassandra/debian/dists/12x/main/binary-amd64/Packages
  is 0 bytes in size, causing new package installs to fail.
 || Name || Last Modified || Size || 
 | Packages | 2014-09-10 23:12 | 0 |



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7952) DataStax Agent Null Pointer Exception

Hari Sekhon created CASSANDRA-7952:
--

 Summary: DataStax Agent Null Pointer Exception
 Key: CASSANDRA-7952
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7952
 Project: Cassandra
  Issue Type: Bug
 Environment: DSE 4.5.1, DataStax OpsCenter  Agent 5.0.0
Reporter: Hari Sekhon


I've got a Null Pointer Exception in my DataStax OpsCenter Agent log, and it's 
not reporting in to the OpsCenter as a result. Here is the log
{code}
 INFO [StompConnection receiver] 2014-09-17 13:01:15,992 New JMX connection 
(127.0.0.1:7199)
 INFO [Jetty] 2014-09-17 13:01:16,019 Jetty server started
 INFO [Initialization] 2014-09-17 13:01:16,031 Using x.x.x.x as the cassandra 
broadcast address
 INFO [StompConnection receiver] 2014-09-17 13:01:16,032 Starting up agent 
collection.
 INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC address is  x.x.x.x
 INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC address is  
x.x.x.x
 INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC broadcast address is  
x.x.x.x
 INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC broadcast 
address is  x.x.x.x
 INFO [StompConnection receiver] 2014-09-17 13:01:16,163 Starting OS metric 
collectors (Linux)
 INFO [Initialization] 2014-09-17 13:01:16,166 Clearing ssl.truststore
 INFO [Initialization] 2014-09-17 13:01:16,166 Clearing ssl.truststore.password
 INFO [Initialization] 2014-09-17 13:01:16,167 Setting ssl.store.type to JKS
 INFO [Initialization] 2014-09-17 13:01:16,167 Clearing 
kerberos.service.principal.name
 INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.principal
 INFO [Initialization] 2014-09-17 13:01:16,167 Setting kerberos.useTicketCache 
to true
 INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.ticketCache
 INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.useKeyTab to 
true
 INFO [Initialization] 2014-09-17 13:01:16,168 Clearing kerberos.keyTab
 INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.renewTGT to true
 INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.debug to false
 INFO [StompConnection receiver] 2014-09-17 13:01:16,171 Starting Cassandra JMX 
metric collectors
 INFO [thrift-init] 2014-09-17 13:01:16,171 Connecting to Cassandra cluster: 
x.x.x.x (port 9160)
 INFO [StompConnection receiver] 2014-09-17 13:01:16,187 New JMX connection 
(127.0.0.1:7199)
 INFO [thrift-init] 2014-09-17 13:01:16,189 Downed Host Retry service started 
with queue size -1 and retry delay 10s
 INFO [thrift-init] 2014-09-17 13:01:16,192 Registering JMX 
me.prettyprint.cassandra.service_Agent 
Cluster:ServiceType=hector,MonitorType=hector
 INFO [pdp-loader] 2014-09-17 13:01:16,231 in execute with client 
org.apache.cassandra.thrift.Cassandra$Client@7a22c094
 INFO [pdp-loader] 2014-09-17 13:01:16,237 Attempting to load stored metric 
values.
 INFO [thrift-init] 2014-09-17 13:01:16,240 Connected to Cassandra cluster: PoC
 INFO [thrift-init] 2014-09-17 13:01:16,240 in execute with client 
org.apache.cassandra.thrift.Cassandra$Client@7a22c094
 INFO [thrift-init] 2014-09-17 13:01:16,240 Using partitioner: 
org.apache.cassandra.dht.Murmur3Partitioner
 INFO [jmx-metrics-1] 2014-09-17 13:01:21,181 New JMX connection 
(127.0.0.1:7199)
ERROR [StompConnection receiver] 2014-09-17 13:01:24,376 Failed to collect 
machine info
java.lang.NullPointerException
at clojure.lang.Numbers.ops(Numbers.java:942)
at clojure.lang.Numbers.divide(Numbers.java:157)
at 
opsagent.nodedetails.machine_info$get_machine_info.invoke(machine_info.clj:76)
at 
opsagent.nodedetails$get_static_properties$fn__4313.invoke(nodedetails.clj:161)
at 
opsagent.nodedetails$get_static_properties.invoke(nodedetails.clj:160)
at 
opsagent.nodedetails$get_longtime_values$fn__4426.invoke(nodedetails.clj:227)
at opsagent.nodedetails$get_longtime_values.invoke(nodedetails.clj:226)
at 
opsagent.nodedetails$send_all_nodedetails$fn__.invoke(nodedetails.clj:245)
at opsagent.jmx$jmx_wrap.doInvoke(jmx.clj:111)
at clojure.lang.RestFn.invoke(RestFn.java:410)
at opsagent.nodedetails$send_all_nodedetails.invoke(nodedetails.clj:241)
at opsagent.opsagent$post_interface_startup.doInvoke(opsagent.clj:125)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at opsagent.conf$handle_new_conf.invoke(conf.clj:179)
at opsagent.messaging$message_callback$fn__6059.invoke(messaging.clj:30)
at 
opsagent.messaging.proxy$java.lang.Object$StompConnection$Listener$7f16bc72.onMessage(Unknown
 Source)
at 
org.jgroups.client.StompConnection.notifyListeners(StompConnection.java:319)
at org.jgroups.client.StompConnection.run(StompConnection.java:269)
at java.lang.Thread.run(Thread.java:745)
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7952) DataStax Agent Null Pointer Exception


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sekhon updated CASSANDRA-7952:
---
Description: 
I've got a Null Pointer Exception in my DataStax OpsCenter Agent log, and it's 
not reporting in to the OpsCenter. Here is the log
{code}
 INFO [StompConnection receiver] 2014-09-17 13:01:15,992 New JMX connection 
(127.0.0.1:7199)
 INFO [Jetty] 2014-09-17 13:01:16,019 Jetty server started
 INFO [Initialization] 2014-09-17 13:01:16,031 Using x.x.x.x as the cassandra 
broadcast address
 INFO [StompConnection receiver] 2014-09-17 13:01:16,032 Starting up agent 
collection.
 INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC address is  x.x.x.x
 INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC address is  
x.x.x.x
 INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC broadcast address is  
x.x.x.x
 INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC broadcast 
address is  x.x.x.x
 INFO [StompConnection receiver] 2014-09-17 13:01:16,163 Starting OS metric 
collectors (Linux)
 INFO [Initialization] 2014-09-17 13:01:16,166 Clearing ssl.truststore
 INFO [Initialization] 2014-09-17 13:01:16,166 Clearing ssl.truststore.password
 INFO [Initialization] 2014-09-17 13:01:16,167 Setting ssl.store.type to JKS
 INFO [Initialization] 2014-09-17 13:01:16,167 Clearing 
kerberos.service.principal.name
 INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.principal
 INFO [Initialization] 2014-09-17 13:01:16,167 Setting kerberos.useTicketCache 
to true
 INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.ticketCache
 INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.useKeyTab to 
true
 INFO [Initialization] 2014-09-17 13:01:16,168 Clearing kerberos.keyTab
 INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.renewTGT to true
 INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.debug to false
 INFO [StompConnection receiver] 2014-09-17 13:01:16,171 Starting Cassandra JMX 
metric collectors
 INFO [thrift-init] 2014-09-17 13:01:16,171 Connecting to Cassandra cluster: 
x.x.x.x (port 9160)
 INFO [StompConnection receiver] 2014-09-17 13:01:16,187 New JMX connection 
(127.0.0.1:7199)
 INFO [thrift-init] 2014-09-17 13:01:16,189 Downed Host Retry service started 
with queue size -1 and retry delay 10s
 INFO [thrift-init] 2014-09-17 13:01:16,192 Registering JMX 
me.prettyprint.cassandra.service_Agent 
Cluster:ServiceType=hector,MonitorType=hector
 INFO [pdp-loader] 2014-09-17 13:01:16,231 in execute with client 
org.apache.cassandra.thrift.Cassandra$Client@7a22c094
 INFO [pdp-loader] 2014-09-17 13:01:16,237 Attempting to load stored metric 
values.
 INFO [thrift-init] 2014-09-17 13:01:16,240 Connected to Cassandra cluster: PoC
 INFO [thrift-init] 2014-09-17 13:01:16,240 in execute with client 
org.apache.cassandra.thrift.Cassandra$Client@7a22c094
 INFO [thrift-init] 2014-09-17 13:01:16,240 Using partitioner: 
org.apache.cassandra.dht.Murmur3Partitioner
 INFO [jmx-metrics-1] 2014-09-17 13:01:21,181 New JMX connection 
(127.0.0.1:7199)
ERROR [StompConnection receiver] 2014-09-17 13:01:24,376 Failed to collect 
machine info
java.lang.NullPointerException
at clojure.lang.Numbers.ops(Numbers.java:942)
at clojure.lang.Numbers.divide(Numbers.java:157)
at 
opsagent.nodedetails.machine_info$get_machine_info.invoke(machine_info.clj:76)
at 
opsagent.nodedetails$get_static_properties$fn__4313.invoke(nodedetails.clj:161)
at 
opsagent.nodedetails$get_static_properties.invoke(nodedetails.clj:160)
at 
opsagent.nodedetails$get_longtime_values$fn__4426.invoke(nodedetails.clj:227)
at opsagent.nodedetails$get_longtime_values.invoke(nodedetails.clj:226)
at 
opsagent.nodedetails$send_all_nodedetails$fn__.invoke(nodedetails.clj:245)
at opsagent.jmx$jmx_wrap.doInvoke(jmx.clj:111)
at clojure.lang.RestFn.invoke(RestFn.java:410)
at opsagent.nodedetails$send_all_nodedetails.invoke(nodedetails.clj:241)
at opsagent.opsagent$post_interface_startup.doInvoke(opsagent.clj:125)
at clojure.lang.RestFn.invoke(RestFn.java:421)
at opsagent.conf$handle_new_conf.invoke(conf.clj:179)
at opsagent.messaging$message_callback$fn__6059.invoke(messaging.clj:30)
at 
opsagent.messaging.proxy$java.lang.Object$StompConnection$Listener$7f16bc72.onMessage(Unknown
 Source)
at 
org.jgroups.client.StompConnection.notifyListeners(StompConnection.java:319)
at org.jgroups.client.StompConnection.run(StompConnection.java:269)
at java.lang.Thread.run(Thread.java:745)
{code}

  was:
I've got a Null Pointer Exception in my DataStax OpsCenter Agent log, and it's 
not reporting in to the OpsCenter as a result. Here is the log
{code}
 INFO [StompConnection receiver] 2014-09-17 13:01:15,992 New JMX connection 
(127.0.0.1:7199)
 INFO

[jira] [Updated] (CASSANDRA-7952) DataStax Agent Null Pointer Exception


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hari Sekhon updated CASSANDRA-7952:
---
Priority: Minor  (was: Major)

 DataStax Agent Null Pointer Exception
 -

 Key: CASSANDRA-7952
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7952
 Project: Cassandra
  Issue Type: Bug
 Environment: DSE 4.5.1, DataStax OpsCenter  Agent 5.0.0
Reporter: Hari Sekhon
Priority: Minor

 I've got a Null Pointer Exception in my DataStax OpsCenter Agent log, and 
 it's not reporting in to the OpsCenter. Here is the log
 {code}
  INFO [StompConnection receiver] 2014-09-17 13:01:15,992 New JMX connection 
 (127.0.0.1:7199)
  INFO [Jetty] 2014-09-17 13:01:16,019 Jetty server started
  INFO [Initialization] 2014-09-17 13:01:16,031 Using x.x.x.x as the cassandra 
 broadcast address
  INFO [StompConnection receiver] 2014-09-17 13:01:16,032 Starting up agent 
 collection.
  INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC address is  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC address is 
  x.x.x.x
  INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC broadcast address is 
  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC broadcast 
 address is  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,163 Starting OS metric 
 collectors (Linux)
  INFO [Initialization] 2014-09-17 13:01:16,166 Clearing ssl.truststore
  INFO [Initialization] 2014-09-17 13:01:16,166 Clearing 
 ssl.truststore.password
  INFO [Initialization] 2014-09-17 13:01:16,167 Setting ssl.store.type to JKS
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing 
 kerberos.service.principal.name
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.principal
  INFO [Initialization] 2014-09-17 13:01:16,167 Setting 
 kerberos.useTicketCache to true
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.ticketCache
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.useKeyTab to 
 true
  INFO [Initialization] 2014-09-17 13:01:16,168 Clearing kerberos.keyTab
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.renewTGT to 
 true
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.debug to false
  INFO [StompConnection receiver] 2014-09-17 13:01:16,171 Starting Cassandra 
 JMX metric collectors
  INFO [thrift-init] 2014-09-17 13:01:16,171 Connecting to Cassandra cluster: 
 x.x.x.x (port 9160)
  INFO [StompConnection receiver] 2014-09-17 13:01:16,187 New JMX connection 
 (127.0.0.1:7199)
  INFO [thrift-init] 2014-09-17 13:01:16,189 Downed Host Retry service started 
 with queue size -1 and retry delay 10s
  INFO [thrift-init] 2014-09-17 13:01:16,192 Registering JMX 
 me.prettyprint.cassandra.service_Agent 
 Cluster:ServiceType=hector,MonitorType=hector
  INFO [pdp-loader] 2014-09-17 13:01:16,231 in execute with client 
 org.apache.cassandra.thrift.Cassandra$Client@7a22c094
  INFO [pdp-loader] 2014-09-17 13:01:16,237 Attempting to load stored metric 
 values.
  INFO [thrift-init] 2014-09-17 13:01:16,240 Connected to Cassandra cluster: 
 PoC
  INFO [thrift-init] 2014-09-17 13:01:16,240 in execute with client 
 org.apache.cassandra.thrift.Cassandra$Client@7a22c094
  INFO [thrift-init] 2014-09-17 13:01:16,240 Using partitioner: 
 org.apache.cassandra.dht.Murmur3Partitioner
  INFO [jmx-metrics-1] 2014-09-17 13:01:21,181 New JMX connection 
 (127.0.0.1:7199)
 ERROR [StompConnection receiver] 2014-09-17 13:01:24,376 Failed to collect 
 machine info
 java.lang.NullPointerException
 at clojure.lang.Numbers.ops(Numbers.java:942)
 at clojure.lang.Numbers.divide(Numbers.java:157)
 at 
 opsagent.nodedetails.machine_info$get_machine_info.invoke(machine_info.clj:76)
 at 
 opsagent.nodedetails$get_static_properties$fn__4313.invoke(nodedetails.clj:161)
 at 
 opsagent.nodedetails$get_static_properties.invoke(nodedetails.clj:160)
 at 
 opsagent.nodedetails$get_longtime_values$fn__4426.invoke(nodedetails.clj:227)
 at 
 opsagent.nodedetails$get_longtime_values.invoke(nodedetails.clj:226)
 at 
 opsagent.nodedetails$send_all_nodedetails$fn__.invoke(nodedetails.clj:245)
 at opsagent.jmx$jmx_wrap.doInvoke(jmx.clj:111)
 at clojure.lang.RestFn.invoke(RestFn.java:410)
 at 
 opsagent.nodedetails$send_all_nodedetails.invoke(nodedetails.clj:241)
 at opsagent.opsagent$post_interface_startup.doInvoke(opsagent.clj:125)
 at clojure.lang.RestFn.invoke(RestFn.java:421)
 at opsagent.conf$handle_new_conf.invoke(conf.clj:179)
 at 
 opsagent.messaging$message_callback$fn__6059.invoke(messaging.clj:30)
 at 
 opsagent.messaging.proxy$java.lang.Object$StompConnection$Listener$7f16bc72.onMessage(Unknown

[jira] [Commented] (CASSANDRA-4762) Support IN clause for any clustering column

2014-09-17 Thread Benjamin Lerer (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137152#comment-14137152
 ] 

Benjamin Lerer commented on CASSANDRA-4762:
---

Pushed the fix to 
[branch|https://github.com/blerer/cassandra/commits/CASSANDRA-4762]. 
I have fixed all the points but I am not fully happy with the new name for the 
add method of CompositesBuilder: {quote}addElementToAll(ByteBuffer){quote} and 
{quote}addEachElementToAll(ListByteBuffer){quote}. I am willing to take any 
suggestion. I tried to make the javadoc explicite by providing some examples to 
compensate for those not so clear names. 

 Support IN clause for any clustering column
 ---

 Key: CASSANDRA-4762
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4762
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: T Jake Luciani
Assignee: Benjamin Lerer
  Labels: cql, docs
 Fix For: 3.0

 Attachments: 4762-1.txt


 Given CASSANDRA-3885
 It seems it should be possible to store multiple ranges for many predicates 
 even the inner parts of a composite column.
 They could be expressed as a expanded set of filter queries.
 example:
 {code}
 CREATE TABLE test (
name text,
tdate timestamp,
tdate2 timestamp,
tdate3 timestamp,
num double,
PRIMARY KEY(name,tdate,tdate2,tdate3)
  ) WITH COMPACT STORAGE;
 SELECT * FROM test WHERE 
   name IN ('a','b') and
   tdate IN ('2010-01-01','2011-01-01') and
   tdate2 IN ('2010-01-01','2011-01-01') and
   tdate3 IN ('2010-01-01','2011-01-01') 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-4762) Support IN clause for any clustering column

2014-09-17 Thread Benjamin Lerer (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137152#comment-14137152
 ] 

Benjamin Lerer edited comment on CASSANDRA-4762 at 9/17/14 12:50 PM:
-

Pushed the fix to 
[branch|https://github.com/blerer/cassandra/commits/CASSANDRA-4762]. 
I have fixed all the points but I am not fully happy with the new name for the 
add method of CompositesBuilder: {code}addElementToAll(ByteBuffer){code} and 
{code}addEachElementToAll(ListByteBuffer){code}. I am willing to take any 
suggestion. I tried to make the javadoc explicite by providing some examples to 
compensate for those not so clear names. 


was (Author: blerer):
Pushed the fix to 
[branch|https://github.com/blerer/cassandra/commits/CASSANDRA-4762]. 
I have fixed all the points but I am not fully happy with the new name for the 
add method of CompositesBuilder: {quote}addElementToAll(ByteBuffer){quote} and 
{quote}addEachElementToAll(ListByteBuffer){quote}. I am willing to take any 
suggestion. I tried to make the javadoc explicite by providing some examples to 
compensate for those not so clear names. 

 Support IN clause for any clustering column
 ---

 Key: CASSANDRA-4762
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4762
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: T Jake Luciani
Assignee: Benjamin Lerer
  Labels: cql, docs
 Fix For: 3.0

 Attachments: 4762-1.txt


 Given CASSANDRA-3885
 It seems it should be possible to store multiple ranges for many predicates 
 even the inner parts of a composite column.
 They could be expressed as a expanded set of filter queries.
 example:
 {code}
 CREATE TABLE test (
name text,
tdate timestamp,
tdate2 timestamp,
tdate3 timestamp,
num double,
PRIMARY KEY(name,tdate,tdate2,tdate3)
  ) WITH COMPACT STORAGE;
 SELECT * FROM test WHERE 
   name IN ('a','b') and
   tdate IN ('2010-01-01','2011-01-01') and
   tdate2 IN ('2010-01-01','2011-01-01') and
   tdate3 IN ('2010-01-01','2011-01-01') 
 {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7282) Faster Memtable map


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136908#comment-14136908
 ] 

Benedict edited comment on CASSANDRA-7282 at 9/17/14 12:52 PM:
---

Some more numbers, with a warmup dataset to populate the map so that 
variability due to throughput rate is reduced. These numbers show the NBHOM 
consistently around 3x+ faster, although it introduces per-run variability due 
to GC.

{noformat}
Benchmark(readWriteRatio)  (type)  (warmup)   Mode  
Samples Score  Score error   Units
b.b.c.HashOrderedCollections.test 0.9CSLM   500  thrpt  
  5  1392.273 2918.717  ops/ms
b.b.c.HashOrderedCollections.test 0.9   NBHOM   500  thrpt  
  5  5088.408 8964.885  ops/ms
b.b.c.HashOrderedCollections.test 0.5CSLM   500  thrpt  
  5  1128.637 2589.679  ops/ms
b.b.c.HashOrderedCollections.test 0.5   NBHOM   500  thrpt  
  5  3406.299 5606.085  ops/ms
b.b.c.HashOrderedCollections.test 0.1CSLM   500  thrpt  
  5   924.642 1802.045  ops/ms
b.b.c.HashOrderedCollections.test 0.1   NBHOM   500  thrpt  
  5  3311.107  999.896  ops/ms
b.b.c.HashOrderedCollections.test   0CSLM   500  thrpt  
  5   939.757 1776.641  ops/ms
b.b.c.HashOrderedCollections.test   0   NBHOM   500  thrpt  
  5  2781.503 4723.844  ops/ms
{noformat}

edit: same principle but fewer items warmed up, so less variability due to GC:

{noformat}
Benchmark(readWriteRatio)  (type)  (warmup)   Mode  
Samples Score  Score error   Units
b.b.c.HashOrderedCollections.test 0.9CSLM   100  thrpt  
 10  2283.934  157.719  ops/ms
b.b.c.HashOrderedCollections.test 0.9   NBHOM   100  thrpt  
 10  8850.066  147.894  ops/ms
b.b.c.HashOrderedCollections.test 0.5CSLM   100  thrpt  
 10  1960.077  145.752  ops/ms
b.b.c.HashOrderedCollections.test 0.5   NBHOM   100  thrpt  
 10  5637.813  688.394  ops/ms
b.b.c.HashOrderedCollections.test 0.1CSLM   100  thrpt  
 10   706.284  162.845  ops/ms
b.b.c.HashOrderedCollections.test 0.1   NBHOM   100  thrpt  
 10  3270.920 1545.698  ops/ms
b.b.c.HashOrderedCollections.test   0CSLM   100  thrpt  
 10  1689.157  176.412  ops/ms
b.b.c.HashOrderedCollections.test   0   NBHOM   100  thrpt  
 10  2737.195 1042.289  ops/ms
{noformat}



was (Author: benedict):
Some more numbers, with a warmup dataset to populate the map so that 
variability due to throughput rate is reduced. These numbers show the NBHOM 
consistently around 3x+ faster, although it introduces per-run variability due 
to GC.

{noformat}
Benchmark(readWriteRatio)  (type)  (warmup)   Mode  
Samples Score  Score error   Units
b.b.c.HashOrderedCollections.test 0.9CSLM   500  thrpt  
  5  1392.273 2918.717  ops/ms
b.b.c.HashOrderedCollections.test 0.9   NBHOM   500  thrpt  
  5  5088.408 8964.885  ops/ms
b.b.c.HashOrderedCollections.test 0.5CSLM   500  thrpt  
  5  1128.637 2589.679  ops/ms
b.b.c.HashOrderedCollections.test 0.5   NBHOM   500  thrpt  
  5  3406.299 5606.085  ops/ms
b.b.c.HashOrderedCollections.test 0.1CSLM   500  thrpt  
  5   924.642 1802.045  ops/ms
b.b.c.HashOrderedCollections.test 0.1   NBHOM   500  thrpt  
  5  3311.107  999.896  ops/ms
b.b.c.HashOrderedCollections.test   0CSLM   500  thrpt  
  5   939.757 1776.641  ops/ms
b.b.c.HashOrderedCollections.test   0   NBHOM   500  thrpt  
  5  2781.503 4723.844  ops/ms
{noformat}

 Faster Memtable map
 ---

 Key: CASSANDRA-7282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 3.0

 Attachments: profile.yaml, reads.svg, run1.svg, writes.svg


 Currently we maintain a ConcurrentSkipLastMap of DecoratedKey - Partition in 
 our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
 majority of users use a hash partitioner, it occurs to me we could maintain a 
 hybrid ordered list / hash map. The list would impose the normal order on the 
 collection, but a hash index would live alongside

[jira] [Resolved] (CASSANDRA-7952) DataStax Agent Null Pointer Exception


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-7952.
-
Resolution: Invalid

This jira is for opensource cassandra issues, you should report this to 
Datastax/opscenter.

 DataStax Agent Null Pointer Exception
 -

 Key: CASSANDRA-7952
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7952
 Project: Cassandra
  Issue Type: Bug
 Environment: DSE 4.5.1, DataStax OpsCenter  Agent 5.0.0
Reporter: Hari Sekhon
Priority: Minor

 I've got a Null Pointer Exception in my DataStax OpsCenter Agent log, and 
 it's not reporting in to the OpsCenter. Here is the log
 {code}
  INFO [StompConnection receiver] 2014-09-17 13:01:15,992 New JMX connection 
 (127.0.0.1:7199)
  INFO [Jetty] 2014-09-17 13:01:16,019 Jetty server started
  INFO [Initialization] 2014-09-17 13:01:16,031 Using x.x.x.x as the cassandra 
 broadcast address
  INFO [StompConnection receiver] 2014-09-17 13:01:16,032 Starting up agent 
 collection.
  INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC address is  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC address is 
  x.x.x.x
  INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC broadcast address is 
  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC broadcast 
 address is  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,163 Starting OS metric 
 collectors (Linux)
  INFO [Initialization] 2014-09-17 13:01:16,166 Clearing ssl.truststore
  INFO [Initialization] 2014-09-17 13:01:16,166 Clearing 
 ssl.truststore.password
  INFO [Initialization] 2014-09-17 13:01:16,167 Setting ssl.store.type to JKS
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing 
 kerberos.service.principal.name
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.principal
  INFO [Initialization] 2014-09-17 13:01:16,167 Setting 
 kerberos.useTicketCache to true
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.ticketCache
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.useKeyTab to 
 true
  INFO [Initialization] 2014-09-17 13:01:16,168 Clearing kerberos.keyTab
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.renewTGT to 
 true
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.debug to false
  INFO [StompConnection receiver] 2014-09-17 13:01:16,171 Starting Cassandra 
 JMX metric collectors
  INFO [thrift-init] 2014-09-17 13:01:16,171 Connecting to Cassandra cluster: 
 x.x.x.x (port 9160)
  INFO [StompConnection receiver] 2014-09-17 13:01:16,187 New JMX connection 
 (127.0.0.1:7199)
  INFO [thrift-init] 2014-09-17 13:01:16,189 Downed Host Retry service started 
 with queue size -1 and retry delay 10s
  INFO [thrift-init] 2014-09-17 13:01:16,192 Registering JMX 
 me.prettyprint.cassandra.service_Agent 
 Cluster:ServiceType=hector,MonitorType=hector
  INFO [pdp-loader] 2014-09-17 13:01:16,231 in execute with client 
 org.apache.cassandra.thrift.Cassandra$Client@7a22c094
  INFO [pdp-loader] 2014-09-17 13:01:16,237 Attempting to load stored metric 
 values.
  INFO [thrift-init] 2014-09-17 13:01:16,240 Connected to Cassandra cluster: 
 PoC
  INFO [thrift-init] 2014-09-17 13:01:16,240 in execute with client 
 org.apache.cassandra.thrift.Cassandra$Client@7a22c094
  INFO [thrift-init] 2014-09-17 13:01:16,240 Using partitioner: 
 org.apache.cassandra.dht.Murmur3Partitioner
  INFO [jmx-metrics-1] 2014-09-17 13:01:21,181 New JMX connection 
 (127.0.0.1:7199)
 ERROR [StompConnection receiver] 2014-09-17 13:01:24,376 Failed to collect 
 machine info
 java.lang.NullPointerException
 at clojure.lang.Numbers.ops(Numbers.java:942)
 at clojure.lang.Numbers.divide(Numbers.java:157)
 at 
 opsagent.nodedetails.machine_info$get_machine_info.invoke(machine_info.clj:76)
 at 
 opsagent.nodedetails$get_static_properties$fn__4313.invoke(nodedetails.clj:161)
 at 
 opsagent.nodedetails$get_static_properties.invoke(nodedetails.clj:160)
 at 
 opsagent.nodedetails$get_longtime_values$fn__4426.invoke(nodedetails.clj:227)
 at 
 opsagent.nodedetails$get_longtime_values.invoke(nodedetails.clj:226)
 at 
 opsagent.nodedetails$send_all_nodedetails$fn__.invoke(nodedetails.clj:245)
 at opsagent.jmx$jmx_wrap.doInvoke(jmx.clj:111)
 at clojure.lang.RestFn.invoke(RestFn.java:410)
 at 
 opsagent.nodedetails$send_all_nodedetails.invoke(nodedetails.clj:241)
 at opsagent.opsagent$post_interface_startup.doInvoke(opsagent.clj:125)
 at clojure.lang.RestFn.invoke(RestFn.java:421)
 at opsagent.conf$handle_new_conf.invoke(conf.clj:179)
 at 
 opsagent.messaging$message_callback$fn__6059.invoke(messaging.clj:30)
 at

[jira] [Comment Edited] (CASSANDRA-7282) Faster Memtable map


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14136908#comment-14136908
 ] 

Benedict edited comment on CASSANDRA-7282 at 9/17/14 1:22 PM:
--

Some more numbers, with a warmup dataset to populate the map so that 
variability due to throughput rate is reduced. These numbers show the NBHOM 
consistently around 3x+ faster.

{noformat}
Benchmark(readWriteRatio)  (type)  (warmup)   Mode  
Samples Score  Score error   Units
b.b.c.HashOrderedCollections.test 0.9CSLM   100  thrpt  
 10  1775.801   57.310  ops/ms
b.b.c.HashOrderedCollections.test 0.9   NBHOM   100  thrpt  
 10  5186.264 1183.869  ops/ms
b.b.c.HashOrderedCollections.test 0.5CSLM   100  thrpt  
 10  1600.233   65.174  ops/ms
b.b.c.HashOrderedCollections.test 0.5   NBHOM   100  thrpt  
 10  5764.222   93.983  ops/ms
b.b.c.HashOrderedCollections.test 0.1CSLM   100  thrpt  
 10   760.552   72.869  ops/ms
b.b.c.HashOrderedCollections.test 0.1   NBHOM   100  thrpt  
 10  3771.195  188.487  ops/ms
b.b.c.HashOrderedCollections.test   0CSLM   100  thrpt  
 10   704.723   74.047  ops/ms
b.b.c.HashOrderedCollections.test   0   NBHOM   100  thrpt  
 10  3561.286  203.780  ops/ms
{noformat}

(edit: updated to ensure reads hit keys that are present - this seems to have 
negative consequences for CSLM, as this is more realistic)


was (Author: benedict):
Some more numbers, with a warmup dataset to populate the map so that 
variability due to throughput rate is reduced. These numbers show the NBHOM 
consistently around 3x+ faster, although it introduces per-run variability due 
to GC.

{noformat}
Benchmark(readWriteRatio)  (type)  (warmup)   Mode  
Samples Score  Score error   Units
b.b.c.HashOrderedCollections.test 0.9CSLM   500  thrpt  
  5  1392.273 2918.717  ops/ms
b.b.c.HashOrderedCollections.test 0.9   NBHOM   500  thrpt  
  5  5088.408 8964.885  ops/ms
b.b.c.HashOrderedCollections.test 0.5CSLM   500  thrpt  
  5  1128.637 2589.679  ops/ms
b.b.c.HashOrderedCollections.test 0.5   NBHOM   500  thrpt  
  5  3406.299 5606.085  ops/ms
b.b.c.HashOrderedCollections.test 0.1CSLM   500  thrpt  
  5   924.642 1802.045  ops/ms
b.b.c.HashOrderedCollections.test 0.1   NBHOM   500  thrpt  
  5  3311.107  999.896  ops/ms
b.b.c.HashOrderedCollections.test   0CSLM   500  thrpt  
  5   939.757 1776.641  ops/ms
b.b.c.HashOrderedCollections.test   0   NBHOM   500  thrpt  
  5  2781.503 4723.844  ops/ms
{noformat}

edit: same principle but fewer items warmed up, so less variability due to GC:

{noformat}
Benchmark(readWriteRatio)  (type)  (warmup)   Mode  
Samples Score  Score error   Units
b.b.c.HashOrderedCollections.test 0.9CSLM   100  thrpt  
 10  2283.934  157.719  ops/ms
b.b.c.HashOrderedCollections.test 0.9   NBHOM   100  thrpt  
 10  8850.066  147.894  ops/ms
b.b.c.HashOrderedCollections.test 0.5CSLM   100  thrpt  
 10  1960.077  145.752  ops/ms
b.b.c.HashOrderedCollections.test 0.5   NBHOM   100  thrpt  
 10  5637.813  688.394  ops/ms
b.b.c.HashOrderedCollections.test 0.1CSLM   100  thrpt  
 10   706.284  162.845  ops/ms
b.b.c.HashOrderedCollections.test 0.1   NBHOM   100  thrpt  
 10  3270.920 1545.698  ops/ms
b.b.c.HashOrderedCollections.test   0CSLM   100  thrpt  
 10  1689.157  176.412  ops/ms
b.b.c.HashOrderedCollections.test   0   NBHOM   100  thrpt  
 10  2737.195 1042.289  ops/ms
{noformat}


 Faster Memtable map
 ---

 Key: CASSANDRA-7282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 3.0

 Attachments: profile.yaml, reads.svg, run1.svg, writes.svg


 Currently we maintain a ConcurrentSkipLastMap of DecoratedKey - Partition in 
 our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
 majority of users use a hash partitioner, it occurs to me we could maintain a 
 hybrid ordered list / hash map. The list would

[jira] [Created] (CASSANDRA-7953) RangeTombstones not merging during compaction

2014-09-17 Thread Marcus Olsson (JIRA)

Marcus Olsson created CASSANDRA-7953:


 Summary: RangeTombstones not merging during compaction
 Key: CASSANDRA-7953
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7953
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1
Reporter: Marcus Olsson
Priority: Minor


When performing a compaction on two sstables that contain the same 
RangeTombstone with different timestamps, the tombstones are not merged in the 
new sstable.

This has been tested using cassandra 2.1 with the following table:
{code}
CREATE TABLE test(
  key text,
  column text,
  data text,
  PRIMARY KEY(key, column)
);
{code}

And then doing the following:
{code}
INSERT INTO test (key, column, data) VALUES (1, 1, 1); // If the sstable 
only contains tombstones during compaction it seems that the sstable either 
gets removed or isn't created (but that could probably be a separate JIRA 
issue).
INSERT INTO test (key, column, data) VALUES (1, 2, 2); // The inserts are 
not actually needed, since the deletes create tombstones either way.
DELETE FROM test WHERE key=1 AND column=2;

nodetool flush

INSERT INTO test (key, column, data) VALUES (1, 2, 2);
DELETE FROM test WHERE key=1 AND column=2;

nodetool flush
nodetool compact
{code}

When checking with the SSTableExport tool two tombstones exists in the 
compacted sstable. This can be repeated, resulting in more and more tombstones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7843) Remove static singleton instances


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137209#comment-14137209
 ] 

Benedict commented on CASSANDRA-7843:
-

I really don't have anything against static singletons. They make testing a bit 
harder, but there's not really anything wrong with them for development, so I 
don't really consider them technical debt as such. I am probably in the 
minority about that, of course, given the amount of disgust I've seen expressed 
about their presence.

Could we discuss some specific testing use cases we want to enable, that cannot 
be achieved through other means? It would be nice to get a handle on what the 
benefit is going to be.

bq.  I've also done the refactor in a way that can be reviewed and merged in 
smaller chunks.

There are  400 _commits_. That will be much more challenging than you let on 
to even navigate, let alone review. It is likely to be both boring _and_ 
stressful.

 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7952) DataStax Agent Null Pointer Exception


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137210#comment-14137210
 ] 

Hari Sekhon commented on CASSANDRA-7952:


Yes I know but I don't think they have a public jira for this, I was hoping 
someone from DataStax would pick this up and copy to their internal jira, 
didn't seem suitable to post on serverfault...

 DataStax Agent Null Pointer Exception
 -

 Key: CASSANDRA-7952
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7952
 Project: Cassandra
  Issue Type: Bug
 Environment: DSE 4.5.1, DataStax OpsCenter  Agent 5.0.0
Reporter: Hari Sekhon
Priority: Minor

 I've got a Null Pointer Exception in my DataStax OpsCenter Agent log, and 
 it's not reporting in to the OpsCenter. Here is the log
 {code}
  INFO [StompConnection receiver] 2014-09-17 13:01:15,992 New JMX connection 
 (127.0.0.1:7199)
  INFO [Jetty] 2014-09-17 13:01:16,019 Jetty server started
  INFO [Initialization] 2014-09-17 13:01:16,031 Using x.x.x.x as the cassandra 
 broadcast address
  INFO [StompConnection receiver] 2014-09-17 13:01:16,032 Starting up agent 
 collection.
  INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC address is  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC address is 
  x.x.x.x
  INFO [Initialization] 2014-09-17 13:01:16,162 agent RPC broadcast address is 
  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,162 agent RPC broadcast 
 address is  x.x.x.x
  INFO [StompConnection receiver] 2014-09-17 13:01:16,163 Starting OS metric 
 collectors (Linux)
  INFO [Initialization] 2014-09-17 13:01:16,166 Clearing ssl.truststore
  INFO [Initialization] 2014-09-17 13:01:16,166 Clearing 
 ssl.truststore.password
  INFO [Initialization] 2014-09-17 13:01:16,167 Setting ssl.store.type to JKS
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing 
 kerberos.service.principal.name
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.principal
  INFO [Initialization] 2014-09-17 13:01:16,167 Setting 
 kerberos.useTicketCache to true
  INFO [Initialization] 2014-09-17 13:01:16,167 Clearing kerberos.ticketCache
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.useKeyTab to 
 true
  INFO [Initialization] 2014-09-17 13:01:16,168 Clearing kerberos.keyTab
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.renewTGT to 
 true
  INFO [Initialization] 2014-09-17 13:01:16,168 Setting kerberos.debug to false
  INFO [StompConnection receiver] 2014-09-17 13:01:16,171 Starting Cassandra 
 JMX metric collectors
  INFO [thrift-init] 2014-09-17 13:01:16,171 Connecting to Cassandra cluster: 
 x.x.x.x (port 9160)
  INFO [StompConnection receiver] 2014-09-17 13:01:16,187 New JMX connection 
 (127.0.0.1:7199)
  INFO [thrift-init] 2014-09-17 13:01:16,189 Downed Host Retry service started 
 with queue size -1 and retry delay 10s
  INFO [thrift-init] 2014-09-17 13:01:16,192 Registering JMX 
 me.prettyprint.cassandra.service_Agent 
 Cluster:ServiceType=hector,MonitorType=hector
  INFO [pdp-loader] 2014-09-17 13:01:16,231 in execute with client 
 org.apache.cassandra.thrift.Cassandra$Client@7a22c094
  INFO [pdp-loader] 2014-09-17 13:01:16,237 Attempting to load stored metric 
 values.
  INFO [thrift-init] 2014-09-17 13:01:16,240 Connected to Cassandra cluster: 
 PoC
  INFO [thrift-init] 2014-09-17 13:01:16,240 in execute with client 
 org.apache.cassandra.thrift.Cassandra$Client@7a22c094
  INFO [thrift-init] 2014-09-17 13:01:16,240 Using partitioner: 
 org.apache.cassandra.dht.Murmur3Partitioner
  INFO [jmx-metrics-1] 2014-09-17 13:01:21,181 New JMX connection 
 (127.0.0.1:7199)
 ERROR [StompConnection receiver] 2014-09-17 13:01:24,376 Failed to collect 
 machine info
 java.lang.NullPointerException
 at clojure.lang.Numbers.ops(Numbers.java:942)
 at clojure.lang.Numbers.divide(Numbers.java:157)
 at 
 opsagent.nodedetails.machine_info$get_machine_info.invoke(machine_info.clj:76)
 at 
 opsagent.nodedetails$get_static_properties$fn__4313.invoke(nodedetails.clj:161)
 at 
 opsagent.nodedetails$get_static_properties.invoke(nodedetails.clj:160)
 at 
 opsagent.nodedetails$get_longtime_values$fn__4426.invoke(nodedetails.clj:227)
 at 
 opsagent.nodedetails$get_longtime_values.invoke(nodedetails.clj:226)
 at 
 opsagent.nodedetails$send_all_nodedetails$fn__.invoke(nodedetails.clj:245)
 at opsagent.jmx$jmx_wrap.doInvoke(jmx.clj:111)
 at clojure.lang.RestFn.invoke(RestFn.java:410)
 at 
 opsagent.nodedetails$send_all_nodedetails.invoke(nodedetails.clj:241)
 at opsagent.opsagent$post_interface_startup.doInvoke(opsagent.clj:125)
 at clojure.lang.RestFn.invoke(RestFn.java:421)
 at

[jira] [Commented] (CASSANDRA-7759) DROP TABLE makes C* unreachable


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137215#comment-14137215
 ] 

Benedict commented on CASSANDRA-7759:
-

[~flarcher]: any news on this? would like to rule out anything sinister

 DROP TABLE makes C* unreachable
 ---

 Key: CASSANDRA-7759
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7759
 Project: Cassandra
  Issue Type: Bug
 Environment: CQLSH of C* 2.10.-rc5 with one or two node(s).
Reporter: Fabrice Larcher

 After a {{DROP KEYSPACE}} or {{DROP TABLE}} command from _CQLSH_ I often get 
 the output :
 {code}
 errors={}, last_host=127.0.0.1
 {code}
 Then, the application can not access the node anymore (it uses the Java 
 Driver 2.1.0-rc1). And I get the following stacktraces from the 
 {{system.log}} file ;
 {code}
 ERROR [MemtableFlushWriter:5] 2014-08-13 11:26:07,577 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[MemtableFlushWriter:5,5,main]
 java.lang.RuntimeException: Last written key 
 DecoratedKey(28149890686391545-8456361251720325, 
 30313034353666392d666464362d343539362d393537372d653539336430333138396437) = 
 current key DecoratedKey(108b5f3f-fc06-4a0d-99f1-3c6484b32e04, 
 31303862356633662d666330362d346130642d393966312d336336343834623332653034) 
 writing into 
 ./../data/data/metrics/run-a3c1fe80216911e49fd1ab9c3338a2ff/metrics-run.run_probe-tmp-ka-2-Data.db
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
  ~[guava-16.0.jar:na]
   at 
 org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1054)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  ~[na:1.7.0_45]
   at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  ~[na:1.7.0_45]
   at java.lang.Thread.run(Thread.java:744) ~[na:1.7.0_45]
 {code}
 I also get the following (perhaps not related) 
 {code}
 ERROR [MemtableFlushWriter:6] 2014-08-13 11:26:08,827 
 CassandraDaemon.java:166 - Exception in thread 
 Thread[MemtableFlushWriter:6,5,main]
 org.apache.cassandra.serializers.MarshalException: Invalid byte for ascii: -50
   at 
 org.apache.cassandra.serializers.AsciiSerializer.validate(AsciiSerializer.java:39)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:78) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at org.apache.cassandra.dht.LocalToken.toString(LocalToken.java:39) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
   at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
   at org.apache.cassandra.db.DecoratedKey.toString(DecoratedKey.java:118) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at java.lang.String.valueOf(String.java:2854) ~[na:1.7.0_45]
   at java.lang.StringBuilder.append(StringBuilder.java:128) ~[na:1.7.0_45]
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:172)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:215) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:351)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:314) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
  ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) 
 ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
   at 
 com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)

[jira] [Assigned] (CASSANDRA-7953) RangeTombstones not merging during compaction

2014-09-17 Thread Marcus Eriksson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson reassigned CASSANDRA-7953:
--

Assignee: Marcus Eriksson

 RangeTombstones not merging during compaction
 -

 Key: CASSANDRA-7953
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7953
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1
Reporter: Marcus Olsson
Assignee: Marcus Eriksson
Priority: Minor
  Labels: compaction, deletes, tombstone

 When performing a compaction on two sstables that contain the same 
 RangeTombstone with different timestamps, the tombstones are not merged in 
 the new sstable.
 This has been tested using cassandra 2.1 with the following table:
 {code}
 CREATE TABLE test(
   key text,
   column text,
   data text,
   PRIMARY KEY(key, column)
 );
 {code}
 And then doing the following:
 {code}
 INSERT INTO test (key, column, data) VALUES (1, 1, 1); // If the 
 sstable only contains tombstones during compaction it seems that the sstable 
 either gets removed or isn't created (but that could probably be a separate 
 JIRA issue).
 INSERT INTO test (key, column, data) VALUES (1, 2, 2); // The inserts 
 are not actually needed, since the deletes create tombstones either way.
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 INSERT INTO test (key, column, data) VALUES (1, 2, 2);
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 nodetool compact
 {code}
 When checking with the SSTableExport tool two tombstones exists in the 
 compacted sstable. This can be repeated, resulting in more and more 
 tombstones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7954) Skip paxos for CAS if RF=1

Benedict created CASSANDRA-7954:
---

 Summary: Skip paxos for CAS if RF=1
 Key: CASSANDRA-7954
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7954
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Priority: Minor


If RF=1, there's no point doing a paxos round - we get just as much safety and 
atomicity by simply conditionally applying the update locally



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7953) RangeTombstones not merging during compaction

2014-09-17 Thread Marcus Olsson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Olsson updated CASSANDRA-7953:
-
Attachment: CASSANDRA-7953.patch

 RangeTombstones not merging during compaction
 -

 Key: CASSANDRA-7953
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7953
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1
Reporter: Marcus Olsson
Assignee: Marcus Eriksson
Priority: Minor
  Labels: compaction, deletes, tombstone
 Fix For: 2.1.0

 Attachments: CASSANDRA-7953.patch


 When performing a compaction on two sstables that contain the same 
 RangeTombstone with different timestamps, the tombstones are not merged in 
 the new sstable.
 This has been tested using cassandra 2.1 with the following table:
 {code}
 CREATE TABLE test(
   key text,
   column text,
   data text,
   PRIMARY KEY(key, column)
 );
 {code}
 And then doing the following:
 {code}
 INSERT INTO test (key, column, data) VALUES (1, 1, 1); // If the 
 sstable only contains tombstones during compaction it seems that the sstable 
 either gets removed or isn't created (but that could probably be a separate 
 JIRA issue).
 INSERT INTO test (key, column, data) VALUES (1, 2, 2); // The inserts 
 are not actually needed, since the deletes create tombstones either way.
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 INSERT INTO test (key, column, data) VALUES (1, 2, 2);
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 nodetool compact
 {code}
 When checking with the SSTableExport tool two tombstones exists in the 
 compacted sstable. This can be repeated, resulting in more and more 
 tombstones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7953) RangeTombstones not merging during compaction

2014-09-17 Thread Marcus Eriksson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-7953:
---
Reviewer: Marcus Eriksson
Assignee: (was: Marcus Eriksson)

 RangeTombstones not merging during compaction
 -

 Key: CASSANDRA-7953
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7953
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1
Reporter: Marcus Olsson
Priority: Minor
  Labels: compaction, deletes, tombstone
 Fix For: 2.1.0

 Attachments: CASSANDRA-7953.patch


 When performing a compaction on two sstables that contain the same 
 RangeTombstone with different timestamps, the tombstones are not merged in 
 the new sstable.
 This has been tested using cassandra 2.1 with the following table:
 {code}
 CREATE TABLE test(
   key text,
   column text,
   data text,
   PRIMARY KEY(key, column)
 );
 {code}
 And then doing the following:
 {code}
 INSERT INTO test (key, column, data) VALUES (1, 1, 1); // If the 
 sstable only contains tombstones during compaction it seems that the sstable 
 either gets removed or isn't created (but that could probably be a separate 
 JIRA issue).
 INSERT INTO test (key, column, data) VALUES (1, 2, 2); // The inserts 
 are not actually needed, since the deletes create tombstones either way.
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 INSERT INTO test (key, column, data) VALUES (1, 2, 2);
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 nodetool compact
 {code}
 When checking with the SSTableExport tool two tombstones exists in the 
 compacted sstable. This can be repeated, resulting in more and more 
 tombstones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7843) Remove static singleton instances

2014-09-17 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137249#comment-14137249
 ] 

Yuki Morishita commented on CASSANDRA-7843:
---

One problem we are getting occasionally is that C* internally has 
(unneccesarily) coupled each other that it is hard to predict what's needed to 
reuse some useful staff outside of C*, for example tools like sstableloader.

The change is huge. To mitigate merging from older versions, we can split it 
up. Some part will change extremely like o.a.c.db, but others like o.a.c.auth, 
o.a.c.config, o.a.c.streaming won't change much. We can start from those 
modules that are less likely to change and have less dependencies.


 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7953) RangeTombstones not merging during compaction

2014-09-17 Thread Marcus Olsson (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Olsson updated CASSANDRA-7953:
-
Fix Version/s: (was: 2.1.0)

 RangeTombstones not merging during compaction
 -

 Key: CASSANDRA-7953
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7953
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: Cassandra 2.1
Reporter: Marcus Olsson
Priority: Minor
  Labels: compaction, deletes, tombstone
 Attachments: CASSANDRA-7953.patch


 When performing a compaction on two sstables that contain the same 
 RangeTombstone with different timestamps, the tombstones are not merged in 
 the new sstable.
 This has been tested using cassandra 2.1 with the following table:
 {code}
 CREATE TABLE test(
   key text,
   column text,
   data text,
   PRIMARY KEY(key, column)
 );
 {code}
 And then doing the following:
 {code}
 INSERT INTO test (key, column, data) VALUES (1, 1, 1); // If the 
 sstable only contains tombstones during compaction it seems that the sstable 
 either gets removed or isn't created (but that could probably be a separate 
 JIRA issue).
 INSERT INTO test (key, column, data) VALUES (1, 2, 2); // The inserts 
 are not actually needed, since the deletes create tombstones either way.
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 INSERT INTO test (key, column, data) VALUES (1, 2, 2);
 DELETE FROM test WHERE key=1 AND column=2;
 nodetool flush
 nodetool compact
 {code}
 When checking with the SSTableExport tool two tombstones exists in the 
 compacted sstable. This can be repeated, resulting in more and more 
 tombstones.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7955) Range movements can violate consistency if RING_DELAY = write_request_timeout_in_ms

Benedict created CASSANDRA-7955:
---

 Summary: Range movements can violate consistency if RING_DELAY = 
write_request_timeout_in_ms
 Key: CASSANDRA-7955
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7955
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Priority: Minor


Cassandra should probably refuse to start if RING_DELAY is too close to (or 
below) write_request_timeout_ms

We should probably also consider throwing a WriteTimeoutException _even if we 
don't get interrupted by the timeout, since there are reasons due to scheduling 
or system overload this could not happen (though it is unlikely to be 
significant enough to have an impact, it's better safe than sorry)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7956) nodetool compactionhistory crashes because of low heap size (GC overhead limit exceeded)

Nikolai Grigoriev created CASSANDRA-7956:


 Summary: nodetool compactionhistory crashes because of low heap 
size (GC overhead limit exceeded)
 Key: CASSANDRA-7956
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7956
 Project: Cassandra
  Issue Type: Bug
 Environment: Cassandra 2.0.8
Reporter: Nikolai Grigoriev
Priority: Trivial
 Attachments: nodetool_compactionhistory_128m_heap_output.txt.gz

{code}
]# nodetool compactionhistory
Compaction History:
Exception in thread main java.lang.OutOfMemoryError: GC overhead limit 
exceeded
at java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:967)
at 
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1782)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at java.util.HashMap.readObject(HashMap.java:1180)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
at 
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at 
java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
at 
java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:500)
at 
javax.management.openmbean.TabularDataSupport.readObject(TabularDataSupport.java:912)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
at 
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:325)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:174)
at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
at 
javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
at 
javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:906)
at 
javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:267)
at com.sun.proxy.$Proxy3.getCompactionHistory(Unknown Source)
{code}

nodetool starts with -Xmx32m. This seems to be not enough at least in my case 
to show the history. I am not sure what would the appropriate amount be but 
increasing it to 128m definitely solves the problem. Output from modified 
nodetool attached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7955) Range movements can violate consistency if RING_DELAY = write_request_timeout_in_ms


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7955:

Description: 
Cassandra should probably refuse to start if RING_DELAY is too close to (or 
below) write_request_timeout_ms, because we depend on this for 
correctness/consistency during range movements

We should probably also consider throwing a WriteTimeoutException _even if we 
don't get interrupted by the timeout, since there are reasons due to scheduling 
or system overload this could not happen (though it is unlikely to be 
significant enough to have an impact, it's better safe than sorry)

  was:
Cassandra should probably refuse to start if RING_DELAY is too close to (or 
below) write_request_timeout_ms

We should probably also consider throwing a WriteTimeoutException _even if we 
don't get interrupted by the timeout, since there are reasons due to scheduling 
or system overload this could not happen (though it is unlikely to be 
significant enough to have an impact, it's better safe than sorry)


 Range movements can violate consistency if RING_DELAY = 
 write_request_timeout_in_ms
 

 Key: CASSANDRA-7955
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7955
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Priority: Minor

 Cassandra should probably refuse to start if RING_DELAY is too close to (or 
 below) write_request_timeout_ms, because we depend on this for 
 correctness/consistency during range movements
 We should probably also consider throwing a WriteTimeoutException _even if we 
 don't get interrupted by the timeout, since there are reasons due to 
 scheduling or system overload this could not happen (though it is unlikely to 
 be significant enough to have an impact, it's better safe than sorry)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7843) Remove static singleton instances


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137304#comment-14137304
 ] 

Benedict commented on CASSANDRA-7843:
-

bq. One problem we are getting occasionally

A small problem we encounter occasionally doesn't seem to warrant this 
upheaval

 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (CASSANDRA-7650) Expose auto_bootstrap as a system property override

2014-09-17 Thread Joe Hohertz (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Hohertz reopened CASSANDRA-7650:

Reproduced In: 2.0.9, 1.2.18  (was: 1.2.18, 2.0.9)

Thank you for applying to the 2.x+ versions.

Would it be possible to get this applied to 1.2, in time for the 1.2.19 build 
as well?

Thanks.

 Expose auto_bootstrap as a system property override
 ---

 Key: CASSANDRA-7650
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7650
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Joe Hohertz
Assignee: Joe Hohertz
Priority: Trivial
  Labels: auto_bootstrap, config, lhf
 Fix For: 2.0.10, 2.1.0

 Attachments: cassandra_auto_bootstrap.patch


 This one-line change gives the auto_bootstrap config parameter the ability to 
 be overridden by a system property.
 As I understand things, setting this false is generally a one-time initial 
 set-up item, and providing this will be potentially easier than swapping out 
 yaml file items when the need to set this false arises.
 The handling is the same as was already implemented for replace_address and 
 the like, look to the system property, and default that to the yaml config 
 value.
 This also will also allow a start-up agent (like Priam) to influence the 
 setting from within it's control over Cassandra's start-up behaviour
 Patch was generated against 1.2 branch but applies ok to 2.0 as well. (It 
 would be great if this could be applied to both code lines)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[2/3] git commit: Make repair -pr work with -local

Make repair -pr work with -local

patch by Paulo Motta; reviewed by yukim for CASSANDRA-7450


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/125b4642
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/125b4642
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/125b4642

Branch: refs/heads/trunk
Commit: 125b4642db63a37c560dec4615ea04662c6e45ef
Parents: 90a2114
Author: Paulo Motta pauloricard...@gmail.com
Authored: Wed Sep 17 09:46:52 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed Sep 17 09:46:52 2014 -0500

--
 .../cassandra/service/StorageService.java   |  92 +---
 .../org/apache/cassandra/tools/NodeTool.java|   2 +-
 .../service/StorageServiceServerTest.java   | 210 ++-
 3 files changed, 267 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/125b4642/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 1aa3b24..b1fe20e 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -43,7 +43,6 @@ import ch.qos.logback.classic.jmx.JMXConfiguratorMBean;
 import ch.qos.logback.classic.spi.ILoggingEvent;
 import ch.qos.logback.core.Appender;
 
-import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
 import com.google.common.collect.*;
 import com.google.common.util.concurrent.FutureCallback;
@@ -156,11 +155,16 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return getRangesForEndpoint(keyspaceName, 
FBUtilities.getBroadcastAddress());
 }
 
-public CollectionRangeToken getLocalPrimaryRanges(String keyspace)
+public CollectionRangeToken getPrimaryRanges(String keyspace)
 {
 return getPrimaryRangesForEndpoint(keyspace, 
FBUtilities.getBroadcastAddress());
 }
 
+public CollectionRangeToken getPrimaryRangesWithinDC(String keyspace)
+{
+return getPrimaryRangeForEndpointWithinDC(keyspace, 
FBUtilities.getBroadcastAddress());
+}
+
 private final SetInetAddress replicatingNodes = 
Collections.synchronizedSet(new HashSetInetAddress());
 private CassandraDaemon daemon;
 
@@ -332,7 +336,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 {
 throw new IllegalStateException(No configured daemon);
 }
-
+
 try
 {
 daemon.nativeServer.start();
@@ -430,10 +434,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if 
(Gossiper.instance.getEndpointStateForEndpoint(DatabaseDescriptor.getReplaceAddress()).getApplicationState(ApplicationState.TOKENS)
 == null)
 throw new RuntimeException(Could not find tokens for  + 
DatabaseDescriptor.getReplaceAddress() +  to replace);
 CollectionToken tokens = 
TokenSerializer.deserialize(getPartitioner(), new DataInputStream(new 
ByteArrayInputStream(getApplicationStateValue(DatabaseDescriptor.getReplaceAddress(),
 ApplicationState.TOKENS;
-
+
 SystemKeyspace.setLocalHostId(hostId); // use the replacee's host 
Id as our own so we receive hints, etc
 Gossiper.instance.resetEndpointStateMap(); // clean up since we 
have what we need
-return tokens;
+return tokens;
 }
 catch (IOException e)
 {
@@ -2464,12 +2468,22 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public int forceRepairAsync(String keyspace, boolean isSequential, 
CollectionString dataCenters, CollectionString hosts, boolean primaryRange, 
boolean fullRepair, String... columnFamilies) throws IOException
 {
-// when repairing only primary range, dataCenter nor hosts can be set
-if (primaryRange  (dataCenters != null || hosts != null))
+CollectionRangeToken ranges;
+if (primaryRange)
+{
+// when repairing only primary range, neither dataCenters nor 
hosts can be set
+if (dataCenters == null  hosts == null)
+ranges = getPrimaryRanges(keyspace);
+// except dataCenters only contain local DC (i.e. -local)
+else if (dataCenters != null  dataCenters.size() == 1  
dataCenters.contains(DatabaseDescriptor.getLocalDataCenter()))
+ranges = getPrimaryRangesWithinDC(keyspace);
+else
+throw new IllegalArgumentException(You need to run primary

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87ba0dc1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87ba0dc1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87ba0dc1

Branch: refs/heads/trunk
Commit: 87ba0dc1bce26ab97f5dd26edfd4a5c91ef9df48
Parents: 50c27cc 125b464
Author: Yuki Morishita yu...@apache.org
Authored: Wed Sep 17 09:48:14 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed Sep 17 09:48:14 2014 -0500

--
 .../cassandra/service/StorageService.java   |  92 +---
 .../org/apache/cassandra/tools/NodeTool.java|   2 +-
 .../service/StorageServiceServerTest.java   | 210 ++-
 3 files changed, 267 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/87ba0dc1/src/java/org/apache/cassandra/service/StorageService.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/87ba0dc1/src/java/org/apache/cassandra/tools/NodeTool.java
--

[1/3] git commit: Make repair -pr work with -local

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 90a211455 - 125b4642d
  refs/heads/trunk 50c27cca4 - 87ba0dc1b


Make repair -pr work with -local

patch by Paulo Motta; reviewed by yukim for CASSANDRA-7450


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/125b4642
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/125b4642
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/125b4642

Branch: refs/heads/cassandra-2.1
Commit: 125b4642db63a37c560dec4615ea04662c6e45ef
Parents: 90a2114
Author: Paulo Motta pauloricard...@gmail.com
Authored: Wed Sep 17 09:46:52 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed Sep 17 09:46:52 2014 -0500

--
 .../cassandra/service/StorageService.java   |  92 +---
 .../org/apache/cassandra/tools/NodeTool.java|   2 +-
 .../service/StorageServiceServerTest.java   | 210 ++-
 3 files changed, 267 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/125b4642/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 1aa3b24..b1fe20e 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -43,7 +43,6 @@ import ch.qos.logback.classic.jmx.JMXConfiguratorMBean;
 import ch.qos.logback.classic.spi.ILoggingEvent;
 import ch.qos.logback.core.Appender;
 
-import com.google.common.annotations.VisibleForTesting;
 import com.google.common.base.Predicate;
 import com.google.common.collect.*;
 import com.google.common.util.concurrent.FutureCallback;
@@ -156,11 +155,16 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 return getRangesForEndpoint(keyspaceName, 
FBUtilities.getBroadcastAddress());
 }
 
-public CollectionRangeToken getLocalPrimaryRanges(String keyspace)
+public CollectionRangeToken getPrimaryRanges(String keyspace)
 {
 return getPrimaryRangesForEndpoint(keyspace, 
FBUtilities.getBroadcastAddress());
 }
 
+public CollectionRangeToken getPrimaryRangesWithinDC(String keyspace)
+{
+return getPrimaryRangeForEndpointWithinDC(keyspace, 
FBUtilities.getBroadcastAddress());
+}
+
 private final SetInetAddress replicatingNodes = 
Collections.synchronizedSet(new HashSetInetAddress());
 private CassandraDaemon daemon;
 
@@ -332,7 +336,7 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 {
 throw new IllegalStateException(No configured daemon);
 }
-
+
 try
 {
 daemon.nativeServer.start();
@@ -430,10 +434,10 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 if 
(Gossiper.instance.getEndpointStateForEndpoint(DatabaseDescriptor.getReplaceAddress()).getApplicationState(ApplicationState.TOKENS)
 == null)
 throw new RuntimeException(Could not find tokens for  + 
DatabaseDescriptor.getReplaceAddress() +  to replace);
 CollectionToken tokens = 
TokenSerializer.deserialize(getPartitioner(), new DataInputStream(new 
ByteArrayInputStream(getApplicationStateValue(DatabaseDescriptor.getReplaceAddress(),
 ApplicationState.TOKENS;
-
+
 SystemKeyspace.setLocalHostId(hostId); // use the replacee's host 
Id as our own so we receive hints, etc
 Gossiper.instance.resetEndpointStateMap(); // clean up since we 
have what we need
-return tokens;
+return tokens;
 }
 catch (IOException e)
 {
@@ -2464,12 +2468,22 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 
 public int forceRepairAsync(String keyspace, boolean isSequential, 
CollectionString dataCenters, CollectionString hosts, boolean primaryRange, 
boolean fullRepair, String... columnFamilies) throws IOException
 {
-// when repairing only primary range, dataCenter nor hosts can be set
-if (primaryRange  (dataCenters != null || hosts != null))
+CollectionRangeToken ranges;
+if (primaryRange)
+{
+// when repairing only primary range, neither dataCenters nor 
hosts can be set
+if (dataCenters == null  hosts == null)
+ranges = getPrimaryRanges(keyspace);
+// except dataCenters only contain local DC (i.e. -local)
+else if (dataCenters != null  dataCenters.size() == 1  
dataCenters.contains(DatabaseDescriptor.getLocalDataCenter()))
+

[jira] [Updated] (CASSANDRA-7650) Expose auto_bootstrap as a system property override

2014-09-17 Thread Joe Hohertz (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Joe Hohertz updated CASSANDRA-7650:
---
Reproduced In: 2.0.9, 1.2.18  (was: 1.2.18, 2.0.9)
Fix Version/s: 1.2.19

 Expose auto_bootstrap as a system property override
 ---

 Key: CASSANDRA-7650
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7650
 Project: Cassandra
  Issue Type: Improvement
  Components: Config
Reporter: Joe Hohertz
Assignee: Joe Hohertz
Priority: Trivial
  Labels: auto_bootstrap, config, lhf
 Fix For: 1.2.19, 2.0.10, 2.1.0

 Attachments: cassandra_auto_bootstrap.patch


 This one-line change gives the auto_bootstrap config parameter the ability to 
 be overridden by a system property.
 As I understand things, setting this false is generally a one-time initial 
 set-up item, and providing this will be potentially easier than swapping out 
 yaml file items when the need to set this false arises.
 The handling is the same as was already implemented for replace_address and 
 the like, look to the system property, and default that to the yaml config 
 value.
 This also will also allow a start-up agent (like Priam) to influence the 
 setting from within it's control over Cassandra's start-up behaviour
 Patch was generated against 1.2 branch but applies ok to 2.0 as well. (It 
 would be great if this could be applied to both code lines)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7395) Support for pure user-defined functions (UDF)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-7395:

Tester: Ryan McGuire

 Support for pure user-defined functions (UDF)
 -

 Key: CASSANDRA-7395
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7395
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Robert Stupp
  Labels: cql
 Fix For: 3.0

 Attachments: 7395-dtest.txt, 7395.txt, udf-create-syntax.png, 
 udf-drop-syntax.png


 We have some tickets for various aspects of UDF (CASSANDRA-4914, 
 CASSANDRA-5970, CASSANDRA-4998) but they all suffer from various degrees of 
 ocean-boiling.
 Let's start with something simple: allowing pure user-defined functions in 
 the SELECT clause of a CQL query.  That's it.
 By pure I mean, must depend only on the input parameters.  No side effects. 
  No exposure to C* internals.  Column values in, result out.  
 http://en.wikipedia.org/wiki/Pure_function



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7955) Range movements can violate consistency if RING_DELAY = write_request_timeout_in_ms


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137362#comment-14137362
 ] 

Brandon Williams commented on CASSANDRA-7955:
-

We could also just automatically bump ring delay up if the write timeout is 
greater.

 Range movements can violate consistency if RING_DELAY = 
 write_request_timeout_in_ms
 

 Key: CASSANDRA-7955
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7955
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Priority: Minor

 Cassandra should probably refuse to start if RING_DELAY is too close to (or 
 below) write_request_timeout_ms, because we depend on this for 
 correctness/consistency during range movements
 We should probably also consider throwing a WriteTimeoutException _even if we 
 don't get interrupted by the timeout, since there are reasons due to 
 scheduling or system overload this could not happen (though it is unlikely to 
 be significant enough to have an impact, it's better safe than sorry)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7523) add date and time types

2014-09-17 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137366#comment-14137366
 ] 

Joshua McKenzie commented on CASSANDRA-7523:


Sit tight on the review on this for a few Tyler - I'm incorporating some of the 
above changes.  Spoke briefly w/Benedict offline; our Time type is 
byte-order-comparable out of the gate since we're bounding it between 0-N, with 
N being nanos per day, and I should be able to get SimpleDate into the same 
boat without too much added complexity.

 add date and time types
 ---

 Key: CASSANDRA-7523
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7523
 Project: Cassandra
  Issue Type: New Feature
  Components: API
Reporter: Jonathan Ellis
Assignee: Joshua McKenzie
Priority: Minor
 Fix For: 2.1.1, 3.0


 http://www.postgresql.org/docs/9.1/static/datatype-datetime.html
 (we already have timestamp; interval is out of scope for now, and see 
 CASSANDRA-6350 for discussion on timestamp-with-time-zone.  but date/time 
 should be pretty easy to add.)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7957) improve active/pending compaction monitoring

Nikolai Grigoriev created CASSANDRA-7957:


 Summary: improve active/pending compaction monitoring
 Key: CASSANDRA-7957
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7957
 Project: Cassandra
  Issue Type: Improvement
  Components: Core, Tools
Reporter: Nikolai Grigoriev
Priority: Minor


I think it might be useful to create a way to see what sstables are being 
compacted into what new sstable. Something like an extension of nodetool 
compactionstats. I think it would be easier with this feature to troubleshoot 
and understand how compactions are happening on your data. Not sure how it is 
useful in everyday life but I could use such a feature when dealing with 
CASSANDRA-7949.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7958) Create jenkins targets to run CQLTester unit tests with prepared statements and without

Ryan McGuire created CASSANDRA-7958:
---

 Summary: Create jenkins targets to run CQLTester unit tests with 
prepared statements and without
 Key: CASSANDRA-7958
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7958
 Project: Cassandra
  Issue Type: Test
Reporter: Ryan McGuire
Assignee: Michael Shuler


The CQL tests within the unit test code has the ability to run with prepared 
statements, or without, using the cassandra.test.use_prepared flag. We should 
create two jenkins targets on CassCI to run it both ways.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7959) CQL unit tests should be run with Java driver

Ryan McGuire created CASSANDRA-7959:
---

 Summary: CQL unit tests should be run with Java driver
 Key: CASSANDRA-7959
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7959
 Project: Cassandra
  Issue Type: Test
Reporter: Ryan McGuire
Assignee: Ryan McGuire


The tests run by the CQLTester use internal C* calls to operate. We should 
refactor these tests to be able to run the current way as well as via the Java 
driver.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7955) Range movements can violate consistency if RING_DELAY = write_request_timeout_in_ms


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137409#comment-14137409
 ] 

Benedict commented on CASSANDRA-7955:
-

That would work, I just favoured letting the user know they were screwing 
around with this. 

If we do it automatically, we should probably gossip the ring delay and take 
the max of all we can see, though, while we're at it. If one node configures a 
high write timeout, the whole cluster is potentially permitted to get into an 
inconsistent state

 Range movements can violate consistency if RING_DELAY = 
 write_request_timeout_in_ms
 

 Key: CASSANDRA-7955
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7955
 Project: Cassandra
  Issue Type: Improvement
Reporter: Benedict
Priority: Minor

 Cassandra should probably refuse to start if RING_DELAY is too close to (or 
 below) write_request_timeout_ms, because we depend on this for 
 correctness/consistency during range movements
 We should probably also consider throwing a WriteTimeoutException _even if we 
 don't get interrupted by the timeout, since there are reasons due to 
 scheduling or system overload this could not happen (though it is unlikely to 
 be significant enough to have an impact, it's better safe than sorry)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7960) cassandra-stress should support CAS

Benedict created CASSANDRA-7960:
---

 Summary: cassandra-stress should support CAS
 Key: CASSANDRA-7960
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7960
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Benedict
Priority: Minor






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7032) Improve vnode allocation


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-7032:

Assignee: Branimir Lambov

 Improve vnode allocation
 

 Key: CASSANDRA-7032
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7032
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Branimir Lambov
  Labels: performance, vnodes
 Fix For: 3.0

 Attachments: TestVNodeAllocation.java, TestVNodeAllocation.java


 It's been known for a little while that random vnode allocation causes 
 hotspots of ownership. It should be possible to improve dramatically on this 
 with deterministic allocation. I have quickly thrown together a simple greedy 
 algorithm that allocates vnodes efficiently, and will repair hotspots in a 
 randomly allocated cluster gradually as more nodes are added, and also 
 ensures that token ranges are fairly evenly spread between nodes (somewhat 
 tunably so). The allocation still permits slight discrepancies in ownership, 
 but it is bound by the inverse of the size of the cluster (as opposed to 
 random allocation, which strangely gets worse as the cluster size increases). 
 I'm sure there is a decent dynamic programming solution to this that would be 
 even better.
 If on joining the ring a new node were to CAS a shared table where a 
 canonical allocation of token ranges lives after running this (or a similar) 
 algorithm, we could then get guaranteed bounds on the ownership distribution 
 in a cluster. This will also help for CASSANDRA-6696.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7566) DROP TABLE should also drop prepared statements associated to


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137524#comment-14137524
 ] 

Tyler Hobbs commented on CASSANDRA-7566:


bq. it fetches the column family name from the statement, but then just uses 
the column family passed as a parameter. Are these always the same? if so, 
probably should just remove 'statementCfName'

The method parameters are the ks and cf that was dropped.  The statement we're 
looking at is the cached prepared statement.  We're just checking that the ks 
and cf (if a table was dropped) match the statement's.

 DROP TABLE should also drop prepared statements associated to
 -

 Key: CASSANDRA-7566
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7566
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: OSX 10.9.4 / JDK 1.8.0_05
Reporter: Ben Hood
Assignee: Viju Kothuvatiparambil
  Labels: bootcamp, cql3
 Fix For: 2.1.1

 Attachments: trunk-7566.txt


 An integration test suite that drops and creates the same column family 3 
 times causes the following error in the server log:
 INFO  15:40:34 Initializing gocql_test.wiki_page
 ERROR 15:40:34 Attempted to write commit log entry for unrecognized column 
 family: b0e167e0-0dc8-11e4-9cbb-29a4872887f2
 ERROR 15:40:34 Attempting to mutate non-existant column family 
 b0e167e0-0dc8-11e4-9cbb-29a4872887f2
 ERROR 15:40:34 Attempted to write commit log entry for unrecognized column 
 family: b0e167e0-0dc8-11e4-9cbb-29a4872887f2
 ERROR 15:40:34 Attempting to mutate non-existant column family 
 b0e167e0-0dc8-11e4-9cbb-29a4872887f2
 The test that reproduces this issue is here:
 https://github.com/gocql/gocql/blob/master/wiki_test.go
 Interestingly this issue only occurs after the common table is 
 dropped/created for the 3rd time. If only one of the tests is run on its own, 
 this issue does not arise.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7961) Run dtests on ARM

Ryan McGuire created CASSANDRA-7961:
---

 Summary: Run dtests on ARM
 Key: CASSANDRA-7961
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7961
 Project: Cassandra
  Issue Type: Test
Reporter: Ryan McGuire
Assignee: Michael Shuler


It looks pretty easy to setup an [emulated ARM 
environment|https://wiki.ubuntu.com/ARM/RootfsFromScratch/QemuDebootstrap], we 
might want to look into setting this up on Cassci.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7961) Run dtests on ARM


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ryan McGuire updated CASSANDRA-7961:

Priority: Minor  (was: Major)

 Run dtests on ARM
 -

 Key: CASSANDRA-7961
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7961
 Project: Cassandra
  Issue Type: Test
Reporter: Ryan McGuire
Assignee: Michael Shuler
Priority: Minor

 It looks pretty easy to setup an [emulated ARM 
 environment|https://wiki.ubuntu.com/ARM/RootfsFromScratch/QemuDebootstrap], 
 we might want to look into setting this up on Cassci.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7843) Remove static singleton instances

2014-09-17 Thread Aleksey Yeschenko (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137598#comment-14137598
 ] 

Aleksey Yeschenko commented on CASSANDRA-7843:
--

Nobody's arguing that singletons are an antipattern, but yeah. Just 'get rid of 
all the singletons' should not be the goal.

We should only refactor stuff if it makes code cleaner - be pragmatic, not 
absolutist. 400 commits. Man.

 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7282) Faster Memtable map

2014-09-17 Thread Jason Brown (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137603#comment-14137603
 ] 

Jason Brown commented on CASSANDRA-7282:


Food for thought:

I've reviewed the branch Benedict has linked, and while I'll run the stress 
later today, I'd like to add some thoughts on the code itself. 

To perhaps state the obvious from reading the patch, the field Memtable.rows is 
now a InsertOnlyOrderedMap instead of the previous ConcurrentSkipListMap. If 
the Partitioner sorts data by hash code (sortsByHashCode()), Memtable will use 
the simple IOOM.Adapter implementation; else Memtable uses the 
NonBlockingHashOrderedMap. I think we can agree NBHOM is considerably more code 
and more involved than IIOM.Adapter (wait 'til the end, please, Benedict :) ).

Currently, only the Murmur3Partitioner is written to sort by hash code. I would 
venture to guess the vast majority of new clusters created = cassandra 1.2 use 
the Murmur3Partitioner, and, further, the vast majority of clusters created 
before 1.2 used RandomPartitioner. Thus, not many clusters use the ordered 
partitioners; we've also steered folks away from those unless they really know 
what they are doing (and then we still try to dissuade them).If we take the 
assumption that most clusters' partioner is either Murmur3 or Random, and weigh 
it against the the current patch, the majority of the code (and, what some have 
argued, the complexity) is to support the RandomPartition and the ordered 
partitioners. If we can make RandomPartitioner be able to sort by hash code, 
and thus be able to take advantage of IIOM.Adapter's functionality, then the 
NBHOM-related functionality would only be necessary for for ordered 
partitioners (let's leave LocalPartitioner, used for indices, out for a 
moment). It seems to me that we could pass on the NBHOM-related functionality 
as it would only be beneficial for a small subset of uses case; not zero use 
cases, mind you, but perhaps enough to take a pass on it for this round. I 
think some commenters concerns about the perceived complexity of the patch 
would then be assuaged, and we can deliver an improvement performance for the 
majority of uses.

[~benedict], Please let me know if I've misunderstood something fundamental 
about the patch that would invalidate my arguments; the 
conclusion/recommendation is, of course, open for discussion.

 Faster Memtable map
 ---

 Key: CASSANDRA-7282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 3.0

 Attachments: profile.yaml, reads.svg, run1.svg, writes.svg


 Currently we maintain a ConcurrentSkipLastMap of DecoratedKey - Partition in 
 our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
 majority of users use a hash partitioner, it occurs to me we could maintain a 
 hybrid ordered list / hash map. The list would impose the normal order on the 
 collection, but a hash index would live alongside as part of the same data 
 structure, simply mapping into the list and permitting O(1) lookups and 
 inserts.
 I've chosen to implement this initial version as a linked-list node per item, 
 but we can optimise this in future by storing fatter nodes that permit a 
 cache-line's worth of hashes to be checked at once,  further reducing the 
 constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7282) Faster Memtable map

[
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137609#comment-14137609
]

Benedict commented on CASSANDRA-7282:
-

I think you may have misunderstood, or I have misunderstood what you're saying.
I assume by IIOM you mean IOOM. But the IOOM.Adapter is not a feature, it's a
support for old functionality. RandomPartitioner is currently supported by
IOOM.Adapter backed by CSLM, because I was too lazy at the time to update RP's
Tokens.

NBHOM is essentially the entirety of the patch, _not_ a legacy support
mechanism. It is an efficient means for supporting Murmur3 (and optionally
Random) partitioners.

Faster Memtable map
---

Key: CASSANDRA-7282
URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Benedict
Assignee: Benedict
Labels: performance
Fix For: 3.0

Attachments: profile.yaml, reads.svg, run1.svg, writes.svg

Currently we maintain a ConcurrentSkipLastMap of DecoratedKey - Partition in
our memtables. Maintaining this is an O(lg(n)) operation; since the vast
majority of users use a hash partitioner, it occurs to me we could maintain a
hybrid ordered list / hash map. The list would impose the normal order on the
collection, but a hash index would live alongside as part of the same data
structure, simply mapping into the list and permitting O(1) lookups and
inserts.
I've chosen to implement this initial version as a linked-list node per item,
but we can optimise this in future by storing fatter nodes that permit a
cache-line's worth of hashes to be checked at once, further reducing the
constant factor costs for lookups.

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7843) Remove static singleton instances

2014-09-17 Thread Jon Haddad (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137612#comment-14137612
 ] 

Jon Haddad commented on CASSANDRA-7843:
---

I'm curious, is the issue the number of commits, LOC, or a general disagreement 
with removing singletons?  

Is the alternative to break it out into smaller commits, or do less over time?


 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (CASSANDRA-7843) Remove static singleton instances

2014-09-17 Thread Jon Haddad (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137612#comment-14137612
 ] 

Jon Haddad edited comment on CASSANDRA-7843 at 9/17/14 5:44 PM:


I'm curious, is the issue the number of commits, LOC, or a general disagreement 
with removing singletons?  

Is the alternative to break it out into smaller commits, or do less over time?  
Or would you rather couple refactoring with new features / bug fixes?  


was (Author: rustyrazorblade):
I'm curious, is the issue the number of commits, LOC, or a general disagreement 
with removing singletons?  

Is the alternative to break it out into smaller commits, or do less over time?


 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7843) Remove static singleton instances


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137621#comment-14137621
 ] 

Brandon Williams commented on CASSANDRA-7843:
-

bq. Nobody's arguing that singletons are an antipattern, but yeah. Just 'get 
rid of all the singletons' should not be the goal.

I agree with this, we need specific goals to solve and this approach seems to 
be using the shotgun method and blasting anything with a singleton.  A good 
example of this is the CLI.  Sure, we'll be removing the CLI in 3.0, but even 
if we weren't, there's no reason to kill the singletons there just because 
they're singletons.

 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7843) Remove static singleton instances

2014-09-17 Thread Joshua McKenzie (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137622#comment-14137622
 ] 

Joshua McKenzie commented on CASSANDRA-7843:


It seems to me that there's some pretty clear lines we could split this patch 
up into.  Rather than boil the ocean, do all the things removing some of the 
static singletons on classes we'd like to be able to do more isolated testing 
with (Gossip anyone?) with individual tickets to track each effort might be 
more palatable.

The linked github branch touches just short of 600 files.  The risk of 
something incorrect slipping through (or diverging over time for that matter) 
is very high with that many changes.

 Remove static singleton instances
 -

 Key: CASSANDRA-7843
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7843
 Project: Cassandra
  Issue Type: Sub-task
Reporter: Blake Eggleston
Assignee: Blake Eggleston

 Initialize and start C* from a static method that takes a configuration as an 
 argument.
 CassandraDaemon and utilities will need to be updated, and we'll need a 
 helper method so tests can instantiate the database with a one or two liner.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7282) Faster Memtable map

2014-09-17 Thread Jason Brown (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137638#comment-14137638
 ] 

Jason Brown commented on CASSANDRA-7282:


damnit, there's a ! operator there

{code}if (!cfs.partitioner.sortsByHashCode()) {code}

oh, well, I tried :)


 Faster Memtable map
 ---

 Key: CASSANDRA-7282
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7282
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Benedict
Assignee: Benedict
  Labels: performance
 Fix For: 3.0

 Attachments: profile.yaml, reads.svg, run1.svg, writes.svg


 Currently we maintain a ConcurrentSkipLastMap of DecoratedKey - Partition in 
 our memtables. Maintaining this is an O(lg(n)) operation; since the vast 
 majority of users use a hash partitioner, it occurs to me we could maintain a 
 hybrid ordered list / hash map. The list would impose the normal order on the 
 collection, but a hash index would live alongside as part of the same data 
 structure, simply mapping into the list and permitting O(1) lookups and 
 inserts.
 I've chosen to implement this initial version as a linked-list node per item, 
 but we can optimise this in future by storing fatter nodes that permit a 
 cache-line's worth of hashes to be checked at once,  further reducing the 
 constant factor costs for lookups.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7941) Fix bin/cassandra cassandra.logdir option in debian package

2014-09-17 Thread Jared Biel (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137654#comment-14137654
 ] 

Jared Biel commented on CASSANDRA-7941:
---

It would be great to get a -2 package version with this fix.

 Fix bin/cassandra cassandra.logdir option in debian package
 ---

 Key: CASSANDRA-7941
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7941
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
Reporter: Michael Shuler
Assignee: Michael Shuler
 Fix For: 2.1.1

 Attachments: 
 0001-Fix-bin-cassandra-cassandra.logdir-option-in-debian-.patch


 Cassandra writes logs to $CASSANDRA_HOME/logs by default, and the debian 
 package needs to write to /var/log/cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7962) usage counters on prepared statements, summary and details

Jonathan Shook created CASSANDRA-7962:
-

 Summary: usage counters on prepared statements, summary and details
 Key: CASSANDRA-7962
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Shook
Priority: Minor


Count the usages of prepared statements on the server side, and expose them via 
JMX. The original query structure should be included. Expose a meaningful id 
which can be used in other places, such as with JMX clients.  If enumerations 
are used to identify these for easier discussion, enumerate according to the 
statement id.

Allow for since last access deltas, or since server start, and require this 
parameter to be specified.

This would allow easier identification of access pattern distribution in the 
case that prepared statements are being used. It would provide useful metrics 
for diagnosis, testing, and monitoring.

.h3 Example nodetool outputs
command syntax:

.bq nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) 
[ ks [ table ] ]

# nodetool spusagecountsummary deltas appks usertable 
(count, statement id)
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9

# nodetool spusagecountdetails alltime appks usertable 
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud  where userid=?;

... and so forth ...





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Shook updated CASSANDRA-7962:
--
Description:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Expose a meaningful id
which can be used in other places, such as with JMX clients. If enumerations
are used to identify these for easier discussion, enumerate according to the
statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

Example nodetool outputs
command syntax:

nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

# nodetool spusagecountsummary deltas appks usertable
(count, statement id)
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9

# nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

was:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Expose a meaningful id
which can be used in other places, such as with JMX clients. If enumerations
are used to identify these for easier discussion, enumerate according to the
statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

.h3 Example nodetool outputs
command syntax:

.bq nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime )
[ ks [ table ] ]

# nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Expose a meaningful
id which can be used in other places, such as with JMX clients. If
enumerations are used to identify these for easier discussion, enumerate
according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
Example nodetool outputs
command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
# nodetool spusagecountsummary deltas appks usertable
(count, statement id)
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
# nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;
265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...
11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;
... and so forth ...

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allow for since last access deltas, or since server start, and require this
parameter to be specified.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

Example nodetool outputs
command syntax:

nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountsummary deltas appks usertable
(count, statement id)
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

Allow for since last access deltas, or since server start, and require this
parameter to be specified.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

Example nodetool outputs
command syntax:

nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

# nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Expose a meaningful
id which can be used in other places, such as with JMX clients. If
enumerations are used to identify these for easier discussion, enumerate
according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
Example nodetool outputs
command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
nodetool spusagecountsummary deltas appks usertable
(count, statement id)
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;
265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...
11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;
... and so forth ...

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

Example nodetool outputs
command syntax:

nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

Allow for since last access deltas, or since server start, and require this
parameter to be specified.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

Example nodetool outputs
command syntax:

nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Expose a meaningful
id which can be used in other places, such as with JMX clients. If
enumerations are used to identify these for easier discussion, enumerate
according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified. When the deltas form is used, show the number
of seconds represent by the current data.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
Example nodetool outputs
command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;
265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...
11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

Example nodetool outputs:
nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

Example nodetool outputs
command syntax:

nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Expose a meaningful
id which can be used in other places, such as with JMX clients. If
enumerations are used to identify these for easier discussion, enumerate
according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified. When the deltas form is used, show the number
of seconds represent by the current data.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
Example nodetool outputs:
nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;
265

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Shook updated CASSANDRA-7962:
--
Description:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

was:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with JMX clients. If enumerations are used to identify these
for easier discussion, enumerate according to the statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified. When the deltas form is used, show the number
of seconds represent by the current data.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
Example nodetool outputs:
nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
nodetool spusagecountdetails

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Shook updated CASSANDRA-7962:
--
Description:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with JMX clients. If enumerations are used to identify these
for easier discussion, enumerate according to the statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with JMX clients. If enumerations are used to identify these
for easier discussion, enumerate according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified. When the deltas form is used, show the number
of seconds represent by the current data.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
Example nodetool outputs:
nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Shook updated CASSANDRA-7962:
--
Description:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represented by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

was:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represent by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified. When the deltas form is used, show the number
of seconds represented by the current data.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
Example nodetool outputs:
nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
nodetool

[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details

[
https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Shook updated CASSANDRA-7962:
--
Description:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified.

Show the number of seconds represented by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n), since 234234233 seconds ago
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

was:
Count the usages of prepared statements on the server side, and expose them via
JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.

Allow for since last access deltas, or since server start, and require this
parameter to be specified. When the deltas form is used, show the number of
seconds represented by the current data.

This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.

nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]

nodetool spusagecountdetails alltime appks usertable
(count, statement id,\n query\n\n)
56320304 24ad327f9bb2578de663fc92336703dd
select user from usertable where userid=?;

265 8743b52063cd84097a65d1633f5c74f5
insert into usertable (userid,...) ...

11 663fc92336703dd24ad327f9bb2578de
select tags from tagcloud where userid=?;

... and so forth ...

usage counters on prepared statements, summary and details
--

Key: CASSANDRA-7962
URL: https://issues.apache.org/jira/browse/CASSANDRA-7962
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Jonathan Shook
Priority: Minor

Count the usages of prepared statements on the server side, and expose them
via JMX. The original query structure should be included. Provide a nodetool
command to see the counts. Expose a meaningful id which can be used in other
places, such as with other JMX clients. If enumerations are used to identify
these for easier discussion, enumerate according to the statement id.
Allow for since last access deltas, or since server start, and require
this parameter to be specified.
Show the number of seconds represented by the current data.
This would allow easier identification of access pattern distribution in the
case that prepared statements are being used. It would provide useful metrics
for diagnosis, testing, and monitoring.
nodetool command syntax:
nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [
ks [ table ] ]
Example nodetool outputs:
nodetool spusagecountsummary deltas appks usertable
(count, statement id), since 234233 seconds ago
56304 24ad327f9bb2578de663fc92336703dd
5 8743b52063cd84097a65d1633f5c74f5
1 663fc92336703dd24ad327f9bb2578de
0 92336703dd24ad327f9bb2578de663fc
0 bb2578de663fc922336703dd24ad327f9
nodetool spusagecountdetails alltime appks usertable

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1413#comment-1413
 ] 

graham sanderson commented on CASSANDRA-7546:
-

OK, so I'm running latest stress.jar on my load machine - given the number of 
changes to stress in 2.1.1 (and the addition by the looks of things of remote 
GC logging via cassandra-stress which would be useful in this case), I guess 
I'll upgrade the cluster as well.

Here is my current config (minus the comments) and the launch command... note 
there were some typos in our conversation above

{code}
keyspace: stresscql

keyspace_definition: |
  CREATE KEYSPACE stresscql WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 3};

table: testtable

table_definition: |
  CREATE TABLE testtable (
p text,
c1 int, c2 int, c3 int,
v blob,
PRIMARY KEY(p, c1, c2, c3)
  ) WITH COMPACT STORAGE 
AND compaction = { 'class':'LeveledCompactionStrategy' }
AND comment='TestTable'

columnspec:
  - name: p
size: fixed(16)
  - name: c1
cluster: fixed(100)
  - name: c2
cluster: fixed(100)
  - name: c3
cluster: fixed(1000) # note I made it slightly bigger since 10M is better 
than 1M for a max - 1M happens pretty quickly
  - name: v
size: gaussian(50..250)

queries:
   simple1:
  cql: select * from testtable where k = ? and v = ? LIMIT 10
  fields: samerow
{code}

{code}
./cassandra-stress user profile=~/cqlstress-7546.yaml ops\(insert=1\) 
cl=LOCAL_QUORUM -node $NODES -mode native prepared cql3 -pop seq=1..10M -insert 
visits=fixed\(10M\) revisit=uniform\(1..1024\) | tee 
results/results-2.1.0-p1024-a.txt
{code}

As of right now, we're still (8 minutes later) at:

{code}
INFO  19:11:51 Using data-center name 'Austin' for DCAwareRoundRobinPolicy (if 
this is incorrect, please provide the correct datacenter name with 
DCAwareRoundRobinPolicy constructor)
Connected to cluster: Austin Multi-Tenant Cassandra 1
INFO  19:11:51 New Cassandra host cassandra4.aus.vast.com/172.17.26.14:9042 
added
Datatacenter: Austin; Host: cassandra4.aus.vast.com/172.17.26.14; Rack: 98.9
Datatacenter: Austin; Host: /172.17.26.15; Rack: 98.9
Datatacenter: Austin; Host: /172.17.26.13; Rack: 98.9
Datatacenter: Austin; Host: /172.17.26.12; Rack: 98.9
Datatacenter: Austin; Host: /172.17.26.11; Rack: 98.9
INFO  19:11:51 New Cassandra host /172.17.26.12:9042 added
INFO  19:11:51 New Cassandra host /172.17.26.11:9042 added
INFO  19:11:51 New Cassandra host /172.17.26.13:9042 added
INFO  19:11:51 New Cassandra host /172.17.26.15:9042 added
Created schema. Sleeping 5s for propagation.
Warming up insert with 25 iterations...
Failed to connect over JMX; not collecting these stats
Generating batches with [1..1] partitions and [1..1] rows (of 
[1000..1000] total rows in the partitions)
{code}

Number of distinct partitions is currently 2365 and growing.

Is this what we expect? doesn't seem like 250,000 should have exhausted any 
partitions?

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 2.1.1

 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 
 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 
 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, hint_spikes.png, 
 suggestion1.txt, suggestion1_21.txt, young_gen_gc.png


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7849) Server logged error messages (in binary protocol) for unexpected exceptions could be more helpful


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137792#comment-14137792
 ] 

Tyler Hobbs commented on CASSANDRA-7849:


{noformat}
1) Leave in the new the extra information in the message (this IS useful) - 
i.e. IP addresses of each end of the channel
2) Just use DEBUG level for Message.exceptionCaught code path
3) Possibly make the decision at that code path for ERROR vs DEBUG based on 
instanceof IOException?
{noformat}

That sounds like a good plan to me.  Log exceptions from the 
{{Message.exceptionCaught}} patch at INFO unless they're an IOException, in 
which case DEBUG is used.  I would even be fine with checking IOException 
messages for Connection reset by peer and only logging those particular 
exceptions at DEBUG.

 Server logged error messages (in binary protocol) for unexpected exceptions 
 could be more helpful
 -

 Key: CASSANDRA-7849
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7849
 Project: Cassandra
  Issue Type: Improvement
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 1.2.19, 2.0.11

 Attachments: cassandra-1.2-7849.txt, cassandra-1.2-7849_v2.txt


 From time to time (actually quite frequently) we get error messages in the 
 server logs like this
 {code}
 ERROR [Native-Transport-Requests:288] 2014-08-29 04:48:07,118 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.io.IOException: Connection reset by peer
 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
 at sun.nio.ch.IOUtil.read(IOUtil.java:192)
 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 {code}
 These particular cases are almost certainly problems with the client driver, 
 client machine, client process, however after the fact this particular 
 exception is practically impossible to debug because there is no indication 
 in the underlying JVM/netty exception of who the peer was. I should note we 
 have lots of different types of applications running against the cluster so 
 it is very hard to correlate these to anything



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137856#comment-14137856
 ] 

Benedict commented on CASSANDRA-7546:
-

Hmm. This looks like a subtle bug with the latest stress when operating over 
such a small domain. But also highlights a problem with using it for this 
workload - I may need to do some tweaking tomorrow to make it suitable. To 
ensure we keep our procedurally generated state for the partition intact we 
only let one insert thread operate over a given partition at a time. If there 
is a conflict, we fall back to the underlying id distribution to avoid wasting 
time. This means that with a small domain we will steadily visit more and more 
partitions, but also that we will never have competing updates to the same 
partition, which is a glaring limitation (especially here). As it happens, the 
latest version of the procedural generation is reasonably easy to safely 
partition the work across multiple threads without mutual exclusivity, so I'll 
try to patch that ASAP.

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 2.1.1

 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 
 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 
 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, hint_spikes.png, 
 suggestion1.txt, suggestion1_21.txt, young_gen_gc.png


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7849) Server logged error messages (in binary protocol) for unexpected exceptions could be more helpful


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137857#comment-14137857
 ] 

graham sanderson commented on CASSANDRA-7849:
-

Yeah the error message string comes from deep in native bowels of native JVM 
code, looks like it hard coded in JVM on Windows (probably to match *nix) and 
comes from the OS on *nix, so is possibly platform dependent and/or localized, 
which is why I was hesitant to touch it originally.

That said, if our intention is to ONLY put at DEBUG things we absolutely know 
to be spurious, then we could test the error message string, and just risk some 
OS or locale getting a different message at INFO - they could always open a new 
issue.

 Server logged error messages (in binary protocol) for unexpected exceptions 
 could be more helpful
 -

 Key: CASSANDRA-7849
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7849
 Project: Cassandra
  Issue Type: Improvement
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 1.2.19, 2.0.11

 Attachments: cassandra-1.2-7849.txt, cassandra-1.2-7849_v2.txt


 From time to time (actually quite frequently) we get error messages in the 
 server logs like this
 {code}
 ERROR [Native-Transport-Requests:288] 2014-08-29 04:48:07,118 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.io.IOException: Connection reset by peer
 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
 at sun.nio.ch.IOUtil.read(IOUtil.java:192)
 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 {code}
 These particular cases are almost certainly problems with the client driver, 
 client machine, client process, however after the fact this particular 
 exception is practically impossible to debug because there is no indication 
 in the underlying JVM/netty exception of who the peer was. I should note we 
 have lots of different types of applications running against the cluster so 
 it is very hard to correlate these to anything



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (CASSANDRA-7963) stress should ignore rate limiting for warmup

Brandon Williams created CASSANDRA-7963:
---

 Summary: stress should ignore rate limiting for warmup
 Key: CASSANDRA-7963
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7963
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Brandon Williams
Assignee: Benedict
Priority: Minor
 Fix For: 2.1.1


When you pass a -rate option to stress, it uses this for the warmup as well.  
If you're trying to test a very low rate for some reason, it takes forever to 
get through the 50k warmup iterations to actually get results.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7932) Corrupt SSTable Cleanup Leak Resources

2014-09-17 Thread Yuki Morishita (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137866#comment-14137866
 ] 

Yuki Morishita commented on CASSANDRA-7932:
---

I'm +1 for this to be in 2.1.
LGTM overall, though I'm seeing the following from output of 
ColumnFamilyStoreTest:

{noformat}
ERROR 19:41:53 Fatal exception in thread Thread[NonPeriodicTasks:1,5,main]
java.lang.NullPointerException: null
at 
org.apache.cassandra.io.sstable.SSTableReader$Tidier$1.run(SSTableReader.java:1993)
 ~[main/:na]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) 
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) 
~[na:1.7.0_51]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
 ~[na:1.7.0_51]
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
 ~[na:1.7.0_51]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
~[na:1.7.0_51]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{noformat}

Nits:

* There is unused AbstractRefCounted class
* You can use {{@Test(expected = IllegalStateException.class)}} for the last 
test in RefCountedTest.

 Corrupt SSTable Cleanup Leak Resources
 --

 Key: CASSANDRA-7932
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7932
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1.1


 CASSANDRA-7705, during the BlackingListingCompactionsTest, detected leaks. 
 I've tracked this down to DataTracker.removeUnreadableSSTables() , which does 
 not release the reference to any sstables from the tracker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137872#comment-14137872
 ] 

graham sanderson commented on CASSANDRA-7546:
-

Cool, thanks, I'll wait on your patch (I have plenty of other things to do ;-) 
). that said, am I relatively safe to upgrade the actual nodes to current head 
of 2.1 branch (and thusly pick up your latest GC monitoring stuff?) if I have a 
spare moment before then? Ideally I'd upgrade to the last commit in 2.1 needed 
to be in place on test nodes for correct latest cassandra-stress operation.

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 2.1.1

 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 
 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 
 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, hint_spikes.png, 
 suggestion1.txt, suggestion1_21.txt, young_gen_gc.png


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7932) Corrupt SSTable Cleanup Leak Resources


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137874#comment-14137874
 ] 

Benedict commented on CASSANDRA-7932:
-

bq. I've patched on top of the leak detection code for trunk

Sounds like you reviewed the underlying patch for CASSANDRA-7705 as well, not 
just the bug fix? In which case, thanks! Wasn't planning on rolling that out to 
2.1, although there's a good argument to be made there. Think that should get 
some wider input though.



 Corrupt SSTable Cleanup Leak Resources
 --

 Key: CASSANDRA-7932
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7932
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1.1


 CASSANDRA-7705, during the BlackingListingCompactionsTest, detected leaks. 
 I've tracked this down to DataTracker.removeUnreadableSSTables() , which does 
 not release the reference to any sstables from the tracker.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7849) Server logged error messages (in binary protocol) for unexpected exceptions could be more helpful


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137886#comment-14137886
 ] 

Tyler Hobbs commented on CASSANDRA-7849:


bq. That said, if our intention is to ONLY put at DEBUG things we absolutely 
know to be spurious, then we could test the error message string, and just risk 
some OS or locale getting a different message at INFO - they could always open 
a new issue.

+1

 Server logged error messages (in binary protocol) for unexpected exceptions 
 could be more helpful
 -

 Key: CASSANDRA-7849
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7849
 Project: Cassandra
  Issue Type: Improvement
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 1.2.19, 2.0.11

 Attachments: cassandra-1.2-7849.txt, cassandra-1.2-7849_v2.txt


 From time to time (actually quite frequently) we get error messages in the 
 server logs like this
 {code}
 ERROR [Native-Transport-Requests:288] 2014-08-29 04:48:07,118 
 ErrorMessage.java (line 222) Unexpected exception during request
 java.io.IOException: Connection reset by peer
 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
 at sun.nio.ch.IOUtil.read(IOUtil.java:192)
 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:64)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
 at 
 org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
 at 
 org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
 at java.lang.Thread.run(Thread.java:745)
 {code}
 These particular cases are almost certainly problems with the client driver, 
 client machine, client process, however after the fact this particular 
 exception is practically impossible to debug because there is no indication 
 in the underlying JVM/netty exception of who the peer was. I should note we 
 have lots of different types of applications running against the cluster so 
 it is very hard to correlate these to anything



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137885#comment-14137885
 ] 

Benedict commented on CASSANDRA-7546:
-

It's hard to say for certain, but glancing at CHANGES.txt, it looks like 
2.1.1-HEAD is same ballpark as safe to run as 2.1.0. There are a lot of changes 
merged, but mostly for tools like cqlsh, and the things in the core application 
are pretty minor. I don't officially endorse it though, since we only just 
shipped to 2.1.0, and haven't had much time to QA 2.1.1 

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 2.1.1

 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 
 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 
 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, hint_spikes.png, 
 suggestion1.txt, suggestion1_21.txt, young_gen_gc.png


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137887#comment-14137887
 ] 

graham sanderson commented on CASSANDRA-7546:
-

Hmm - i'll definitely have to try again - it didn't respond to SIGHUP or non -F 
jstack, and isn't responding to ctrl+C, so maybe close to OOM

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 2.1.1

 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 
 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 
 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, hint_spikes.png, 
 suggestion1.txt, suggestion1_21.txt, young_gen_gc.png


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137888#comment-14137888
 ] 

graham sanderson commented on CASSANDRA-7546:
-

Yeah, this is only a cluster for my testing this... I just don't want a massive 
breakage that stops it working completely! I'll install head

 AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
 -

 Key: CASSANDRA-7546
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: graham sanderson
Assignee: graham sanderson
 Fix For: 2.1.1

 Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_3.txt, 
 7546.20_4.txt, 7546.20_5.txt, 7546.20_6.txt, 7546.20_7.txt, 7546.20_7b.txt, 
 7546.20_alt.txt, 7546.20_async.txt, 7546.21_v1.txt, hint_spikes.png, 
 suggestion1.txt, suggestion1_21.txt, young_gen_gc.png


 In order to preserve atomicity, this code attempts to read, clone/update, 
 then CAS the state of the partition.
 Under heavy contention for updating a single partition this can cause some 
 fairly staggering memory growth (the more cores on your machine the worst it 
 gets).
 Whilst many usage patterns don't do highly concurrent updates to the same 
 partition, hinting today, does, and in this case wild (order(s) of magnitude 
 more than expected) memory allocation rates can be seen (especially when the 
 updates being hinted are small updates to different partitions which can 
 happen very fast on their own) - see CASSANDRA-7545
 It would be best to eliminate/reduce/limit the spinning memory allocation 
 whilst not slowing down the very common un-contended case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/3] git commit: Add CASSANDRA-7450 to CHANGES

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.1 125b4642d - 7c34d8bb7
  refs/heads/trunk 9553dae93 - 134c2e969


Add CASSANDRA-7450 to CHANGES


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c34d8bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c34d8bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c34d8bb

Branch: refs/heads/cassandra-2.1
Commit: 7c34d8bb72672747317963f96c1a22c74d955d84
Parents: 125b464
Author: Yuki Morishita yu...@apache.org
Authored: Wed Sep 17 15:14:41 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed Sep 17 15:14:41 2014 -0500

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c34d8bb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca578f3..cc8c691 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -40,6 +40,7 @@
  * Optimise NativeCell comparisons (CASSANDRA-6755)
  * Configurable client timeout for cqlsh (CASSANDRA-7516)
  * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
+ * Make repair -pr work with -local (CASSANDRA-7450)
 Merged from 2.0:
  * Make CQLSSTableWriter sync within partitions (CASSANDRA-7360)
  * Potentially use non-local replicas in CqlConfigHelper (CASSANDRA-7906)

[2/3] git commit: Add CASSANDRA-7450 to CHANGES

Add CASSANDRA-7450 to CHANGES


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7c34d8bb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7c34d8bb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7c34d8bb

Branch: refs/heads/trunk
Commit: 7c34d8bb72672747317963f96c1a22c74d955d84
Parents: 125b464
Author: Yuki Morishita yu...@apache.org
Authored: Wed Sep 17 15:14:41 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed Sep 17 15:14:41 2014 -0500

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7c34d8bb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca578f3..cc8c691 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -40,6 +40,7 @@
  * Optimise NativeCell comparisons (CASSANDRA-6755)
  * Configurable client timeout for cqlsh (CASSANDRA-7516)
  * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111)
+ * Make repair -pr work with -local (CASSANDRA-7450)
 Merged from 2.0:
  * Make CQLSSTableWriter sync within partitions (CASSANDRA-7360)
  * Potentially use non-local replicas in CqlConfigHelper (CASSANDRA-7906)

[3/3] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/134c2e96
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/134c2e96
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/134c2e96

Branch: refs/heads/trunk
Commit: 134c2e96929ce82f347815d72a476184332c6016
Parents: 9553dae 7c34d8b
Author: Yuki Morishita yu...@apache.org
Authored: Wed Sep 17 15:14:46 2014 -0500
Committer: Yuki Morishita yu...@apache.org
Committed: Wed Sep 17 15:14:46 2014 -0500

--
 CHANGES.txt | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/134c2e96/CHANGES.txt
--

[jira] [Commented] (CASSANDRA-7941) Fix bin/cassandra cassandra.logdir option in debian package

2014-09-17 Thread Michael Shuler (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137897#comment-14137897
 ] 

Michael Shuler commented on CASSANDRA-7941:
---

[~jared.b...@bolderthinking.com] we're working on that  :)

 Fix bin/cassandra cassandra.logdir option in debian package
 ---

 Key: CASSANDRA-7941
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7941
 Project: Cassandra
  Issue Type: Bug
  Components: Packaging
Reporter: Michael Shuler
Assignee: Michael Shuler
 Fix For: 2.1.1

 Attachments: 
 0001-Fix-bin-cassandra-cassandra.logdir-option-in-debian-.patch


 Cassandra writes logs to $CASSANDRA_HOME/logs by default, and the debian 
 package needs to write to /var/log/cassandra.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7935) Make assassinate a first class command


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-7935:

Attachment: 7935.txt

Patch emits a deprecation warning when unsafeAssassinate is called, but passes 
through to assassinate anyway, adds a check on the heartbeat, and adds the 
nodetool command to assassinate.  The check on the heartbeat being stable is 
arguably something we should backport, but my impetus for adding it here was to 
make sure the node is completely and truly dead to prevent operator error since 
assassinate is more accessible now. (Without the hearbeat check you can 
actually assassinate a live node, so I justify removing the unsafe moniker with 
this check :))

 Make assassinate a first class command
 --

 Key: CASSANDRA-7935
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7935
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Brandon Williams
Assignee: Brandon Williams
 Fix For: 3.0

 Attachments: 7935.txt


 Assassinate has proven itself as the top tier in the hierarchy of node 
 removal.  It's time we removed the unsafe moniker and made it a nodetool 
 command so people don't have to mess with JMX themselves when they need it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7833) Collection types validation is incomplete


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137939#comment-14137939
 ] 

Tyler Hobbs commented on CASSANDRA-7833:


+1, but in 2.1 you should also make the same change to 
{{validateForNativeProtocol()}} in the serializer classes.

 Collection types validation is incomplete
 -

 Key: CASSANDRA-7833
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7833
 Project: Cassandra
  Issue Type: Bug
Reporter: Sylvain Lebresne
Assignee: Sylvain Lebresne
Priority: Minor
 Fix For: 2.0.11

 Attachments: 7833.txt


 The collection types will complain if a value has less values than 
 advertised, or if some of those values don't validate, but it does check that 
 there is no remaining bytes after the collection. One consequence is that if 
 you prepare {{INSERT INTO t(k, s) VALUES (0, ?)}} where {{s}} is a set, and 
 you pass a map value (with the same type for keys and values than the set), 
 then no error will be issued.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7940) Gossip only node removal can race with FD.convict


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137941#comment-14137941
 ] 

Tyler Hobbs commented on CASSANDRA-7940:


+1

 Gossip only node removal can race with FD.convict
 -

 Key: CASSANDRA-7940
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7940
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Reporter: Brandon Williams
Assignee: Brandon Williams
Priority: Minor
 Fix For: 2.0.11, 2.1.1

 Attachments: 7940.txt


 {noformat}
 INFO  00:04:48 FatClient /10.208.8.63 has been silent for 3ms, removing 
 from gossip
 INFO  00:04:49 Handshaking version with /10.208.8.63
 ERROR 00:04:53 Exception in thread Thread[GossipStage:1,5,main]
 java.lang.NullPointerException: null
 at org.apache.cassandra.gms.Gossiper.convict(Gossiper.java:301) 
 ~[main/:na]
 at 
 org.apache.cassandra.gms.FailureDetector.forceConviction(FailureDetector.java:251)
  ~[main/:na]
 at 
 org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:37)
  ~[main/:na]
 at 
 org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) 
 ~[main/:na]
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
  ~[na:1.7.0_60]
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
  ~[na:1.7.0_60]
 at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_60]
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (CASSANDRA-7935) Make assassinate a first class command


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tyler Hobbs updated CASSANDRA-7935:
---
Labels: jmx  (was: )

 Make assassinate a first class command
 --

 Key: CASSANDRA-7935
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7935
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Brandon Williams
Assignee: Brandon Williams
  Labels: jmx
 Fix For: 3.0

 Attachments: 7935.txt


 Assassinate has proven itself as the top tier in the hierarchy of node 
 removal.  It's time we removed the unsafe moniker and made it a nodetool 
 command so people don't have to mess with JMX themselves when they need it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7935) Make assassinate a first class command


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137959#comment-14137959
 ] 

Tyler Hobbs commented on CASSANDRA-7935:


Overall this looks good, I just have a couple of usability improvements:
* Instead of this error message: {{Endpoint  + endpoint +  heartbeat changed 
while trying to assassinate it}}, just say that the node appears to be alive 
and gossiping. Users that aren't familiar with gossip probably don't know what 
that means.
* Instead of this description: {{Obliterate an endpoint from the ring without 
re-replicating any data}}, describe when to use it and be a little more 
precise about what it does.

 Make assassinate a first class command
 --

 Key: CASSANDRA-7935
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7935
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Brandon Williams
Assignee: Brandon Williams
  Labels: jmx
 Fix For: 3.0

 Attachments: 7935.txt


 Assassinate has proven itself as the top tier in the hierarchy of node 
 removal.  It's time we removed the unsafe moniker and made it a nodetool 
 command so people don't have to mess with JMX themselves when they need it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

git commit: Support IN clause for any clustering column

2014-09-17 Thread tylerhobbs

Repository: cassandra
Updated Branches:
  refs/heads/trunk 87ba0dc1b - 9553dae93


Support IN clause for any clustering column

Patch by Benjamin Lerer; reviewed by Tyler Hobbs for CASSANDRA-4762


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9553dae9
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9553dae9
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9553dae9

Branch: refs/heads/trunk
Commit: 9553dae93ba328f3538b213051d180c21717c158
Parents: 87ba0dc
Author: blerer b_le...@hotmail.com
Authored: Wed Sep 17 13:54:45 2014 -0500
Committer: Tyler Hobbs ty...@datastax.com
Committed: Wed Sep 17 13:54:45 2014 -0500

--
 CHANGES.txt |   1 +
 .../cql3/statements/SelectStatement.java|  90 +++-
 .../db/composites/CompositesBuilder.java| 219 +++
 .../cql3/SingleColumnRelationTest.java  | 119 ++
 4 files changed, 366 insertions(+), 63 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9553dae9/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ca192a7..a11de41 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 3.0
+ * Support IN clause on any clustering column (CASSANDRA-4762)
  * Improve compaction logging (CASSANDRA-7818)
  * Remove YamlFileNetworkTopologySnitch (CASSANDRA-7917)
  * Support Java source code for user-defined functions (CASSANDRA-7562)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9553dae9/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
index ad66232..e135c0a 100644
--- a/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/SelectStatement.java
@@ -31,6 +31,7 @@ import org.github.jamm.MemoryMeter;
 import org.apache.cassandra.auth.Permission;
 import org.apache.cassandra.cql3.*;
 import org.apache.cassandra.db.composites.*;
+import org.apache.cassandra.db.composites.Composite.EOC;
 import org.apache.cassandra.transport.messages.ResultMessage;
 import org.apache.cassandra.config.CFMetaData;
 import org.apache.cassandra.config.ColumnDefinition;
@@ -694,7 +695,7 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 // we always do a slice for CQL3 tables, so it's ok to ignore them here
 assert !isColumnRange();
 
-CBuilder builder = cfm.comparator.prefixBuilder();
+CompositesBuilder builder = new 
CompositesBuilder(cfm.comparator.prefixBuilder(), cfm.comparator);
 IteratorColumnDefinition idIter = cfm.clusteringColumns().iterator();
 for (Restriction r : columnRestrictions)
 {
@@ -702,36 +703,20 @@ public class SelectStatement implements CQLStatement, 
MeasurableForPreparedCache
 assert r != null  !r.isSlice();
 
 ListByteBuffer values = r.values(options);
-if (values.size() == 1)
-{
-ByteBuffer val = values.get(0);
-if (val == null)
-throw new InvalidRequestException(String.format(Invalid 
null value for clustering key part %s, def.name));
-builder.add(val);
-}
-else
-{
-// We have a IN, which we only support for the last column.
-// If compact, just add all values and we're done. Otherwise,
-// for each value of the IN, creates all the columns 
corresponding to the selection.
-if (values.isEmpty())
-return null;
-SortedSetCellName columns = new 
TreeSetCellName(cfm.comparator);
-IteratorByteBuffer iter = values.iterator();
-while (iter.hasNext())
-{
-ByteBuffer val = iter.next();
-if (val == null)
-throw new 
InvalidRequestException(String.format(Invalid null value for clustering key 
part %s, def.name));
 
-Composite prefix = builder.buildWith(val);
-columns.addAll(addSelectedColumns(prefix));
-}
-return columns;
-}
+if (values.isEmpty())
+return null;
+
+builder.addEachElementToAll(values);
+if (builder.containsNull())
+throw new InvalidRequestException(String.format(Invalid null 
value for clustering key part %s,
+

[jira] [Created] (CASSANDRA-7964) cassandra-stress over schema should support multiple simultaneous inserts over the same seed

Benedict created CASSANDRA-7964:
---

 Summary: cassandra-stress over schema should support multiple 
simultaneous inserts over the same seed
 Key: CASSANDRA-7964
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7964
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Benedict
Assignee: Benedict
Priority: Minor


This constraint makes testing contention essentially impossible



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7935) Make assassinate a first class command


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137981#comment-14137981
 ] 

Brandon Williams commented on CASSANDRA-7935:
-

I kind of like having the specificity between generation/heartbeat so it if 
fails, we can know why.  For the description, I'm open to your bikeshed. :)

 Make assassinate a first class command
 --

 Key: CASSANDRA-7935
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7935
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Brandon Williams
Assignee: Brandon Williams
  Labels: jmx
 Fix For: 3.0

 Attachments: 7935.txt


 Assassinate has proven itself as the top tier in the hierarchy of node 
 removal.  It's time we removed the unsafe moniker and made it a nodetool 
 command so people don't have to mess with JMX themselves when they need it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (CASSANDRA-7935) Make assassinate a first class command


[ 
https://issues.apache.org/jira/browse/CASSANDRA-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14137990#comment-14137990
 ] 

Tyler Hobbs commented on CASSANDRA-7935:


Okay, you can keep something about the heartbeat in there, just explain that it 
appears that the node is alive and gossiping.

 Make assassinate a first class command
 --

 Key: CASSANDRA-7935
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7935
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Brandon Williams
Assignee: Brandon Williams
  Labels: jmx
 Fix For: 3.0

 Attachments: 7935.txt


 Assassinate has proven itself as the top tier in the hierarchy of node 
 removal.  It's time we removed the unsafe moniker and made it a nodetool 
 command so people don't have to mess with JMX themselves when they need it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[1/6] git commit: Prevent NPE on FD conviction race

Repository: cassandra
Updated Branches:
  refs/heads/cassandra-2.0 f19b4cb80 - 05920cb38
  refs/heads/cassandra-2.1 7c34d8bb7 - 83d15631f
  refs/heads/trunk 134c2e969 - 47cda388f


Prevent NPE on FD conviction race

Patch by brandonwilliams, reviewed by thobbs for CASSANDRA-7940


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05920cb3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05920cb3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05920cb3

Branch: refs/heads/cassandra-2.0
Commit: 05920cb38bc8559704450965ce4bb326a917c86a
Parents: f19b4cb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Sep 17 14:06:10 2014 +
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Sep 17 14:06:10 2014 +

--
 src/java/org/apache/cassandra/gms/Gossiper.java | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/05920cb3/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 01299ea..4514da7 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -298,6 +298,8 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 public void convict(InetAddress endpoint, double phi)
 {
 EndpointState epState = endpointStateMap.get(endpoint);
+if (epState == null)
+return;
 if (epState.isAlive()  !isDeadState(epState))
 {
 markDead(endpoint, epState);

[3/6] git commit: Prevent NPE on FD conviction race

Prevent NPE on FD conviction race

Patch by brandonwilliams, reviewed by thobbs for CASSANDRA-7940


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05920cb3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05920cb3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05920cb3

Branch: refs/heads/trunk
Commit: 05920cb38bc8559704450965ce4bb326a917c86a
Parents: f19b4cb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Sep 17 14:06:10 2014 +
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Sep 17 14:06:10 2014 +

--
 src/java/org/apache/cassandra/gms/Gossiper.java | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/05920cb3/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 01299ea..4514da7 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -298,6 +298,8 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 public void convict(InetAddress endpoint, double phi)
 {
 EndpointState epState = endpointStateMap.get(endpoint);
+if (epState == null)
+return;
 if (epState.isAlive()  !isDeadState(epState))
 {
 markDead(endpoint, epState);

[2/6] git commit: Prevent NPE on FD conviction race

Prevent NPE on FD conviction race

Patch by brandonwilliams, reviewed by thobbs for CASSANDRA-7940


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05920cb3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05920cb3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05920cb3

Branch: refs/heads/cassandra-2.1
Commit: 05920cb38bc8559704450965ce4bb326a917c86a
Parents: f19b4cb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Sep 17 14:06:10 2014 +
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Sep 17 14:06:10 2014 +

--
 src/java/org/apache/cassandra/gms/Gossiper.java | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/05920cb3/src/java/org/apache/cassandra/gms/Gossiper.java
--
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java 
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 01299ea..4514da7 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -298,6 +298,8 @@ public class Gossiper implements 
IFailureDetectionEventListener, GossiperMBean
 public void convict(InetAddress endpoint, double phi)
 {
 EndpointState epState = endpointStateMap.get(endpoint);
+if (epState == null)
+return;
 if (epState.isAlive()  !isDeadState(epState))
 {
 markDead(endpoint, epState);

[4/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83d15631
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83d15631
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83d15631

Branch: refs/heads/trunk
Commit: 83d15631f09da9cdd8d57b422ee9842ad9e5ba74
Parents: 7c34d8b 05920cb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Sep 17 14:07:18 2014 +
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Sep 17 14:07:18 2014 +

--
 src/java/org/apache/cassandra/gms/Gossiper.java | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83d15631/src/java/org/apache/cassandra/gms/Gossiper.java
--

[5/6] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1

Merge branch 'cassandra-2.0' into cassandra-2.1


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/83d15631
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/83d15631
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/83d15631

Branch: refs/heads/cassandra-2.1
Commit: 83d15631f09da9cdd8d57b422ee9842ad9e5ba74
Parents: 7c34d8b 05920cb
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Sep 17 14:07:18 2014 +
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Sep 17 14:07:18 2014 +

--
 src/java/org/apache/cassandra/gms/Gossiper.java | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/83d15631/src/java/org/apache/cassandra/gms/Gossiper.java
--

[6/6] git commit: Merge branch 'cassandra-2.1' into trunk

Merge branch 'cassandra-2.1' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/47cda388
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/47cda388
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/47cda388

Branch: refs/heads/trunk
Commit: 47cda388f57531c3f9801ff22f8541ee64ce6bb5
Parents: 134c2e9 83d1563
Author: Brandon Williams brandonwilli...@apache.org
Authored: Wed Sep 17 14:07:25 2014 +
Committer: Brandon Williams brandonwilli...@apache.org
Committed: Wed Sep 17 14:07:25 2014 +

--
 src/java/org/apache/cassandra/gms/Gossiper.java | 2 ++
 1 file changed, 2 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/47cda388/src/java/org/apache/cassandra/gms/Gossiper.java
--

[jira] [Updated] (CASSANDRA-7801) A successful INSERT with CAS does not always store data in the DB after a DELETE

2014-09-17 Thread Mikhail Stepura (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-7801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Stepura updated CASSANDRA-7801:
---
Description: 
When I run a loop with CQL statements to DELETE, INSERT with CAS and then a GET.
The INSERT opertion is successful (Applied), but no data is stored in the 
database. I have checked the database manually after the test to verify that 
the DB is empty.
{code}
for (int i = 0; i  1; ++i)
{
try
{
t.del();
t.cas();
t.select();
}
catch (Exception e)
{
System.err.println(i= + i);
e.printStackTrace();
break;
}
}


myCluster = 
Cluster.builder().addContactPoint(localhost).withPort(12742).build();
mySession = myCluster.connect();

mySession.execute(CREATE KEYSPACE IF NOT EXISTS castest WITH 
REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };);
mySession.execute(CREATE TABLE IF NOT EXISTS castest.users (userid 
text PRIMARY KEY, name text));

myInsert = mySession.prepare(INSERT INTO castest.users (userid, name) 
values ('user1', 'calle') IF NOT EXISTS);
myDelete = mySession.prepare(DELETE FROM castest.users where 
userid='user1');
myGet = mySession.prepare(SELECT * FROM castest.users where 
userid='user1');
}
{code}

I can reproduce the fault with the attached program on a PC with windows 7.
You need a cassandra runing and you need to set the port in the program.



  was:
When I run a loop with CQL statements to DELETE, INSERT with CAS and then a GET.
The INSERT opertion is successful (Applied), but no data is stored in the 
database. I have checked the database manually after the test to verify that 
the DB is empty.

for (int i = 0; i  1; ++i)
{
try
{
t.del();
t.cas();
t.select();
}
catch (Exception e)
{
System.err.println(i= + i);
e.printStackTrace();
break;
}
}


myCluster = 
Cluster.builder().addContactPoint(localhost).withPort(12742).build();
mySession = myCluster.connect();

mySession.execute(CREATE KEYSPACE IF NOT EXISTS castest WITH 
REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };);
mySession.execute(CREATE TABLE IF NOT EXISTS castest.users (userid 
text PRIMARY KEY, name text));

myInsert = mySession.prepare(INSERT INTO castest.users (userid, name) 
values ('user1', 'calle') IF NOT EXISTS);
myDelete = mySession.prepare(DELETE FROM castest.users where 
userid='user1');
myGet = mySession.prepare(SELECT * FROM castest.users where 
userid='user1');
}


I can reproduce the fault with the attached program on a PC with windows 7.
You need a cassandra runing and you need to set the port in the program.




 A successful INSERT with CAS does not always store data in the DB after a 
 DELETE
 

 Key: CASSANDRA-7801
 URL: https://issues.apache.org/jira/browse/CASSANDRA-7801
 Project: Cassandra
  Issue Type: Bug
  Components: Core
 Environment: PC with Windows 7 and on Linux installation.
 Have seen the fault on Cassandra 2.0.9 and Cassandra 2.1.0-rc5 
Reporter: Martin Fransson
Assignee: Sylvain Lebresne
 Fix For: 2.0.11

 Attachments: 7801.txt, cas.zip


 When I run a loop with CQL statements to DELETE, INSERT with CAS and then a 
 GET.
 The INSERT opertion is successful (Applied), but no data is stored in the 
 database. I have checked the database manually after the test to verify that 
 the DB is empty.
 {code}
 for (int i = 0; i  1; ++i)
 {
 try
 {
 t.del();
 t.cas();
 t.select();
 }
 catch (Exception e)
 {
 System.err.println(i= + i);
 e.printStackTrace();
 break;
 }
 }
 myCluster = 
 Cluster.builder().addContactPoint(localhost).withPort(12742).build();
 mySession = myCluster.connect();
 mySession.execute(CREATE KEYSPACE IF NOT EXISTS castest WITH 
 REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 1 };);
 mySession.execute(CREATE TABLE IF NOT EXISTS castest.users (userid 
 text PRIMARY KEY, name text));
 myInsert = mySession.prepare(INSERT INTO castest.users (userid, 
 name) values ('user1', 'calle') IF NOT EXISTS);
 myDelete = mySession.prepare(DELETE FROM castest.users where 
 userid='user1');

[jira] [Updated] (CASSANDRA-7935) Make assassinate a first class command