[jira] [Commented] (CASSANDRA-6663) Connecting to a Raspberry PI Cassandra Cluster crashes the node being connected to

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898877#comment-13898877
 ] 

Sylvain Lebresne commented on CASSANDRA-6663:
-

bq. https://github.com/xerial/snappy-java/issues/37

So let me sum this up: when executing the OPTIONS message, which asks for which 
compression methods are available, Snappy segfault on raspberry PI and we we 
can't catch that (since it segfault the JVM). So really, the "right" solution 
is probably to help on that snappy-java tracking the source of the problem down 
so it gets fixed there.

But in meantime, if there was a simple and reliable way to detect that "we are 
on raspberry PI", we could probably just mark Snappy as unavailable on 
raspberry without actually trying to initialize it. But for that, someone that 
does have a raspberry PI will have to step up with a patch because I don't 
personally have one.

Lastly, the python driver could probably work around that by not sending the 
OPTIONS message if the user hasn't asked for compression in the first place 
(not saying it's ideal, but currently there isn't much reason to send OPTIONS 
unless for knowing which compressions are available, so it could be a 
relatively simple work-around until the snappy-java library ticket is fixed).

> Connecting to a Raspberry PI Cassandra Cluster crashes the node being 
> connected to
> --
>
> Key: CASSANDRA-6663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6663
> Project: Cassandra
>  Issue Type: Bug
>  Components: Drivers (now out of tree)
> Environment: 4x node Raspberry PI cluster
> Macbook running Idle 2.7
>Reporter: ian mccrae
> Attachments: Python Client Log.txt, hs_err_pid6327.log
>
>
> I have a working 4x node Raspberry Pi cluster and
> # DevCenter happily connects to this (...which has an option to turn Snappy 
> compression off)
> # ...however the Python Driver fails to connect and crashes the node being 
> connected to with the errors in the error-log below.
> There appears to be a problem with Snappy compression (not supported on the 
> Raspberry Pi).  So I also tried "compression = None" with the same result.
> How might I fix this?
> *Python Code*
> {noformat}
> >>> from cassandra.cluster import Cluster
> >>> cluster = Cluster(['192.168.200.151'], compression = None)
> >>> session = cluster.connect()
> {noformat}
> *Error Log*
> {noformat}
> Traceback (most recent call last):
>   File "", line 1, in 
> session = cluster.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 471, in connect
> self.control_connection.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1351, in connect
> self._set_new_connection(self._reconnect_internal())
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1386, in _reconnect_internal
> raise NoHostAvailable("Unable to connect to any servers", errors)
> NoHostAvailable: ('Unable to connect to any servers', {'192.168.200.151': 
> ConnectionShutdown('Connection to 192.168.200.151 is closed',))
> {noformat}
> *A Dump of the "cluster" class attributes*
> {noformat}
> >>> pprint(vars(cluster))
> {'_core_connections_per_host': {0: 2, 1: 1},
>  '_is_setup': True,
>  '_is_shutdown': True,
>  '_listener_lock': ,
>  '_listeners': set([]),
>  '_lock': <_RLock owner=None count=0>,
>  '_max_connections_per_host': {0: 8, 1: 2},
>  '_max_requests_per_connection': {0: 100, 1: 100},
>  '_min_requests_per_connection': {0: 5, 1: 5},
>  '_prepared_statements': ,
>  'compression': None,
>  'contact_points': ['192.168.200.151'],
>  'control_connection':  0x106168cd0>,
>  'control_connection_timeout': 2.0,
>  'cql_version': None,
>  'executor':  0x106148410>,
>  'load_balancing_policy':  0x104adae50>,
>  'max_schema_agreement_wait': 10,
>  'metadata': ,
>  'metrics_enabled': False,
>  'port': 9042,
>  'scheduler': ,
>  'sessions': <_weakrefset.WeakSet object at 0x106148750>,
>  'sockopts': None,
>  'ssl_options': None}
> >>>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6663) Connecting to a Raspberry PI Cassandra Cluster crashes the node being connected to

[Mikhail Stepura (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898812#comment-13898812
 ] 

Mikhail Stepura commented on CASSANDRA-6663:


https://github.com/xerial/snappy-java/issues/37

> Connecting to a Raspberry PI Cassandra Cluster crashes the node being 
> connected to
> --
>
> Key: CASSANDRA-6663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6663
> Project: Cassandra
>  Issue Type: Bug
>  Components: Drivers (now out of tree)
> Environment: 4x node Raspberry PI cluster
> Macbook running Idle 2.7
>Reporter: ian mccrae
> Attachments: Python Client Log.txt, hs_err_pid6327.log
>
>
> I have a working 4x node Raspberry Pi cluster and
> # DevCenter happily connects to this (...which has an option to turn Snappy 
> compression off)
> # ...however the Python Driver fails to connect and crashes the node being 
> connected to with the errors in the error-log below.
> There appears to be a problem with Snappy compression (not supported on the 
> Raspberry Pi).  So I also tried "compression = None" with the same result.
> How might I fix this?
> *Python Code*
> {noformat}
> >>> from cassandra.cluster import Cluster
> >>> cluster = Cluster(['192.168.200.151'], compression = None)
> >>> session = cluster.connect()
> {noformat}
> *Error Log*
> {noformat}
> Traceback (most recent call last):
>   File "", line 1, in 
> session = cluster.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 471, in connect
> self.control_connection.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1351, in connect
> self._set_new_connection(self._reconnect_internal())
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1386, in _reconnect_internal
> raise NoHostAvailable("Unable to connect to any servers", errors)
> NoHostAvailable: ('Unable to connect to any servers', {'192.168.200.151': 
> ConnectionShutdown('Connection to 192.168.200.151 is closed',))
> {noformat}
> *A Dump of the "cluster" class attributes*
> {noformat}
> >>> pprint(vars(cluster))
> {'_core_connections_per_host': {0: 2, 1: 1},
>  '_is_setup': True,
>  '_is_shutdown': True,
>  '_listener_lock': ,
>  '_listeners': set([]),
>  '_lock': <_RLock owner=None count=0>,
>  '_max_connections_per_host': {0: 8, 1: 2},
>  '_max_requests_per_connection': {0: 100, 1: 100},
>  '_min_requests_per_connection': {0: 5, 1: 5},
>  '_prepared_statements': ,
>  'compression': None,
>  'contact_points': ['192.168.200.151'],
>  'control_connection':  0x106168cd0>,
>  'control_connection_timeout': 2.0,
>  'cql_version': None,
>  'executor':  0x106148410>,
>  'load_balancing_policy':  0x104adae50>,
>  'max_schema_agreement_wait': 10,
>  'metadata': ,
>  'metrics_enabled': False,
>  'port': 9042,
>  'scheduler': ,
>  'sessions': <_weakrefset.WeakSet object at 0x106148750>,
>  'sockopts': None,
>  'ssl_options': None}
> >>>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Comment Edited] (CASSANDRA-6663) Connecting to a Raspberry PI Cassandra Cluster crashes the node being connected to

[Mikhail Stepura (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898747#comment-13898747
 ] 

Mikhail Stepura edited comment on CASSANDRA-6663 at 2/12/14 3:53 AM:
-

Yep, I've added some logging to the SnappyCompressor and run the python code 
from the above. The {{cluster.connect()}} (i.e. the very 1st {{OptionsMessage}} 
) triggers the initialization of {{SnappyCompressor.instance}}


was (Author: mishail):
Yep, I've added some logging to the SnappyCompressor and run the python code 
from the above. The {{cluster.connect()}} triggers the initialization of 
{{SnappyCompressor.instance}}

> Connecting to a Raspberry PI Cassandra Cluster crashes the node being 
> connected to
> --
>
> Key: CASSANDRA-6663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6663
> Project: Cassandra
>  Issue Type: Bug
>  Components: Drivers (now out of tree)
> Environment: 4x node Raspberry PI cluster
> Macbook running Idle 2.7
>Reporter: ian mccrae
> Attachments: Python Client Log.txt, hs_err_pid6327.log
>
>
> I have a working 4x node Raspberry Pi cluster and
> # DevCenter happily connects to this (...which has an option to turn Snappy 
> compression off)
> # ...however the Python Driver fails to connect and crashes the node being 
> connected to with the errors in the error-log below.
> There appears to be a problem with Snappy compression (not supported on the 
> Raspberry Pi).  So I also tried "compression = None" with the same result.
> How might I fix this?
> *Python Code*
> {noformat}
> >>> from cassandra.cluster import Cluster
> >>> cluster = Cluster(['192.168.200.151'], compression = None)
> >>> session = cluster.connect()
> {noformat}
> *Error Log*
> {noformat}
> Traceback (most recent call last):
>   File "", line 1, in 
> session = cluster.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 471, in connect
> self.control_connection.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1351, in connect
> self._set_new_connection(self._reconnect_internal())
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1386, in _reconnect_internal
> raise NoHostAvailable("Unable to connect to any servers", errors)
> NoHostAvailable: ('Unable to connect to any servers', {'192.168.200.151': 
> ConnectionShutdown('Connection to 192.168.200.151 is closed',))
> {noformat}
> *A Dump of the "cluster" class attributes*
> {noformat}
> >>> pprint(vars(cluster))
> {'_core_connections_per_host': {0: 2, 1: 1},
>  '_is_setup': True,
>  '_is_shutdown': True,
>  '_listener_lock': ,
>  '_listeners': set([]),
>  '_lock': <_RLock owner=None count=0>,
>  '_max_connections_per_host': {0: 8, 1: 2},
>  '_max_requests_per_connection': {0: 100, 1: 100},
>  '_min_requests_per_connection': {0: 5, 1: 5},
>  '_prepared_statements': ,
>  'compression': None,
>  'contact_points': ['192.168.200.151'],
>  'control_connection':  0x106168cd0>,
>  'control_connection_timeout': 2.0,
>  'cql_version': None,
>  'executor':  0x106148410>,
>  'load_balancing_policy':  0x104adae50>,
>  'max_schema_agreement_wait': 10,
>  'metadata': ,
>  'metrics_enabled': False,
>  'port': 9042,
>  'scheduler': ,
>  'sessions': <_weakrefset.WeakSet object at 0x106148750>,
>  'sockopts': None,
>  'ssl_options': None}
> >>>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6663) Connecting to a Raspberry PI Cassandra Cluster crashes the node being connected to

[Mikhail Stepura (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898747#comment-13898747
 ] 

Mikhail Stepura commented on CASSANDRA-6663:


Yep, I've added some logging to the SnappyCompressor and run the python code 
from the above. The {{cluster.connect()}} triggers the initialization of 
{{SnappyCompressor.instance}}

> Connecting to a Raspberry PI Cassandra Cluster crashes the node being 
> connected to
> --
>
> Key: CASSANDRA-6663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6663
> Project: Cassandra
>  Issue Type: Bug
>  Components: Drivers (now out of tree)
> Environment: 4x node Raspberry PI cluster
> Macbook running Idle 2.7
>Reporter: ian mccrae
> Attachments: Python Client Log.txt, hs_err_pid6327.log
>
>
> I have a working 4x node Raspberry Pi cluster and
> # DevCenter happily connects to this (...which has an option to turn Snappy 
> compression off)
> # ...however the Python Driver fails to connect and crashes the node being 
> connected to with the errors in the error-log below.
> There appears to be a problem with Snappy compression (not supported on the 
> Raspberry Pi).  So I also tried "compression = None" with the same result.
> How might I fix this?
> *Python Code*
> {noformat}
> >>> from cassandra.cluster import Cluster
> >>> cluster = Cluster(['192.168.200.151'], compression = None)
> >>> session = cluster.connect()
> {noformat}
> *Error Log*
> {noformat}
> Traceback (most recent call last):
>   File "", line 1, in 
> session = cluster.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 471, in connect
> self.control_connection.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1351, in connect
> self._set_new_connection(self._reconnect_internal())
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1386, in _reconnect_internal
> raise NoHostAvailable("Unable to connect to any servers", errors)
> NoHostAvailable: ('Unable to connect to any servers', {'192.168.200.151': 
> ConnectionShutdown('Connection to 192.168.200.151 is closed',))
> {noformat}
> *A Dump of the "cluster" class attributes*
> {noformat}
> >>> pprint(vars(cluster))
> {'_core_connections_per_host': {0: 2, 1: 1},
>  '_is_setup': True,
>  '_is_shutdown': True,
>  '_listener_lock': ,
>  '_listeners': set([]),
>  '_lock': <_RLock owner=None count=0>,
>  '_max_connections_per_host': {0: 8, 1: 2},
>  '_max_requests_per_connection': {0: 100, 1: 100},
>  '_min_requests_per_connection': {0: 5, 1: 5},
>  '_prepared_statements': ,
>  'compression': None,
>  'contact_points': ['192.168.200.151'],
>  'control_connection':  0x106168cd0>,
>  'control_connection_timeout': 2.0,
>  'cql_version': None,
>  'executor':  0x106148410>,
>  'load_balancing_policy':  0x104adae50>,
>  'max_schema_agreement_wait': 10,
>  'metadata': ,
>  'metrics_enabled': False,
>  'port': 9042,
>  'scheduler': ,
>  'sessions': <_weakrefset.WeakSet object at 0x106148750>,
>  'sockopts': None,
>  'ssl_options': None}
> >>>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-5962) Support trigger parametrization

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-5962:
-

Fix Version/s: (was: 2.1)
 Assignee: (was: Aleksey Yeschenko)

> Support trigger parametrization
> ---
>
> Key: CASSANDRA-5962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5962
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Priority: Minor
>  Labels: cql3, triggers
>
> We don't have a convenient way to parametrize triggers, which limits their 
> reusability and usability in general. For any configuration you have to rely 
> on external config files.
> We already have [trigger_options map] column in 
> system.schema_triggers, all we need is to add the right syntax to CQL3 
> (CREATE TRIGGER foo ON bar USING class WITH options = {..}) and modify 
> ITrigger to support it.
> Setting fixver to 2.1, but might move to 2.0.x later.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

git commit: Replace EmptyColumns with updated ArrayBackedSortedColumns

[aleksey]

Updated Branches:
  refs/heads/trunk 67101c29c -> 5cf381f57


Replace EmptyColumns with updated ArrayBackedSortedColumns

patch by Aleksey Yeschenko; reviewed by Jonathan Ellis for
CASSANDRA-6690


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5cf381f5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5cf381f5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5cf381f5

Branch: refs/heads/trunk
Commit: 5cf381f57da41e986e7434fb7a46531c42c2cdd3
Parents: 67101c2
Author: Aleksey Yeschenko 
Authored: Wed Feb 12 05:22:29 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Wed Feb 12 05:22:29 2014 +0300

--
 CHANGES.txt |   4 +-
 .../db/AbstractThreadUnsafeSortedColumns.java   |  70 
 .../cassandra/db/ArrayBackedSortedColumns.java  |  50 -
 .../org/apache/cassandra/db/EmptyColumns.java   | 112 ---
 .../org/apache/cassandra/db/SystemKeyspace.java |   2 +-
 .../db/columniterator/IndexedSliceReader.java   |   2 +-
 .../db/columniterator/SimpleSliceReader.java|   2 +-
 .../db/compaction/LazilyCompactedRow.java   |   2 +-
 .../io/sstable/SSTableIdentityIterator.java |   2 +-
 .../apache/cassandra/service/StorageProxy.java  |   4 +-
 .../cassandra/service/pager/QueryPagers.java|   2 +-
 .../apache/cassandra/service/paxos/Commit.java  |   4 +-
 12 files changed, 56 insertions(+), 200 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5cf381f5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 1025510..3831b38 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -25,8 +25,8 @@
  * CF id is changed to be non-deterministic. Data dir/key cache are created
uniquely for CF id (CASSANDRA-5202)
  * New counters implementation (CASSANDRA-6504)
- * Replace UnsortedColumns and TreeMapBackedSortedColumns with rewritten
-   ArrayBackedSortedColumns (CASSANDRA-6630, CASSANDRA-6662)
+ * Replace UnsortedColumns, EmptyColumns, TreeMapBackedSortedColumns with new
+   ArrayBackedSortedColumns (CASSANDRA-6630, CASSANDRA-6662, CASSANDRA-6690)
  * Add option to use row cache with a given amount of rows (CASSANDRA-5357)
  * Avoid repairing already repaired data (CASSANDRA-5351)
  * Reject counter updates with USING TTL/TIMESTAMP (CASSANDRA-6649)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/5cf381f5/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
--
diff --git 
a/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java 
b/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
deleted file mode 100644
index 36b051b..000
--- a/src/java/org/apache/cassandra/db/AbstractThreadUnsafeSortedColumns.java
+++ /dev/null
@@ -1,70 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements.  See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership.  The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License.  You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.db;
-
-import org.apache.cassandra.config.CFMetaData;
-
-public abstract class AbstractThreadUnsafeSortedColumns extends ColumnFamily
-{
-protected DeletionInfo deletionInfo;
-
-public AbstractThreadUnsafeSortedColumns(CFMetaData metadata)
-{
-this(metadata, DeletionInfo.live());
-}
-
-protected AbstractThreadUnsafeSortedColumns(CFMetaData metadata, 
DeletionInfo deletionInfo)
-{
-super(metadata);
-this.deletionInfo = deletionInfo;
-}
-
-public DeletionInfo deletionInfo()
-{
-return deletionInfo;
-}
-
-public void delete(DeletionTime delTime)
-{
-deletionInfo.add(delTime);
-}
-
-public void delete(DeletionInfo newInfo)
-{
-deletionInfo.add(newInfo);
-}
-
-protected void delete(RangeTombstone tombstone)
-{
-deletionInfo.add(tombstone, getComparator());
-}
-
-public void setDeletionInfo(DeletionInfo newInfo)
-{
-deletionInfo = newInfo;
-}
-
-/**
- * Purges any tombstones with a local 

[jira] [Commented] (CASSANDRA-6690) Replace EmptyColumns with updated ArrayBackedSortedColumns

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898681#comment-13898681
 ] 

Jonathan Ellis commented on CASSANDRA-6690:
---

+1

> Replace EmptyColumns with updated ArrayBackedSortedColumns
> --
>
> Key: CASSANDRA-6690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6690
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 2.1
>
> Attachments: 6690.txt
>
>
> Now that UnsortedColumns is gone (CASSANDRA-6630) and 
> TreeMapBackedSortedColumns is also gone (CASSANDRA-6662), we can also get rid 
> of EmptyColumns and, further, inline AbstractThreadUnsafeSortedColumns into 
> ABSC, leaving us with just two ColumnFamily implementations - 
> AtomicBTreeColumns for the memtable and ABSC for everything else.
> The modified ABSC in the patch starts with the empty static array, so in 
> terms of memory it's about the same as EmptyColumns used to be.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Assigned] (CASSANDRA-6475) Control nodetool history logging directory

[sankalp kohli (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sankalp kohli reassigned CASSANDRA-6475:


Assignee: sankalp kohli

> Control nodetool history logging directory
> --
>
> Key: CASSANDRA-6475
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Brian Nixon
>Assignee: sankalp kohli
>Priority: Trivial
>  Labels: lhf
>
> Nodetool history is logged to a directory based on the current user home. 
> This leads to splintering of history with more than one user and, in one 
> case, a failure to run any nodetool commands as the user did not have write 
> access to their home directory.
> Suggested fix is to make the base directory for the history logging (both 
> nodetool and cli) configurable. A way to disable the logging of these tools 
> would also help.
> Reference:
> https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables

[Pavel Yaskevich (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898665#comment-13898665
 ] 

Pavel Yaskevich commented on CASSANDRA-6689:


Thanks, [~jbellis], I'm already in the watcher list :)

> Partially Off Heap Memtables
> 
>
> Key: CASSANDRA-6689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 2.1
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables

[Jason Brown (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898661#comment-13898661
 ] 

Jason Brown commented on CASSANDRA-6689:


I can jump in if [~krummas] is busy.

> Partially Off Heap Memtables
> 
>
> Key: CASSANDRA-6689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 2.1
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6689) Partially Off Heap Memtables

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-6689:
--

   Reviewer: Marcus Eriksson
Component/s: Core

Assigning [~krummas] as reviewer so I don't become the bottleneck this time. :)

[~xedin] and [~jasobrown] might also be interested.

> Partially Off Heap Memtables
> 
>
> Key: CASSANDRA-6689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 2.1
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory

[sankalp kohli (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898653#comment-13898653
 ] 

sankalp kohli commented on CASSANDRA-6475:
--

I think we should fix cqlsh to not use ~ and instead use user.home variable. 

> Control nodetool history logging directory
> --
>
> Key: CASSANDRA-6475
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Brian Nixon
>Priority: Trivial
>  Labels: lhf
>
> Nodetool history is logged to a directory based on the current user home. 
> This leads to splintering of history with more than one user and, in one 
> case, a failure to run any nodetool commands as the user did not have write 
> access to their home directory.
> Suggested fix is to make the base directory for the history logging (both 
> nodetool and cli) configurable. A way to disable the logging of these tools 
> would also help.
> Reference:
> https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6690) Replace EmptyColumns with updated ArrayBackedSortedColumns

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-6690:
-

Attachment: 6690.txt

> Replace EmptyColumns with updated ArrayBackedSortedColumns
> --
>
> Key: CASSANDRA-6690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6690
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Minor
> Fix For: 2.1
>
> Attachments: 6690.txt
>
>
> Now that UnsortedColumns is gone (CASSANDRA-6630) and 
> TreeMapBackedSortedColumns is also gone (CASSANDRA-6662), we can also get rid 
> of EmptyColumns and, further, inline AbstractThreadUnsafeSortedColumns into 
> ABSC, leaving us with just two ColumnFamily implementations - 
> AtomicBTreeColumns for the memtable and ABSC for everything else.
> The modified ABSC in the patch starts with the empty static array, so in 
> terms of memory it's about the same as EmptyColumns used to be.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6663) Connecting to a Raspberry PI Cassandra Cluster crashes the node being connected to

[Mikhail Stepura (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898617#comment-13898617
 ] 

Mikhail Stepura commented on CASSANDRA-6663:


I might be wrong, but still.

* DevCenter uses Java Driver 1.0.x, and it initiates a connection with STARTUP 
message
* Python driver initiates connection with OPTIONS message. 

Probably the {{org.apache.cassandra.transport.messages.OptionsMessage}} code 
triggers the initialization of {{FrameCompressor.SnappyCompressor.instance}} 
which fails

> Connecting to a Raspberry PI Cassandra Cluster crashes the node being 
> connected to
> --
>
> Key: CASSANDRA-6663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6663
> Project: Cassandra
>  Issue Type: Bug
>  Components: Drivers (now out of tree)
> Environment: 4x node Raspberry PI cluster
> Macbook running Idle 2.7
>Reporter: ian mccrae
> Attachments: Python Client Log.txt, hs_err_pid6327.log
>
>
> I have a working 4x node Raspberry Pi cluster and
> # DevCenter happily connects to this (...which has an option to turn Snappy 
> compression off)
> # ...however the Python Driver fails to connect and crashes the node being 
> connected to with the errors in the error-log below.
> There appears to be a problem with Snappy compression (not supported on the 
> Raspberry Pi).  So I also tried "compression = None" with the same result.
> How might I fix this?
> *Python Code*
> {noformat}
> >>> from cassandra.cluster import Cluster
> >>> cluster = Cluster(['192.168.200.151'], compression = None)
> >>> session = cluster.connect()
> {noformat}
> *Error Log*
> {noformat}
> Traceback (most recent call last):
>   File "", line 1, in 
> session = cluster.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 471, in connect
> self.control_connection.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1351, in connect
> self._set_new_connection(self._reconnect_internal())
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1386, in _reconnect_internal
> raise NoHostAvailable("Unable to connect to any servers", errors)
> NoHostAvailable: ('Unable to connect to any servers', {'192.168.200.151': 
> ConnectionShutdown('Connection to 192.168.200.151 is closed',))
> {noformat}
> *A Dump of the "cluster" class attributes*
> {noformat}
> >>> pprint(vars(cluster))
> {'_core_connections_per_host': {0: 2, 1: 1},
>  '_is_setup': True,
>  '_is_shutdown': True,
>  '_listener_lock': ,
>  '_listeners': set([]),
>  '_lock': <_RLock owner=None count=0>,
>  '_max_connections_per_host': {0: 8, 1: 2},
>  '_max_requests_per_connection': {0: 100, 1: 100},
>  '_min_requests_per_connection': {0: 5, 1: 5},
>  '_prepared_statements': ,
>  'compression': None,
>  'contact_points': ['192.168.200.151'],
>  'control_connection':  0x106168cd0>,
>  'control_connection_timeout': 2.0,
>  'cql_version': None,
>  'executor':  0x106148410>,
>  'load_balancing_policy':  0x104adae50>,
>  'max_schema_agreement_wait': 10,
>  'metadata': ,
>  'metrics_enabled': False,
>  'port': 9042,
>  'scheduler': ,
>  'sessions': <_weakrefset.WeakSet object at 0x106148750>,
>  'sockopts': None,
>  'ssl_options': None}
> >>>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (CASSANDRA-6690) Replace EmptyColumns with updated ArrayBackedSortedColumns

[Aleksey Yeschenko (JIRA)]

Aleksey Yeschenko created CASSANDRA-6690:


 Summary: Replace EmptyColumns with updated ArrayBackedSortedColumns
 Key: CASSANDRA-6690
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6690
 Project: Cassandra
  Issue Type: Improvement
Reporter: Aleksey Yeschenko
Assignee: Aleksey Yeschenko
Priority: Minor
 Fix For: 2.1


Now that UnsortedColumns is gone (CASSANDRA-6630) and 
TreeMapBackedSortedColumns is also gone (CASSANDRA-6662), we can also get rid 
of EmptyColumns and, further, inline AbstractThreadUnsafeSortedColumns into 
ABSC, leaving us with just two ColumnFamily implementations - 
AtomicBTreeColumns for the memtable and ABSC for everything else.

The modified ABSC in the patch starts with the empty static array, so in terms 
of memory it's about the same as EmptyColumns used to be.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6157) Selectively Disable hinted handoff for a data center

[sankalp kohli (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

sankalp kohli updated CASSANDRA-6157:
-

Attachment: super-csv-2.1.0.jar
trunk-6157-v5.diff

I have done the changes based on Jonathan comments. I am using Super CSV 
library(http://supercsv.sourceforge.net). I am not sure what is the exact 
procedure of adding new jars. I have added the dependency in build.xml. Also I 
am attaching the jar in case it needs to go in lib. 

> Selectively Disable hinted handoff for a data center
> 
>
> Key: CASSANDRA-6157
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6157
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
>Reporter: sankalp kohli
>Assignee: sankalp kohli
>Priority: Minor
> Fix For: 2.0.6
>
> Attachments: super-csv-2.1.0.jar, trunk-6157-v2.diff, 
> trunk-6157-v3.diff, trunk-6157-v4.diff, trunk-6157-v5.diff, trunk-6157.txt
>
>
> Cassandra supports disabling the hints or reducing the window for hints. 
> It would be helpful to have a switch which stops hints to a down data center 
> but continue hints to other DCs.
> This is helpful during data center fail over as hints will put more 
> unnecessary pressure on the DC taking double traffic. Also since now 
> Cassandra is under reduced reduncany, we don't want to disable hints within 
> the DC. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6685) Nodes never bootstrap if schema is empty

[Edward Capriolo (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898598#comment-13898598
 ] 

Edward Capriolo commented on CASSANDRA-6685:


I am confused as to what was committed. No one mentioned a git hash or included 
a patch.

> Nodes never bootstrap if schema is empty
> 
>
> Key: CASSANDRA-6685
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6685
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Richard Low
>Assignee: Brandon Williams
> Fix For: 1.2.16
>
>
> Since 1.2.15, bootstrap never completes if the schema is empty. The 
> bootstrapping node endlessly prints:
> bq. {{INFO 12:37:44,863 JOINING: waiting for schema information to complete}}
> until you add something to the schema (i.e. create a keyspace).
> The problem looks to be caused by CASSANDRA-6648, where 
> MigrationManager.isReadForBootstrap() was changed to:
> bq. {{return Schema.instance.getVersion() != null && 
> !Schema.emptyVersion.equals(Schema.instance.getVersion());}}
> This is wrong since 
> {{Schema.emptyVersion.equals(Schema.instance.getVersion())}} is always true 
> if there is no schema.
> We need some different logic for determining when the schema is propagated.
> I haven't tested, but I expect this issue appears in 2.0.5 too.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6663) Connecting to a Raspberry PI Cassandra Cluster crashes the node being connected to

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898593#comment-13898593
 ] 

Tyler Hobbs commented on CASSANDRA-6663:


[~slebresne] I confirmed today that the python driver is not requesting 
compression on the connections, so it doesn't appear to be a driver problem.

> Connecting to a Raspberry PI Cassandra Cluster crashes the node being 
> connected to
> --
>
> Key: CASSANDRA-6663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6663
> Project: Cassandra
>  Issue Type: Bug
>  Components: Drivers (now out of tree)
> Environment: 4x node Raspberry PI cluster
> Macbook running Idle 2.7
>Reporter: ian mccrae
> Attachments: Python Client Log.txt, hs_err_pid6327.log
>
>
> I have a working 4x node Raspberry Pi cluster and
> # DevCenter happily connects to this (...which has an option to turn Snappy 
> compression off)
> # ...however the Python Driver fails to connect and crashes the node being 
> connected to with the errors in the error-log below.
> There appears to be a problem with Snappy compression (not supported on the 
> Raspberry Pi).  So I also tried "compression = None" with the same result.
> How might I fix this?
> *Python Code*
> {noformat}
> >>> from cassandra.cluster import Cluster
> >>> cluster = Cluster(['192.168.200.151'], compression = None)
> >>> session = cluster.connect()
> {noformat}
> *Error Log*
> {noformat}
> Traceback (most recent call last):
>   File "", line 1, in 
> session = cluster.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 471, in connect
> self.control_connection.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1351, in connect
> self._set_new_connection(self._reconnect_internal())
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1386, in _reconnect_internal
> raise NoHostAvailable("Unable to connect to any servers", errors)
> NoHostAvailable: ('Unable to connect to any servers', {'192.168.200.151': 
> ConnectionShutdown('Connection to 192.168.200.151 is closed',))
> {noformat}
> *A Dump of the "cluster" class attributes*
> {noformat}
> >>> pprint(vars(cluster))
> {'_core_connections_per_host': {0: 2, 1: 1},
>  '_is_setup': True,
>  '_is_shutdown': True,
>  '_listener_lock': ,
>  '_listeners': set([]),
>  '_lock': <_RLock owner=None count=0>,
>  '_max_connections_per_host': {0: 8, 1: 2},
>  '_max_requests_per_connection': {0: 100, 1: 100},
>  '_min_requests_per_connection': {0: 5, 1: 5},
>  '_prepared_statements': ,
>  'compression': None,
>  'contact_points': ['192.168.200.151'],
>  'control_connection':  0x106168cd0>,
>  'control_connection_timeout': 2.0,
>  'cql_version': None,
>  'executor':  0x106148410>,
>  'load_balancing_policy':  0x104adae50>,
>  'max_schema_agreement_wait': 10,
>  'metadata': ,
>  'metrics_enabled': False,
>  'port': 9042,
>  'scheduler': ,
>  'sessions': <_weakrefset.WeakSet object at 0x106148750>,
>  'sockopts': None,
>  'ssl_options': None}
> >>>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6689) Partially Off Heap Memtables

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898564#comment-13898564
 ] 

Benedict commented on CASSANDRA-6689:
-

I've uploaded a patch 
[here|https://github.com/belliottsmith/cassandra/tree/offheap.merge]

In the simplest terms this boils down to a new off heap allocator. However it 
is much more involved than that, as we need to now manage the life cycle of the 
memory in question.

I won't go into excessive detail here, because I have tried to do so as much as 
possible in the code itself. For full details see the comments in 
OffHeapCleaner, RefAction/Referrer, OffHeapAllocator, OffHeapRegion.

The basic thrust of the approach is that we guard accesses to the memory 
through the OpOrder synchronisation primitive introduced with the 
CASSANDRA-5549. However to prevent stalls and make management of the memory 
easier, we have some further stages to the lifecycle which are much more 
GC-like. A corollary is that we also effectively perform GC of the data in the 
memtable, so that space that is no longer needed as a result of overwritten 
records can be made available again without first flushing the memtable.


> Partially Off Heap Memtables
> 
>
> Key: CASSANDRA-6689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Benedict
>Assignee: Benedict
> Fix For: 2.1
>
>
> Move the contents of ByteBuffers off-heap for records written to a memtable.
> (See comments for details)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (CASSANDRA-6689) Partially Off Heap Memtables

[Benedict (JIRA)]

Benedict created CASSANDRA-6689:
---

 Summary: Partially Off Heap Memtables
 Key: CASSANDRA-6689
 URL: https://issues.apache.org/jira/browse/CASSANDRA-6689
 Project: Cassandra
  Issue Type: New Feature
Reporter: Benedict
Assignee: Benedict
 Fix For: 2.1


Move the contents of ByteBuffers off-heap for records written to a memtable.

(See comments for details)




--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[4/5] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

[brandonwilliams]

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/80cebec5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/80cebec5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/80cebec5

Branch: refs/heads/cassandra-2.0
Commit: 80cebec5d03c8dfb1afd416e138b65ccf5c791d9
Parents: c5f225b 00a8b1e
Author: Brandon Williams 
Authored: Tue Feb 11 17:14:50 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:14:50 2014 -0600

--
 CHANGES.txt | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/80cebec5/CHANGES.txt
--
diff --cc CHANGES.txt
index fd640cd,0674dde..f9b2032
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,4 +1,16 @@@
- 2.0.6
 -1.2.16
 + * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
 + * Fix EstimatedHistogram races (CASSANDRA-6682)
 + * Failure detector correctly converts initial value to nanos (CASSANDRA-6658)
 + * Add nodetool taketoken to relocate vnodes (CASSANDRA-4445)
 + * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
 + * Improve nodetool cfhistograms formatting (CASSANDRA-6360)
 + * Expose bulk loading progress over JMX (CASSANDRA-4757)
 + * Correctly handle null with IF conditions and TTL (CASSANDRA-6623)
 + * Account for range/row tombstones in tombstone drop
 +   time histogram (CASSANDRA-6522)
 + * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
 + * Make commitlog failure handling configurable (CASSANDRA-6364)
 +Merged from 1.2:
   * Fix broken streams when replacing with same IP (CASSANDRA-6622)
   * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)

[1/5] git commit: Sleep for RING_DELAY before replacing with same IP Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622

[brandonwilliams]

Updated Branches:
  refs/heads/cassandra-2.0 c5f225bd6 -> 80cebec5d
  refs/heads/trunk afd636874 -> 67101c29c


Sleep for RING_DELAY before replacing with same IP
Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00a8b1e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00a8b1e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00a8b1e6

Branch: refs/heads/cassandra-2.0
Commit: 00a8b1e6ece0f89ae038589f36c7b9a9012fb7f7
Parents: 7787dea
Author: Brandon Williams 
Authored: Tue Feb 11 17:12:45 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:14:18 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00a8b1e6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 56059a1..0674dde 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.16
+ * Fix broken streams when replacing with same IP (CASSANDRA-6622)
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00a8b1e6/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 94dd73d..ec8c7c3 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -735,6 +735,18 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 }
+else
+{
+try
+{
+Thread.sleep(RING_DELAY);
+}
+catch (InterruptedException e)
+{
+throw new AssertionError(e);
+}
+
+}
 setMode(Mode.JOINING, "Replacing a node with token(s): " + 
tokens, true);
 }
 

[3/5] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

[brandonwilliams]

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/80cebec5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/80cebec5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/80cebec5

Branch: refs/heads/trunk
Commit: 80cebec5d03c8dfb1afd416e138b65ccf5c791d9
Parents: c5f225b 00a8b1e
Author: Brandon Williams 
Authored: Tue Feb 11 17:14:50 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:14:50 2014 -0600

--
 CHANGES.txt | 1 -
 1 file changed, 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/80cebec5/CHANGES.txt
--
diff --cc CHANGES.txt
index fd640cd,0674dde..f9b2032
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,4 +1,16 @@@
- 2.0.6
 -1.2.16
 + * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
 + * Fix EstimatedHistogram races (CASSANDRA-6682)
 + * Failure detector correctly converts initial value to nanos (CASSANDRA-6658)
 + * Add nodetool taketoken to relocate vnodes (CASSANDRA-4445)
 + * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
 + * Improve nodetool cfhistograms formatting (CASSANDRA-6360)
 + * Expose bulk loading progress over JMX (CASSANDRA-4757)
 + * Correctly handle null with IF conditions and TTL (CASSANDRA-6623)
 + * Account for range/row tombstones in tombstone drop
 +   time histogram (CASSANDRA-6522)
 + * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
 + * Make commitlog failure handling configurable (CASSANDRA-6364)
 +Merged from 1.2:
   * Fix broken streams when replacing with same IP (CASSANDRA-6622)
   * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)

[5/5] git commit: Merge branch 'cassandra-2.0' into trunk

[brandonwilliams]

Merge branch 'cassandra-2.0' into trunk

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/67101c29
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/67101c29
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/67101c29

Branch: refs/heads/trunk
Commit: 67101c29c95984e4e34d5a7c5d71246ad79da2bc
Parents: afd6368 80cebec
Author: Brandon Williams 
Authored: Tue Feb 11 17:15:27 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:15:27 2014 -0600

--

--

[2/5] git commit: Sleep for RING_DELAY before replacing with same IP Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622

[brandonwilliams]

Sleep for RING_DELAY before replacing with same IP
Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00a8b1e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00a8b1e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00a8b1e6

Branch: refs/heads/trunk
Commit: 00a8b1e6ece0f89ae038589f36c7b9a9012fb7f7
Parents: 7787dea
Author: Brandon Williams 
Authored: Tue Feb 11 17:12:45 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:14:18 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00a8b1e6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 56059a1..0674dde 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.16
+ * Fix broken streams when replacing with same IP (CASSANDRA-6622)
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00a8b1e6/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 94dd73d..ec8c7c3 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -735,6 +735,18 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 }
+else
+{
+try
+{
+Thread.sleep(RING_DELAY);
+}
+catch (InterruptedException e)
+{
+throw new AssertionError(e);
+}
+
+}
 setMode(Mode.JOINING, "Replacing a node with token(s): " + 
tokens, true);
 }
 

git commit: Sleep for RING_DELAY before replacing with same IP Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622

[brandonwilliams]

Updated Branches:
  refs/heads/cassandra-1.2 7787deab7 -> 00a8b1e6e


Sleep for RING_DELAY before replacing with same IP
Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/00a8b1e6
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/00a8b1e6
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/00a8b1e6

Branch: refs/heads/cassandra-1.2
Commit: 00a8b1e6ece0f89ae038589f36c7b9a9012fb7f7
Parents: 7787dea
Author: Brandon Williams 
Authored: Tue Feb 11 17:12:45 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:14:18 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/00a8b1e6/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 56059a1..0674dde 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.16
+ * Fix broken streams when replacing with same IP (CASSANDRA-6622)
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/00a8b1e6/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 94dd73d..ec8c7c3 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -735,6 +735,18 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 }
+else
+{
+try
+{
+Thread.sleep(RING_DELAY);
+}
+catch (InterruptedException e)
+{
+throw new AssertionError(e);
+}
+
+}
 setMode(Mode.JOINING, "Replacing a node with token(s): " + 
tokens, true);
 }
 

[5/5] git commit: Merge branch 'cassandra-2.0' into trunk

[brandonwilliams]

Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/afd63687
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/afd63687
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/afd63687

Branch: refs/heads/trunk
Commit: afd6368745d6ac4bc43e70cc1c4b47ce59f754e7
Parents: aac421a c5f225b
Author: Brandon Williams 
Authored: Tue Feb 11 17:14:01 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:14:01 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/afd63687/CHANGES.txt
--
diff --cc CHANGES.txt
index 8e7e249,fd640cd..1025510
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -42,7 -9,10 +42,8 @@@
   * Correctly handle null with IF conditions and TTL (CASSANDRA-6623)
   * Account for range/row tombstones in tombstone drop
 time histogram (CASSANDRA-6522)
 - * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
 - * Make commitlog failure handling configurable (CASSANDRA-6364)
  Merged from 1.2:
+  * Fix broken streams when replacing with same IP (CASSANDRA-6622)
   * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
   * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/afd63687/src/java/org/apache/cassandra/service/StorageService.java
--

[2/5] git commit: Sleep for RING_DELAY before replacing with same IP Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622

[brandonwilliams]

Sleep for RING_DELAY before replacing with same IP
Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf6b4888
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf6b4888
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf6b4888

Branch: refs/heads/trunk
Commit: cf6b4888242d9f4ce891d051b02e6f9eb529d300
Parents: 1b8b7b5
Author: Brandon Williams 
Authored: Tue Feb 11 17:12:45 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:12:45 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6b4888/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 82783f8..1bd9c6e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.16
+ * Fix broken streams when replacing with same IP (CASSANDRA-6622)
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6b4888/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 94dd73d..ec8c7c3 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -735,6 +735,18 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 }
+else
+{
+try
+{
+Thread.sleep(RING_DELAY);
+}
+catch (InterruptedException e)
+{
+throw new AssertionError(e);
+}
+
+}
 setMode(Mode.JOINING, "Replacing a node with token(s): " + 
tokens, true);
 }
 

[3/5] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

[brandonwilliams]

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5f225bd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5f225bd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5f225bd

Branch: refs/heads/trunk
Commit: c5f225bd6bd85250ed8025720ba77b2a6d6be061
Parents: b0e4f00 cf6b488
Author: Brandon Williams 
Authored: Tue Feb 11 17:13:52 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:13:52 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c5f225bd/CHANGES.txt
--
diff --cc CHANGES.txt
index aec6f5e,1bd9c6e..fd640cd
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,5 +1,18 @@@
 -1.2.16
 +2.0.6
 + * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
 + * Fix EstimatedHistogram races (CASSANDRA-6682)
 + * Failure detector correctly converts initial value to nanos (CASSANDRA-6658)
 + * Add nodetool taketoken to relocate vnodes (CASSANDRA-4445)
 + * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
 + * Improve nodetool cfhistograms formatting (CASSANDRA-6360)
 + * Expose bulk loading progress over JMX (CASSANDRA-4757)
 + * Correctly handle null with IF conditions and TTL (CASSANDRA-6623)
 + * Account for range/row tombstones in tombstone drop
 +   time histogram (CASSANDRA-6522)
 + * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
 + * Make commitlog failure handling configurable (CASSANDRA-6364)
 +Merged from 1.2:
+  * Fix broken streams when replacing with same IP (CASSANDRA-6622)
   * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
   * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c5f225bd/src/java/org/apache/cassandra/service/StorageService.java
--

[4/5] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

[brandonwilliams]

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
CHANGES.txt


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c5f225bd
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c5f225bd
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c5f225bd

Branch: refs/heads/cassandra-2.0
Commit: c5f225bd6bd85250ed8025720ba77b2a6d6be061
Parents: b0e4f00 cf6b488
Author: Brandon Williams 
Authored: Tue Feb 11 17:13:52 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:13:52 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c5f225bd/CHANGES.txt
--
diff --cc CHANGES.txt
index aec6f5e,1bd9c6e..fd640cd
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,17 -1,5 +1,18 @@@
 -1.2.16
 +2.0.6
 + * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
 + * Fix EstimatedHistogram races (CASSANDRA-6682)
 + * Failure detector correctly converts initial value to nanos (CASSANDRA-6658)
 + * Add nodetool taketoken to relocate vnodes (CASSANDRA-4445)
 + * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
 + * Improve nodetool cfhistograms formatting (CASSANDRA-6360)
 + * Expose bulk loading progress over JMX (CASSANDRA-4757)
 + * Correctly handle null with IF conditions and TTL (CASSANDRA-6623)
 + * Account for range/row tombstones in tombstone drop
 +   time histogram (CASSANDRA-6522)
 + * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
 + * Make commitlog failure handling configurable (CASSANDRA-6364)
 +Merged from 1.2:
+  * Fix broken streams when replacing with same IP (CASSANDRA-6622)
   * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
   * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c5f225bd/src/java/org/apache/cassandra/service/StorageService.java
--

[1/5] git commit: Sleep for RING_DELAY before replacing with same IP Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622

[brandonwilliams]

Updated Branches:
  refs/heads/cassandra-2.0 b0e4f00c8 -> c5f225bd6
  refs/heads/trunk aac421aaa -> afd636874


Sleep for RING_DELAY before replacing with same IP
Patch by brandonwilliams reviewed by Tyler Hobbs for CASSANDRA-6622


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cf6b4888
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cf6b4888
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cf6b4888

Branch: refs/heads/cassandra-2.0
Commit: cf6b4888242d9f4ce891d051b02e6f9eb529d300
Parents: 1b8b7b5
Author: Brandon Williams 
Authored: Tue Feb 11 17:12:45 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 17:12:45 2014 -0600

--
 CHANGES.txt |  1 +
 .../org/apache/cassandra/service/StorageService.java| 12 
 2 files changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6b4888/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 82783f8..1bd9c6e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.16
+ * Fix broken streams when replacing with same IP (CASSANDRA-6622)
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cf6b4888/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java 
b/src/java/org/apache/cassandra/service/StorageService.java
index 94dd73d..ec8c7c3 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -735,6 +735,18 @@ public class StorageService extends 
NotificationBroadcasterSupport implements IE
 }
 }
 }
+else
+{
+try
+{
+Thread.sleep(RING_DELAY);
+}
+catch (InterruptedException e)
+{
+throw new AssertionError(e);
+}
+
+}
 setMode(Mode.JOINING, "Replacing a node with token(s): " + 
tokens, true);
 }
 

[jira] [Commented] (CASSANDRA-6622) Streaming session failures during node replace of same address

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898472#comment-13898472
 ] 

Tyler Hobbs commented on CASSANDRA-6622:


+1

> Streaming session failures during node replace of same address
> --
>
> Key: CASSANDRA-6622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6622
> Project: Cassandra
>  Issue Type: Bug
> Environment: RHEL6, cassandra-2.0.4
>Reporter: Ravi Prasad
>Assignee: Brandon Williams
> Fix For: 1.2.16, 2.0.6
>
> Attachments: 0001-don-t-signal-restart-of-dead-states.txt, 
> 6622-2.0.txt, 6622-v2.txt, 6622_logs.tgz, logs.tgz
>
>
> When using replace_address, Gossiper ApplicationState is set to hibernate, 
> which is a down state. We are seeing that the peer nodes are seeing streaming 
> plan request even before the Gossiper on them marks the replacing node as 
> dead. As a result, streaming on peer nodes convicts the replacing node by 
> closing the stream handler.  
> I think, making the StorageService thread on the replacing node, sleep for 
> BROADCAST_INTERVAL before bootstrapping, would avoid this scenario.
> Relevant logs from peer node (see that the Gossiper on peer node mark the 
> replacing node as down, 2 secs after  the streaming init request):
> {noformat}
>  INFO [STREAM-INIT-/x.x.x.x:46436] 2014-01-26 20:42:24,388 
> StreamResultFuture.java (line 116) [Stream 
> #5c6cd940-86ca-11e3-90a0-411b913c0e88] Received streaming plan for Bootstrap
> 
>  INFO [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with /x.x.x.x is 
> complete
>  WARN [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
>  INFO [GossipStage:1] 2014-01-26 20:42:25,242 Gossiper.java (line 850) 
> InetAddress /x.x.x.x is now DOWN
> ERROR [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,766 StreamSession.java (line 
> 410) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Streaming error occurred
> java.lang.RuntimeException: Outgoing stream handler has been closed
> at 
> org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:175)
> at 
> org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:436)
> at 
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:358)
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
> at java.lang.Thread.run(Thread.java:722)
>  INFO [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with 
> /x.x.x.x is complete
>  WARN [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[1/5] git commit: Fix CleanupTest

[aleksey]

Updated Branches:
  refs/heads/trunk f007a3535 -> aac421aaa


Fix CleanupTest

patch by marcuse, reviewed by jbellis for CASSANDRA-6679


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b8b7b54
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b8b7b54
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b8b7b54

Branch: refs/heads/trunk
Commit: 1b8b7b54087dd2dc2aeb2b9a0b06fbb4f13030be
Parents: c562700
Author: Marcus Eriksson 
Authored: Mon Feb 10 18:32:06 2014 +0100
Committer: Marcus Eriksson 
Committed: Mon Feb 10 18:36:28 2014 +0100

--
 test/unit/org/apache/cassandra/db/CleanupTest.java | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b8b7b54/test/unit/org/apache/cassandra/db/CleanupTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/CleanupTest.java 
b/test/unit/org/apache/cassandra/db/CleanupTest.java
index 5872407..d5c2b07 100644
--- a/test/unit/org/apache/cassandra/db/CleanupTest.java
+++ b/test/unit/org/apache/cassandra/db/CleanupTest.java
@@ -64,8 +64,7 @@ public class CleanupTest extends SchemaLoader
 @Test
 public void testCleanup() throws IOException, ExecutionException, 
InterruptedException, ConfigurationException
 {
-StorageService.instance.initServer(0);
-
+StorageService.instance.getTokenMetadata().clearUnsafe();
 Table table = Table.open(TABLE1);
 ColumnFamilyStore cfs = table.getColumnFamilyStore(CF2);
 

[3/5] git commit: Log USING TTL/TIMESTAMP in a counter update warning

[aleksey]

Log USING TTL/TIMESTAMP in a counter update warning

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6649


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7787deab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7787deab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7787deab

Branch: refs/heads/trunk
Commit: 7787deab72fe5772170c0cf6b6712ed2211158ed
Parents: 1b8b7b5
Author: Aleksey Yeschenko 
Authored: Wed Feb 12 01:39:40 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Wed Feb 12 01:39:40 2014 +0300

--
 CHANGES.txt |  1 +
 .../cql3/statements/ModificationStatement.java  | 27 
 2 files changed, 28 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7787deab/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 82783f8..56059a1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
  * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
+ * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
 
 
 1.2.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7787deab/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index bfbf511..1b4dc37 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -20,6 +20,9 @@ package org.apache.cassandra.cql3.statements;
 import java.nio.ByteBuffer;
 import java.util.*;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.auth.Permission;
 import org.apache.cassandra.cql3.*;
 import org.apache.cassandra.db.*;
@@ -45,6 +48,11 @@ public abstract class ModificationStatement extends 
CFStatement implements CQLSt
 LOGGED, UNLOGGED, COUNTER
 }
 
+private static final Logger logger = 
LoggerFactory.getLogger(ModificationStatement.class);
+
+private static boolean loggedCounterTTL = false;
+private static boolean loggedCounterTimestamp = false;
+
 protected Type type;
 
 private Long timestamp;
@@ -74,6 +82,25 @@ public abstract class ModificationStatement extends 
CFStatement implements CQLSt
 
 if (timeToLive > ExpiringColumn.MAX_TTL)
 throw new InvalidRequestException(String.format("ttl is too large. 
requested (%d) maximum (%d)", timeToLive, ExpiringColumn.MAX_TTL));
+
+if (type == Type.COUNTER)
+{
+if (timestamp != null && !loggedCounterTimestamp)
+{
+logger.warn("Detected use of 'USING TIMESTAMP' in a counter 
UPDATE. This is invalid " +
+"because counters do not use timestamps, and the 
timestamp has been ignored. " +
+"Such queries will be rejected in Cassandra 2.1+ - 
please fix your queries before then.");
+loggedCounterTimestamp = true;
+}
+
+if (timeToLive != 0 && !loggedCounterTTL)
+{
+logger.warn("Detected use of 'USING TTL' in a counter UPDATE. 
This is invalid " +
+"because counter tables do not support TTL, and 
the TTL value has been ignored. " +
+"Such queries will be rejected in Cassandra 2.1+ - 
please fix your queries before then.");
+loggedCounterTTL = true;
+}
+}
 }
 
 protected abstract void validateConsistency(ConsistencyLevel cl) throws 
InvalidRequestException;

[2/5] git commit: who forgets to add files? I do.

[aleksey]

who forgets to add files? I do.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/662f546a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/662f546a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/662f546a

Branch: refs/heads/trunk
Commit: 662f546aa76b7f6473b8da8b9ea675a9fd9757e1
Parents: b9aece4
Author: Brandon Williams 
Authored: Tue Feb 11 11:30:02 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 11:30:02 2014 -0600

--
 .../cassandra/hadoop/ReporterWrapper.java   | 57 
 1 file changed, 57 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/662f546a/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java 
b/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
new file mode 100644
index 000..9940ba4
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
@@ -0,0 +1,57 @@
+package org.apache.cassandra.hadoop;
+
+import org.apache.hadoop.mapred.Counters;
+import org.apache.hadoop.mapred.InputSplit;
+import org.apache.hadoop.mapred.Reporter;
+import org.apache.hadoop.mapreduce.StatusReporter;
+
+/**
+ * A reporter that works with both mapred and mapreduce APIs.
+ */
+public class ReporterWrapper extends StatusReporter implements Reporter {
+private Reporter wrappedReporter;
+
+public ReporterWrapper(Reporter reporter) {
+wrappedReporter = reporter;
+}
+
+@Override
+public Counters.Counter getCounter(Enum anEnum) {
+return wrappedReporter.getCounter(anEnum);
+}
+
+@Override
+public Counters.Counter getCounter(String s, String s1) {
+return wrappedReporter.getCounter(s, s1);
+}
+
+@Override
+public void incrCounter(Enum anEnum, long l) {
+wrappedReporter.incrCounter(anEnum, l);
+}
+
+@Override
+public void incrCounter(String s, String s1, long l) {
+wrappedReporter.incrCounter(s, s1, l);
+}
+
+@Override
+public InputSplit getInputSplit() throws UnsupportedOperationException {
+return wrappedReporter.getInputSplit();
+}
+
+@Override
+public void progress() {
+wrappedReporter.progress();
+}
+
+// @Override
+public float getProgress() {
+throw new UnsupportedOperationException();
+}
+
+@Override
+public void setStatus(String s) {
+wrappedReporter.setStatus(s);
+}
+}
\ No newline at end of file

[5/5] git commit: Merge branch 'cassandra-2.0' into trunk

[aleksey]

Merge branch 'cassandra-2.0' into trunk

Conflicts:
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/aac421aa
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/aac421aa
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/aac421aa

Branch: refs/heads/trunk
Commit: aac421aaa1d2cf214839eedd55abf35958ae4487
Parents: f007a35 b0e4f00
Author: Aleksey Yeschenko 
Authored: Wed Feb 12 02:05:16 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Wed Feb 12 02:05:16 2014 +0300

--
 CHANGES.txt |  3 +++
 .../cql3/statements/ModificationStatement.java  | 12 +---
 2 files changed, 12 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/aac421aa/CHANGES.txt
--
diff --cc CHANGES.txt
index eec6296,aec6f5e..8e7e249
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,35 -1,3 +1,37 @@@
 +2.1
 + * add listsnapshots command to nodetool (CASSANDRA-5742)
 + * Introduce AtomicBTreeColumns (CASSANDRA-6271)
 + * Multithreaded commitlog (CASSANDRA-3578)
 + * allocate fixed index summary memory pool and resample cold index summaries 
 +   to use less memory (CASSANDRA-5519)
 + * Removed multithreaded compaction (CASSANDRA-6142)
 + * Parallelize fetching rows for low-cardinality indexes (CASSANDRA-1337)
 + * change logging from log4j to logback (CASSANDRA-5883)
 + * switch to LZ4 compression for internode communication (CASSANDRA-5887)
 + * Stop using Thrift-generated Index* classes internally (CASSANDRA-5971)
 + * Remove 1.2 network compatibility code (CASSANDRA-5960)
 + * Remove leveled json manifest migration code (CASSANDRA-5996)
 + * Remove CFDefinition (CASSANDRA-6253)
 + * Use AtomicIntegerFieldUpdater in RefCountedMemory (CASSANDRA-6278)
 + * User-defined types for CQL3 (CASSANDRA-5590)
 + * Use of o.a.c.metrics in nodetool (CASSANDRA-5871, 6406)
 + * Batch read from OTC's queue and cleanup (CASSANDRA-1632)
 + * Secondary index support for collections (CASSANDRA-4511, 6383)
 + * SSTable metadata(Stats.db) format change (CASSANDRA-6356)
 + * Push composites support in the storage engine
 +   (CASSANDRA-5417, CASSANDRA-6520)
 + * Add snapshot space used to cfstats (CASSANDRA-6231)
 + * Add cardinality estimator for key count estimation (CASSANDRA-5906)
 + * CF id is changed to be non-deterministic. Data dir/key cache are created
 +   uniquely for CF id (CASSANDRA-5202)
 + * New counters implementation (CASSANDRA-6504)
 + * Replace UnsortedColumns and TreeMapBackedSortedColumns with rewritten
 +   ArrayBackedSortedColumns (CASSANDRA-6630, CASSANDRA-6662)
 + * Add option to use row cache with a given amount of rows (CASSANDRA-5357)
 + * Avoid repairing already repaired data (CASSANDRA-5351)
++ * Reject counter updates with USING TTL/TIMESTAMP (CASSANDRA-6649)
++
 +
  2.0.6
   * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
   * Fix EstimatedHistogram races (CASSANDRA-6682)
@@@ -45,8 -16,8 +47,9 @@@ Merged from 1.2
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
   * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
   * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
+  * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
  
 +
  2.0.5
   * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609)
   * Add ks.cf names to tombstone logging (CASSANDRA-6597)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/aac421aa/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 3775bde,676286c..6d18f1b
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@@ -126,7 -131,26 +126,13 @@@ public abstract class ModificationState
  public void validate(ClientState state) throws InvalidRequestException
  {
  if (hasConditions() && attrs.isTimestampSet())
--throw new InvalidRequestException("Custom timestamps are not 
allowed when conditions are used");
++throw new InvalidRequestException("Cannot provide custom 
timestamp for conditional updates");
+ 
 -if (isCounter())
 -{
 -if (attrs.isTimestampSet() && !loggedCounterTimestamp)
 -{
 -logger.warn("Detected use of 'USING TIMESTAMP' in a counter 
UPDATE. This is invalid " +
 -"because counters do not use timestamps, and the 
t

[4/5] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

[aleksey]

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
test/unit/org/apache/cassandra/db/CleanupTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b0e4f00c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b0e4f00c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b0e4f00c

Branch: refs/heads/trunk
Commit: b0e4f00c8c3986c0702f2b08b0d2cd4dd18b1dbf
Parents: 662f546 7787dea
Author: Aleksey Yeschenko 
Authored: Wed Feb 12 01:51:09 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Wed Feb 12 01:51:09 2014 +0300

--
 CHANGES.txt |  1 +
 .../cql3/statements/ModificationStatement.java  | 28 +++-
 2 files changed, 28 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b0e4f00c/CHANGES.txt
--
diff --cc CHANGES.txt
index 16cbd0a,56059a1..aec6f5e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -16,33 -3,24 +16,34 @@@ Merged from 1.2
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
   * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
   * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
+  * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
  
 -
 -1.2.15
 - * Move handling of migration event source to solve bootstrap race 
(CASSANDRA-6648)
 - * Make sure compaction throughput value doesn't overflow with int math 
(CASSANDRA-6647)
 -
 -
 -1.2.14
 - * Reverted code to limit CQL prepared statement cache by size 
(CASSANDRA-6592)
 - * add cassandra.default_messaging_version property to allow easier
 -   upgrading from 1.1 (CASSANDRA-6619)
 - * Allow executing CREATE statements multiple times (CASSANDRA-6471)
 - * Don't send confusing info with timeouts (CASSANDRA-6491)
 - * Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
 - * Don't drop local mutations without a hint (CASSANDRA-6510)
 - * Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
 - * Validate SliceRange start and finish lengths (CASSANDRA-6521)
 +2.0.5
 + * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609)
 + * Add ks.cf names to tombstone logging (CASSANDRA-6597)
 + * Use LOCAL_QUORUM for LWT operations at LOCAL_SERIAL (CASSANDRA-6495)
 + * Wait for gossip to settle before accepting client connections 
(CASSANDRA-4288)
 + * Delete unfinished compaction incrementally (CASSANDRA-6086)
 + * Allow specifying custom secondary index options in CQL3 (CASSANDRA-6480)
 + * Improve replica pinning for cache efficiency in DES (CASSANDRA-6485)
 + * Fix LOCAL_SERIAL from thrift (CASSANDRA-6584)
 + * Don't special case received counts in CAS timeout exceptions 
(CASSANDRA-6595)
 + * Add support for 2.1 global counter shards (CASSANDRA-6505)
 + * Fix NPE when streaming connection is not yet established (CASSANDRA-6210)
 + * Avoid rare duplicate read repair triggering (CASSANDRA-6606)
 + * Fix paging discardFirst (CASSANDRA-6555)
 + * Fix ArrayIndexOutOfBoundsException in 2ndary index query (CASSANDRA-6470)
 + * Release sstables upon rebuilding 2i (CASSANDRA-6635)
 + * Add AbstractCompactionStrategy.startup() method (CASSANDRA-6637)
 + * SSTableScanner may skip rows during cleanup (CASSANDRA-6638)
 + * sstables from stalled repair sessions can resurrect deleted data 
(CASSANDRA-6503)
 + * Switch stress to use ITransportFactory (CASSANDRA-6641)
 + * Fix IllegalArgumentException during prepare (CASSANDRA-6592)
 + * Fix possible loss of 2ndary index entries during compaction 
(CASSANDRA-6517)
 + * Fix direct Memory on architectures that do not support unaligned long 
access
 +   (CASSANDRA-6628)
 + * Let scrub optionally skip broken counter partitions (CASSANDRA-5930)
 +Merged from 1.2:
   * fsync compression metadata (CASSANDRA-6531)
   * Validate CF existence on execution for prepared statement (CASSANDRA-6535)
   * Add ability to throttle batchlog replay (CASSANDRA-6550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b0e4f00c/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 2567043,1b4dc37..676286c
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@@ -20,10 -20,10 +20,12 @@@ package org.apache.cassandra.cql3.state
  import java.nio.ByteBuffer;
  import java.util.*;
  
 +import org.github.jamm.MemoryMeter;
+ import org.slf4j.Logger;
+ import org.slf4j.LoggerFactor

[3/3] git commit: Merge branch 'cassandra-1.2' into cassandra-2.0

[aleksey]

Merge branch 'cassandra-1.2' into cassandra-2.0

Conflicts:
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
test/unit/org/apache/cassandra/db/CleanupTest.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b0e4f00c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b0e4f00c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b0e4f00c

Branch: refs/heads/cassandra-2.0
Commit: b0e4f00c8c3986c0702f2b08b0d2cd4dd18b1dbf
Parents: 662f546 7787dea
Author: Aleksey Yeschenko 
Authored: Wed Feb 12 01:51:09 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Wed Feb 12 01:51:09 2014 +0300

--
 CHANGES.txt |  1 +
 .../cql3/statements/ModificationStatement.java  | 28 +++-
 2 files changed, 28 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b0e4f00c/CHANGES.txt
--
diff --cc CHANGES.txt
index 16cbd0a,56059a1..aec6f5e
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -16,33 -3,24 +16,34 @@@ Merged from 1.2
   * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
   * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
   * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
+  * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
  
 -
 -1.2.15
 - * Move handling of migration event source to solve bootstrap race 
(CASSANDRA-6648)
 - * Make sure compaction throughput value doesn't overflow with int math 
(CASSANDRA-6647)
 -
 -
 -1.2.14
 - * Reverted code to limit CQL prepared statement cache by size 
(CASSANDRA-6592)
 - * add cassandra.default_messaging_version property to allow easier
 -   upgrading from 1.1 (CASSANDRA-6619)
 - * Allow executing CREATE statements multiple times (CASSANDRA-6471)
 - * Don't send confusing info with timeouts (CASSANDRA-6491)
 - * Don't resubmit counter mutation runnables internally (CASSANDRA-6427)
 - * Don't drop local mutations without a hint (CASSANDRA-6510)
 - * Don't allow null max_hint_window_in_ms (CASSANDRA-6419)
 - * Validate SliceRange start and finish lengths (CASSANDRA-6521)
 +2.0.5
 + * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609)
 + * Add ks.cf names to tombstone logging (CASSANDRA-6597)
 + * Use LOCAL_QUORUM for LWT operations at LOCAL_SERIAL (CASSANDRA-6495)
 + * Wait for gossip to settle before accepting client connections 
(CASSANDRA-4288)
 + * Delete unfinished compaction incrementally (CASSANDRA-6086)
 + * Allow specifying custom secondary index options in CQL3 (CASSANDRA-6480)
 + * Improve replica pinning for cache efficiency in DES (CASSANDRA-6485)
 + * Fix LOCAL_SERIAL from thrift (CASSANDRA-6584)
 + * Don't special case received counts in CAS timeout exceptions 
(CASSANDRA-6595)
 + * Add support for 2.1 global counter shards (CASSANDRA-6505)
 + * Fix NPE when streaming connection is not yet established (CASSANDRA-6210)
 + * Avoid rare duplicate read repair triggering (CASSANDRA-6606)
 + * Fix paging discardFirst (CASSANDRA-6555)
 + * Fix ArrayIndexOutOfBoundsException in 2ndary index query (CASSANDRA-6470)
 + * Release sstables upon rebuilding 2i (CASSANDRA-6635)
 + * Add AbstractCompactionStrategy.startup() method (CASSANDRA-6637)
 + * SSTableScanner may skip rows during cleanup (CASSANDRA-6638)
 + * sstables from stalled repair sessions can resurrect deleted data 
(CASSANDRA-6503)
 + * Switch stress to use ITransportFactory (CASSANDRA-6641)
 + * Fix IllegalArgumentException during prepare (CASSANDRA-6592)
 + * Fix possible loss of 2ndary index entries during compaction 
(CASSANDRA-6517)
 + * Fix direct Memory on architectures that do not support unaligned long 
access
 +   (CASSANDRA-6628)
 + * Let scrub optionally skip broken counter partitions (CASSANDRA-5930)
 +Merged from 1.2:
   * fsync compression metadata (CASSANDRA-6531)
   * Validate CF existence on execution for prepared statement (CASSANDRA-6535)
   * Add ability to throttle batchlog replay (CASSANDRA-6550)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b0e4f00c/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 2567043,1b4dc37..676286c
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@@ -20,10 -20,10 +20,12 @@@ package org.apache.cassandra.cql3.state
  import java.nio.ByteBuffer;
  import java.util.*;
  
 +import org.github.jamm.MemoryMeter;
+ import org.slf4j.Logger;
+ import org.slf4j.Logg

[2/3] git commit: Log USING TTL/TIMESTAMP in a counter update warning

[aleksey]

Log USING TTL/TIMESTAMP in a counter update warning

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6649


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7787deab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7787deab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7787deab

Branch: refs/heads/cassandra-2.0
Commit: 7787deab72fe5772170c0cf6b6712ed2211158ed
Parents: 1b8b7b5
Author: Aleksey Yeschenko 
Authored: Wed Feb 12 01:39:40 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Wed Feb 12 01:39:40 2014 +0300

--
 CHANGES.txt |  1 +
 .../cql3/statements/ModificationStatement.java  | 27 
 2 files changed, 28 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7787deab/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 82783f8..56059a1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
  * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
+ * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
 
 
 1.2.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7787deab/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index bfbf511..1b4dc37 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -20,6 +20,9 @@ package org.apache.cassandra.cql3.statements;
 import java.nio.ByteBuffer;
 import java.util.*;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.auth.Permission;
 import org.apache.cassandra.cql3.*;
 import org.apache.cassandra.db.*;
@@ -45,6 +48,11 @@ public abstract class ModificationStatement extends 
CFStatement implements CQLSt
 LOGGED, UNLOGGED, COUNTER
 }
 
+private static final Logger logger = 
LoggerFactory.getLogger(ModificationStatement.class);
+
+private static boolean loggedCounterTTL = false;
+private static boolean loggedCounterTimestamp = false;
+
 protected Type type;
 
 private Long timestamp;
@@ -74,6 +82,25 @@ public abstract class ModificationStatement extends 
CFStatement implements CQLSt
 
 if (timeToLive > ExpiringColumn.MAX_TTL)
 throw new InvalidRequestException(String.format("ttl is too large. 
requested (%d) maximum (%d)", timeToLive, ExpiringColumn.MAX_TTL));
+
+if (type == Type.COUNTER)
+{
+if (timestamp != null && !loggedCounterTimestamp)
+{
+logger.warn("Detected use of 'USING TIMESTAMP' in a counter 
UPDATE. This is invalid " +
+"because counters do not use timestamps, and the 
timestamp has been ignored. " +
+"Such queries will be rejected in Cassandra 2.1+ - 
please fix your queries before then.");
+loggedCounterTimestamp = true;
+}
+
+if (timeToLive != 0 && !loggedCounterTTL)
+{
+logger.warn("Detected use of 'USING TTL' in a counter UPDATE. 
This is invalid " +
+"because counter tables do not support TTL, and 
the TTL value has been ignored. " +
+"Such queries will be rejected in Cassandra 2.1+ - 
please fix your queries before then.");
+loggedCounterTTL = true;
+}
+}
 }
 
 protected abstract void validateConsistency(ConsistencyLevel cl) throws 
InvalidRequestException;

[1/3] git commit: Fix CleanupTest

[aleksey]

Updated Branches:
  refs/heads/cassandra-2.0 662f546aa -> b0e4f00c8


Fix CleanupTest

patch by marcuse, reviewed by jbellis for CASSANDRA-6679


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1b8b7b54
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1b8b7b54
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1b8b7b54

Branch: refs/heads/cassandra-2.0
Commit: 1b8b7b54087dd2dc2aeb2b9a0b06fbb4f13030be
Parents: c562700
Author: Marcus Eriksson 
Authored: Mon Feb 10 18:32:06 2014 +0100
Committer: Marcus Eriksson 
Committed: Mon Feb 10 18:36:28 2014 +0100

--
 test/unit/org/apache/cassandra/db/CleanupTest.java | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/1b8b7b54/test/unit/org/apache/cassandra/db/CleanupTest.java
--
diff --git a/test/unit/org/apache/cassandra/db/CleanupTest.java 
b/test/unit/org/apache/cassandra/db/CleanupTest.java
index 5872407..d5c2b07 100644
--- a/test/unit/org/apache/cassandra/db/CleanupTest.java
+++ b/test/unit/org/apache/cassandra/db/CleanupTest.java
@@ -64,8 +64,7 @@ public class CleanupTest extends SchemaLoader
 @Test
 public void testCleanup() throws IOException, ExecutionException, 
InterruptedException, ConfigurationException
 {
-StorageService.instance.initServer(0);
-
+StorageService.instance.getTokenMetadata().clearUnsafe();
 Table table = Table.open(TABLE1);
 ColumnFamilyStore cfs = table.getColumnFamilyStore(CF2);
 

[jira] [Updated] (CASSANDRA-6622) Streaming session failures during node replace of same address

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6622:


Attachment: 6622-v2.txt

> Streaming session failures during node replace of same address
> --
>
> Key: CASSANDRA-6622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6622
> Project: Cassandra
>  Issue Type: Bug
> Environment: RHEL6, cassandra-2.0.4
>Reporter: Ravi Prasad
>Assignee: Brandon Williams
> Fix For: 1.2.16, 2.0.6
>
> Attachments: 0001-don-t-signal-restart-of-dead-states.txt, 
> 6622-2.0.txt, 6622-v2.txt, 6622_logs.tgz, logs.tgz
>
>
> When using replace_address, Gossiper ApplicationState is set to hibernate, 
> which is a down state. We are seeing that the peer nodes are seeing streaming 
> plan request even before the Gossiper on them marks the replacing node as 
> dead. As a result, streaming on peer nodes convicts the replacing node by 
> closing the stream handler.  
> I think, making the StorageService thread on the replacing node, sleep for 
> BROADCAST_INTERVAL before bootstrapping, would avoid this scenario.
> Relevant logs from peer node (see that the Gossiper on peer node mark the 
> replacing node as down, 2 secs after  the streaming init request):
> {noformat}
>  INFO [STREAM-INIT-/x.x.x.x:46436] 2014-01-26 20:42:24,388 
> StreamResultFuture.java (line 116) [Stream 
> #5c6cd940-86ca-11e3-90a0-411b913c0e88] Received streaming plan for Bootstrap
> 
>  INFO [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with /x.x.x.x is 
> complete
>  WARN [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
>  INFO [GossipStage:1] 2014-01-26 20:42:25,242 Gossiper.java (line 850) 
> InetAddress /x.x.x.x is now DOWN
> ERROR [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,766 StreamSession.java (line 
> 410) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Streaming error occurred
> java.lang.RuntimeException: Outgoing stream handler has been closed
> at 
> org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:175)
> at 
> org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:436)
> at 
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:358)
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
> at java.lang.Thread.run(Thread.java:722)
>  INFO [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with 
> /x.x.x.x is complete
>  WARN [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6622) Streaming session failures during node replace of same address

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6622:


Attachment: (was: 6622-v2.txt)

> Streaming session failures during node replace of same address
> --
>
> Key: CASSANDRA-6622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6622
> Project: Cassandra
>  Issue Type: Bug
> Environment: RHEL6, cassandra-2.0.4
>Reporter: Ravi Prasad
>Assignee: Brandon Williams
> Fix For: 1.2.16, 2.0.6
>
> Attachments: 0001-don-t-signal-restart-of-dead-states.txt, 
> 6622-2.0.txt, 6622-v2.txt, 6622_logs.tgz, logs.tgz
>
>
> When using replace_address, Gossiper ApplicationState is set to hibernate, 
> which is a down state. We are seeing that the peer nodes are seeing streaming 
> plan request even before the Gossiper on them marks the replacing node as 
> dead. As a result, streaming on peer nodes convicts the replacing node by 
> closing the stream handler.  
> I think, making the StorageService thread on the replacing node, sleep for 
> BROADCAST_INTERVAL before bootstrapping, would avoid this scenario.
> Relevant logs from peer node (see that the Gossiper on peer node mark the 
> replacing node as down, 2 secs after  the streaming init request):
> {noformat}
>  INFO [STREAM-INIT-/x.x.x.x:46436] 2014-01-26 20:42:24,388 
> StreamResultFuture.java (line 116) [Stream 
> #5c6cd940-86ca-11e3-90a0-411b913c0e88] Received streaming plan for Bootstrap
> 
>  INFO [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with /x.x.x.x is 
> complete
>  WARN [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
>  INFO [GossipStage:1] 2014-01-26 20:42:25,242 Gossiper.java (line 850) 
> InetAddress /x.x.x.x is now DOWN
> ERROR [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,766 StreamSession.java (line 
> 410) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Streaming error occurred
> java.lang.RuntimeException: Outgoing stream handler has been closed
> at 
> org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:175)
> at 
> org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:436)
> at 
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:358)
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
> at java.lang.Thread.run(Thread.java:722)
>  INFO [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with 
> /x.x.x.x is complete
>  WARN [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

git commit: Log USING TTL/TIMESTAMP in a counter update warning

[aleksey]

Updated Branches:
  refs/heads/cassandra-1.2 1b8b7b540 -> 7787deab7


Log USING TTL/TIMESTAMP in a counter update warning

patch by Aleksey Yeschenko; reviewed by Sylvain Lebresne for
CASSANDRA-6649


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7787deab
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7787deab
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7787deab

Branch: refs/heads/cassandra-1.2
Commit: 7787deab72fe5772170c0cf6b6712ed2211158ed
Parents: 1b8b7b5
Author: Aleksey Yeschenko 
Authored: Wed Feb 12 01:39:40 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Wed Feb 12 01:39:40 2014 +0300

--
 CHANGES.txt |  1 +
 .../cql3/statements/ModificationStatement.java  | 27 
 2 files changed, 28 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7787deab/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 82783f8..56059a1 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -3,6 +3,7 @@
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
  * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
+ * Log USING TTL/TIMESTAMP in a counter update warning (CASSANDRA-6649)
 
 
 1.2.15

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7787deab/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index bfbf511..1b4dc37 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -20,6 +20,9 @@ package org.apache.cassandra.cql3.statements;
 import java.nio.ByteBuffer;
 import java.util.*;
 
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
 import org.apache.cassandra.auth.Permission;
 import org.apache.cassandra.cql3.*;
 import org.apache.cassandra.db.*;
@@ -45,6 +48,11 @@ public abstract class ModificationStatement extends 
CFStatement implements CQLSt
 LOGGED, UNLOGGED, COUNTER
 }
 
+private static final Logger logger = 
LoggerFactory.getLogger(ModificationStatement.class);
+
+private static boolean loggedCounterTTL = false;
+private static boolean loggedCounterTimestamp = false;
+
 protected Type type;
 
 private Long timestamp;
@@ -74,6 +82,25 @@ public abstract class ModificationStatement extends 
CFStatement implements CQLSt
 
 if (timeToLive > ExpiringColumn.MAX_TTL)
 throw new InvalidRequestException(String.format("ttl is too large. 
requested (%d) maximum (%d)", timeToLive, ExpiringColumn.MAX_TTL));
+
+if (type == Type.COUNTER)
+{
+if (timestamp != null && !loggedCounterTimestamp)
+{
+logger.warn("Detected use of 'USING TIMESTAMP' in a counter 
UPDATE. This is invalid " +
+"because counters do not use timestamps, and the 
timestamp has been ignored. " +
+"Such queries will be rejected in Cassandra 2.1+ - 
please fix your queries before then.");
+loggedCounterTimestamp = true;
+}
+
+if (timeToLive != 0 && !loggedCounterTTL)
+{
+logger.warn("Detected use of 'USING TTL' in a counter UPDATE. 
This is invalid " +
+"because counter tables do not support TTL, and 
the TTL value has been ignored. " +
+"Such queries will be rejected in Cassandra 2.1+ - 
please fix your queries before then.");
+loggedCounterTTL = true;
+}
+}
 }
 
 protected abstract void validateConsistency(ConsistencyLevel cl) throws 
InvalidRequestException;

[jira] [Commented] (CASSANDRA-5266) FileNotFoundException during AntiEntropy Stream Session after compaction

[Arya Goudarzi (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898426#comment-13898426
 ] 

Arya Goudarzi commented on CASSANDRA-5266:
--

The one failing above is SizeTiered.

> FileNotFoundException during AntiEntropy Stream Session after compaction
> 
>
> Key: CASSANDRA-5266
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5266
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 1.1.6
> Environment: Ubuntu 10.04
> Cassandra 1.2.14
> Jave 7 64 Bit Sun JVM
>Reporter: Arya Goudarzi
>Assignee: Yuki Morishita
>
> It seems that there is a race condition in the compaction process vs 
> AntiEntropy Streams. We often have repairs tossing lots of exceptions in the 
> logs as follows:
> java.lang.RuntimeException: java.io.FileNotFoundException: 
> /var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db
>  (No such file or directory)
> at 
> org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.FileNotFoundException: 
> /var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db
>  (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.(RandomAccessFile.java:216)
> at 
> org.apache.cassandra.io.util.RandomAccessReader.(RandomAccessReader.java:67)
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.(CompressedRandomAccessReader.java:64)
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:46)
> at 
> org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:145)
> at 
> org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> Now about 1-2 minutes before that, I see Secondary Index buildup and 
> compaction which does compact the same SSTable to a new one. 
> INFO [Thread-90838] 2013-02-07 13:51:58,362 SecondaryIndexManager.java (line 
> 126) Submitting index build of publisher_id, for data in 
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20054-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20055-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20057-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20058-Data.db')
>  INFO [CompactionExecutor:436] 2013-02-07 13:51:58,363 CompactionTask.java 
> (line 109) Compacting 
> [SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20055-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20054-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20058-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20057-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db')]
> Now, does this make the repair sessions to fail? It seems the node was trying 
> to stream a file to a node being repaired but apparently it did not.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (CASSANDRA-5201) Cassandra/Hadoop does not support current Hadoop releases

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-5201.
-

   Resolution: Fixed
Fix Version/s: 2.0.6

Committed.

> Cassandra/Hadoop does not support current Hadoop releases
> -
>
> Key: CASSANDRA-5201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5201
> Project: Cassandra
>  Issue Type: Bug
>  Components: Hadoop
>Affects Versions: 1.2.0
>Reporter: Brian Jeltema
>Assignee: Benjamin Coverston
> Fix For: 2.0.6
>
> Attachments: 5201_a.txt, hadoopCompat.patch, hadoopcompat-trunk.patch
>
>
> Using Hadoop 0.22.0 with Cassandra results in the stack trace below.
> It appears that version 0.21+ changed org.apache.hadoop.mapreduce.JobContext
> from a class to an interface.
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found 
> interface org.apache.hadoop.mapreduce.JobContext, but class was expected
>   at 
> org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSplits(ColumnFamilyInputFormat.java:103)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:357)
>   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1045)
>   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1042)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1042)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1062)
>   at MyHadoopApp.run(MyHadoopApp.java:163)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
>   at MyHadoopApp.main(MyHadoopApp.java:82)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:192)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6475) Control nodetool history logging directory

[Brian Nixon (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898383#comment-13898383
 ] 

Brian Nixon commented on CASSANDRA-6475:


After a discussion with a coworker, I made the connection to bash history files 
and the design makes more sense to me. At this point, I'm less concerned about 
log splitting than nodetool commands being blocked by a failure to create 
logging directories.

To keep our current setup, some parameter will have to be passed since default 
behavior will not be able capture the necessary information through our current 
environment variables. It would still be nice to have a single parameter to opt 
out of logging on nodetool/cassandra-cli/cqlsh or a more descriptive parameter 
to direct the tools to our logging directory than user home (that defaults to 
user home, if not set).

Note this hits CQL as well since cqlsh uses '~' to initialize its history 
logging.



> Control nodetool history logging directory
> --
>
> Key: CASSANDRA-6475
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6475
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tools
>Reporter: Brian Nixon
>Priority: Trivial
>  Labels: lhf
>
> Nodetool history is logged to a directory based on the current user home. 
> This leads to splintering of history with more than one user and, in one 
> case, a failure to run any nodetool commands as the user did not have write 
> access to their home directory.
> Suggested fix is to make the base directory for the history logging (both 
> nodetool and cli) configurable. A way to disable the logging of these tools 
> would also help.
> Reference:
> https://issues.apache.org/jira/browse/CASSANDRA-5823



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

git commit: Use EB HadoopCompat for compat with Hadoop 0.2.x Patch by Ben Coversaton, reviewed by brandonwilliams for CASSANDRA-5201

[brandonwilliams]

Updated Branches:
  refs/heads/trunk 6de2fd9bf -> f007a3535


Use EB HadoopCompat for compat with Hadoop 0.2.x
Patch by Ben Coversaton, reviewed by brandonwilliams for CASSANDRA-5201


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f007a353
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f007a353
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f007a353

Branch: refs/heads/trunk
Commit: f007a35357da582e928dc1ac872e4ebb4c09b708
Parents: 6de2fd9
Author: Brandon Williams 
Authored: Tue Feb 11 15:52:07 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 15:53:43 2014 -0600

--
 CHANGES.txt   |  1 +
 build.xml |  3 +++
 .../hadoop/AbstractColumnFamilyInputFormat.java   | 11 ++-
 .../hadoop/AbstractColumnFamilyOutputFormat.java  |  3 ++-
 .../apache/cassandra/hadoop/BulkOutputFormat.java |  3 ++-
 .../apache/cassandra/hadoop/BulkRecordWriter.java |  3 ++-
 .../cassandra/hadoop/ColumnFamilyInputFormat.java | 17 +
 .../hadoop/ColumnFamilyRecordReader.java  |  3 ++-
 .../hadoop/ColumnFamilyRecordWriter.java  |  3 ++-
 .../hadoop/cql3/CqlPagingInputFormat.java | 18 ++
 .../hadoop/cql3/CqlPagingRecordReader.java|  5 +++--
 .../cassandra/hadoop/cql3/CqlRecordWriter.java|  3 ++-
 .../cassandra/hadoop/pig/CassandraStorage.java|  5 +++--
 .../apache/cassandra/hadoop/pig/CqlStorage.java   |  5 +++--
 14 files changed, 50 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f007a353/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d8478e9..eec6296 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -31,6 +31,7 @@
  * Avoid repairing already repaired data (CASSANDRA-5351)
 
 2.0.6
+ * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
  * Fix EstimatedHistogram races (CASSANDRA-6682)
  * Failure detector correctly converts initial value to nanos (CASSANDRA-6658)
  * Add nodetool taketoken to relocate vnodes (CASSANDRA-4445)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f007a353/build.xml
--
diff --git a/build.xml b/build.xml
index d91e0e0..1494585 100644
--- a/build.xml
+++ b/build.xml
@@ -374,6 +374,7 @@

   
   
+  
   
   
 
@@ -417,6 +418,7 @@
 
 

+
 

   
@@ -482,6 +484,7 @@
 
 
 
+
 
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f007a353/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java 
b/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
index 74b..760193f 100644
--- a/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
@@ -29,6 +29,7 @@ import java.util.concurrent.TimeUnit;
 
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.Lists;
+import com.twitter.elephantbird.util.HadoopCompat;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -117,16 +118,16 @@ public abstract class AbstractColumnFamilyInputFormat extends InputFormat<
 
 public List getSplits(JobContext context) throws IOException
 {
-Configuration conf = context.getConfiguration();
+Configuration conf = HadoopCompat.getConfiguration(context);;
 
 validateConfiguration(conf);
 
 // cannonical ranges and nodes holding replicas
 List masterRangeNodes = getRangeMap(conf);
 
-keyspace = ConfigHelper.getInputKeyspace(context.getConfiguration());
-cfName = ConfigHelper.getInputColumnFamily(context.getConfiguration());
-partitioner = 
ConfigHelper.getInputPartitioner(context.getConfiguration());
+keyspace = ConfigHelper.getInputKeyspace(conf);
+cfName = ConfigHelper.getInputColumnFamily(conf);
+partitioner = ConfigHelper.getInputPartitioner(conf);
 logger.debug("partitioner is {}", partitioner);
 
 
@@ -344,7 +345,7 @@ public abstract class AbstractColumnFamilyInputFormat 
extends InputFormat<
 //
 public org.apache.hadoop.mapred.InputSplit[] getSplits(JobConf jobConf, 
int numSplits) throws IOException
 {
-TaskAttemptContext tac = new TaskAttemptContext(jobConf, new 
TaskAttemptID());
+TaskAttemptContext tac = HadoopCompat.newTaskAttem

[jira] [Updated] (CASSANDRA-5201) Cassandra/Hadoop does not support current Hadoop releases

[Benjamin Coverston (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Coverston updated CASSANDRA-5201:
--

Attachment: hadoopcompat-trunk.patch

Attached trunk-rebased patch.

> Cassandra/Hadoop does not support current Hadoop releases
> -
>
> Key: CASSANDRA-5201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5201
> Project: Cassandra
>  Issue Type: Bug
>  Components: Hadoop
>Affects Versions: 1.2.0
>Reporter: Brian Jeltema
>Assignee: Benjamin Coverston
> Attachments: 5201_a.txt, hadoopCompat.patch, hadoopcompat-trunk.patch
>
>
> Using Hadoop 0.22.0 with Cassandra results in the stack trace below.
> It appears that version 0.21+ changed org.apache.hadoop.mapreduce.JobContext
> from a class to an interface.
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found 
> interface org.apache.hadoop.mapreduce.JobContext, but class was expected
>   at 
> org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSplits(ColumnFamilyInputFormat.java:103)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:357)
>   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1045)
>   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1042)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1042)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1062)
>   at MyHadoopApp.run(MyHadoopApp.java:163)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
>   at MyHadoopApp.main(MyHadoopApp.java:82)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:192)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (CASSANDRA-6683) BADNESS_THRESHOLD does not working correctly with DynamicEndpointSnitch

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams resolved CASSANDRA-6683.
-

Resolution: Not A Problem

> BADNESS_THRESHOLD does not working correctly with DynamicEndpointSnitch
> ---
>
> Key: CASSANDRA-6683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6683
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Linux 3.8.0-33-generic
>Reporter: Kirill Bogdanov
>  Labels: snitch
> Fix For: 2.0.6
>
>
> There is a problem in *DynamicEndpointSnitch.java* in 
> sortByProximityWithBadness()
> Before calling sortByProximityWithScore we comparing each nodes score ratios 
> to the badness threshold.
> {code}
> if ((first - next) / first >  BADNESS_THRESHOLD)
> {
> sortByProximityWithScore(address, addresses);
> return;
> }
> {code}
> This is not always the correct comparison because *first* score can be less 
> than *next*  score and in that case we will compare a negative number with 
> positive.
> The solution is to compute absolute value of the ratio:
> {code}
> if (Math.abs((first - next) / first) > BADNESS_THRESHOLD)
> {code}
> This issue causing an incorrect sorting of DCs based on their performance and 
> affects performance of the snitch.
> Thanks.
>  



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6683) BADNESS_THRESHOLD does not working correctly with DynamicEndpointSnitch

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898320#comment-13898320
 ] 

Tyler Hobbs commented on CASSANDRA-6683:


CASSANDRA-6465 doesn't affect what [~kirill.sc] is referring to.  However, when 
the score for {{first}} is less than {{next}}, the DES is behaving correctly.  
Although {{first - next / first}} results in a negative number, that result is 
less than {{BADNESS_THRESHOLD}}, which results in the DES saying that {{first}} 
should be used.  This is the correct behavior because {{first}} has a lower 
score (less "badness") than {{next}}.

The whole point of this check is that the DES should only use something other 
than {{first}} if the score for {{next}} is lower (less bad) than the score for 
{{first}} by a certain margin (BADNESS_THRESHOLD).

> BADNESS_THRESHOLD does not working correctly with DynamicEndpointSnitch
> ---
>
> Key: CASSANDRA-6683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6683
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Linux 3.8.0-33-generic
>Reporter: Kirill Bogdanov
>  Labels: snitch
> Fix For: 2.0.6
>
>
> There is a problem in *DynamicEndpointSnitch.java* in 
> sortByProximityWithBadness()
> Before calling sortByProximityWithScore we comparing each nodes score ratios 
> to the badness threshold.
> {code}
> if ((first - next) / first >  BADNESS_THRESHOLD)
> {
> sortByProximityWithScore(address, addresses);
> return;
> }
> {code}
> This is not always the correct comparison because *first* score can be less 
> than *next*  score and in that case we will compare a negative number with 
> positive.
> The solution is to compute absolute value of the ratio:
> {code}
> if (Math.abs((first - next) / first) > BADNESS_THRESHOLD)
> {code}
> This issue causing an incorrect sorting of DCs based on their performance and 
> affects performance of the snitch.
> Thanks.
>  



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[Cassandra Wiki] Update of "Committers" by JonathanEllis

[Apache Wiki]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for 
change notification.

The "Committers" page has been changed by JonathanEllis:
https://wiki.apache.org/cassandra/Committers?action=diff&rev1=37&rev2=38

  ||Gary Dusbabek||Dec 2009||Rackspace||PMC member||
  ||Jaakko Laine||Dec 2009||?|| ||
  ||Brandon Williams||Jun 2010||Datastax||PMC member||
- ||Jake Luciani||Jan 2011||!BlueMountain||Also a 
[[http://thrift.apache.org/|Thrift]] PMC member||
+ ||Jake Luciani||Jan 2011||!BlueMountain||PMC member, 
[[http://thrift.apache.org/|Thrift]] PMC member||
  ||Sylvain Lebresne||Mar 2011||Datastax||PMC member, Release manager||
  ||Pavel Yaskevich||Aug 2011||Apple||PMC member||
  ||Vijay Parthasarathy||Jan 2012||Apple|| ||

[jira] [Commented] (CASSANDRA-5266) FileNotFoundException during AntiEntropy Stream Session after compaction

[Yuki Morishita (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898288#comment-13898288
 ] 

Yuki Morishita commented on CASSANDRA-5266:
---

[~arya] What compaction strategy are you using?

> FileNotFoundException during AntiEntropy Stream Session after compaction
> 
>
> Key: CASSANDRA-5266
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5266
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Affects Versions: 1.1.6
> Environment: Ubuntu 10.04
> Cassandra 1.2.14
> Jave 7 64 Bit Sun JVM
>Reporter: Arya Goudarzi
>Assignee: Yuki Morishita
>
> It seems that there is a race condition in the compaction process vs 
> AntiEntropy Streams. We often have repairs tossing lots of exceptions in the 
> logs as follows:
> java.lang.RuntimeException: java.io.FileNotFoundException: 
> /var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db
>  (No such file or directory)
> at 
> org.apache.cassandra.utils.FBUtilities.unchecked(FBUtilities.java:628)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:34)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> Caused by: java.io.FileNotFoundException: 
> /var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db
>  (No such file or directory)
> at java.io.RandomAccessFile.open(Native Method)
> at java.io.RandomAccessFile.(RandomAccessFile.java:216)
> at 
> org.apache.cassandra.io.util.RandomAccessReader.(RandomAccessReader.java:67)
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.(CompressedRandomAccessReader.java:64)
> at 
> org.apache.cassandra.io.compress.CompressedRandomAccessReader.open(CompressedRandomAccessReader.java:46)
> at 
> org.apache.cassandra.streaming.FileStreamTask.stream(FileStreamTask.java:145)
> at 
> org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTask.java:94)
> at 
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> ... 3 more
> Now about 1-2 minutes before that, I see Secondary Index buildup and 
> compaction which does compact the same SSTable to a new one. 
> INFO [Thread-90838] 2013-02-07 13:51:58,362 SecondaryIndexManager.java (line 
> 126) Submitting index build of publisher_id, for data in 
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20054-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20055-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20057-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20058-Data.db')
>  INFO [CompactionExecutor:436] 2013-02-07 13:51:58,363 CompactionTask.java 
> (line 109) Compacting 
> [SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20055-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20054-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20058-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20057-Data.db'),
>  
> SSTableReader(path='/var/lib/cassandra/data/keyspace_production/User/keyspace_production-User-hf-20056-Data.db')]
> Now, does this make the repair sessions to fail? It seems the node was trying 
> to stream a file to a node being repaired but apparently it did not.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6673) Compaction assertion error on beforeAppend test due to decommision - add node and/or overlapping sstables

[Ravi Prasad (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898255#comment-13898255
 ] 

Ravi Prasad commented on CASSANDRA-6673:


I think this can be marked as dup of 
https://issues.apache.org/jira/browse/CASSANDRA-6688.  

> Compaction assertion error on beforeAppend test due to decommision - add node 
> and/or overlapping sstables
> -
>
> Key: CASSANDRA-6673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6673
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: linux, replication factor 1, 4 node, LCS, 3 compaction 
> executors
>Reporter: Ignace Desimpel
>Priority: Minor
> Fix For: 2.0.6
>
> Attachments: beforeAppend.txt
>
>
> Compaction assertion on test for LastWrittenKey compared to current key to 
> write during compaction. This happens after a decommission of a node, and 
> then the node is added again immediately (short time after decommssion). This 
> makes the node shutdown. Restart of node a couple of time finally finishes 
> the bootstrapping. During restarts to finish the bootstrapping there is also 
> a warning message about overlapping sstables



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6683) BADNESS_THRESHOLD does not working correctly with DynamicEndpointSnitch

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-6683:
-

Fix Version/s: (was: 2.0.5)
   2.0.6

> BADNESS_THRESHOLD does not working correctly with DynamicEndpointSnitch
> ---
>
> Key: CASSANDRA-6683
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6683
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: Linux 3.8.0-33-generic
>Reporter: Kirill Bogdanov
>  Labels: snitch
> Fix For: 2.0.6
>
>
> There is a problem in *DynamicEndpointSnitch.java* in 
> sortByProximityWithBadness()
> Before calling sortByProximityWithScore we comparing each nodes score ratios 
> to the badness threshold.
> {code}
> if ((first - next) / first >  BADNESS_THRESHOLD)
> {
> sortByProximityWithScore(address, addresses);
> return;
> }
> {code}
> This is not always the correct comparison because *first* score can be less 
> than *next*  score and in that case we will compare a negative number with 
> positive.
> The solution is to compute absolute value of the ratio:
> {code}
> if (Math.abs((first - next) / first) > BADNESS_THRESHOLD)
> {code}
> This issue causing an incorrect sorting of DCs based on their performance and 
> affects performance of the snitch.
> Thanks.
>  



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6673) Compaction assertion error on beforeAppend test due to decommision - add node and/or overlapping sstables

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-6673:
-

Fix Version/s: (was: 2.0.4)
   2.0.6

> Compaction assertion error on beforeAppend test due to decommision - add node 
> and/or overlapping sstables
> -
>
> Key: CASSANDRA-6673
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6673
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
> Environment: linux, replication factor 1, 4 node, LCS, 3 compaction 
> executors
>Reporter: Ignace Desimpel
>Priority: Minor
> Fix For: 2.0.6
>
> Attachments: beforeAppend.txt
>
>
> Compaction assertion on test for LastWrittenKey compared to current key to 
> write during compaction. This happens after a decommission of a node, and 
> then the node is added again immediately (short time after decommssion). This 
> makes the node shutdown. Restart of node a couple of time finally finishes 
> the bootstrapping. During restarts to finish the bootstrapping there is also 
> a warning message about overlapping sstables



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6561) Static columns in CQL3

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898173#comment-13898173
 ] 

Aleksey Yeschenko commented on CASSANDRA-6561:
--

Still not a full review, just things that don't work:

- ALTER TABLE ADD should support adding a static column, but doesn't
- dropping a static column doesn't work fully (it won't be compacted away)

> Static columns in CQL3
> --
>
> Key: CASSANDRA-6561
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6561
> Project: Cassandra
>  Issue Type: New Feature
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
> Fix For: 2.0.6
>
>
> I'd like to suggest the following idea for adding "static" columns to CQL3.  
> I'll note that the basic idea has been suggested by jhalliday on irc but the 
> rest of the details are mine and I should be blamed for anything stupid in 
> what follows.
> Let me start with a rational: there is 2 main family of CF that have been 
> historically used in Thrift: static ones and dynamic ones. CQL3 handles both 
> family through the presence or not of clustering columns. There is however 
> some cases where mixing both behavior has its use. I like to think of those 
> use cases as 3 broad category:
> # to denormalize small amounts of not-entirely-static data in otherwise 
> static entities. It's say "tags" for a product or "custom properties" in a 
> user profile. This is why we've added CQL3 collections. Importantly, this is 
> the *only* use case for which collections are meant (which doesn't diminishes 
> their usefulness imo, and I wouldn't disagree that we've maybe not 
> communicated this too well).
> # to optimize fetching both a static entity and related dynamic ones. Say you 
> have blog posts, and each post has associated comments (chronologically 
> ordered). *And* say that a very common query is "fetch a post and its 50 last 
> comments". In that case, it *might* be beneficial to store a blog post 
> (static entity) in the same underlying CF than it's comments for performance 
> reason.  So that "fetch a post and it's 50 last comments" is just one slice 
> internally.
> # you want to CAS rows of a dynamic partition based on some partition 
> condition. This is the same use case than why CASSANDRA-5633 exists for.
> As said above, 1) is already covered by collections, but 2) and 3) are not 
> (and
> I strongly believe collections are not the right fit, API wise, for those).
> Also, note that I don't want to underestimate the usefulness of 2). In most 
> cases, using a separate table for the blog posts and the comments is The 
> Right Solution, and trying to do 2) is premature optimisation. Yet, when used 
> properly, that kind of optimisation can make a difference, so I think having 
> a relatively native solution for it in CQL3 could make sense.
> Regarding 3), though CASSANDRA-5633 would provide one solution for it, I have 
> the feeling that static columns actually are a more natural approach (in term 
> of API). That's arguably more of a personal opinion/feeling though.
> So long story short, CQL3 lacks a way to mix both some "static" and "dynamic" 
> rows in the same partition of the same CQL3 table, and I think such a tool 
> could have it's use.
> The proposal is thus to allow "static" columns. Static columns would only 
> make sense in table with clustering columns (the "dynamic" ones). A static 
> column value would be static to the partition (all rows of the partition 
> would share the value for such column). The syntax would just be:
> {noformat}
> CREATE TABLE t (
>   k text,
>   s text static,
>   i int,
>   v text,
>   PRIMARY KEY (k, i)
> )
> {noformat}
> then you'd get:
> {noformat}
> INSERT INTO t(k, s, i, v) VALUES ("k0", "I'm shared",   0, "foo");
> INSERT INTO t(k, s, i, v) VALUES ("k0", "I'm still shared", 1, "bar");
> SELECT * FROM t;
>  k |  s | i |v
> 
> k0 | "I'm still shared" | 0 | "bar"
> k0 | "I'm still shared" | 1 | "foo"
> {noformat}
> There would be a few semantic details to decide on regarding deletions, ttl, 
> etc. but let's see if we agree it's a good idea first before ironing those 
> out.
> One last point is the implementation. Though I do think this idea has merits, 
> it's definitively not useful enough to justify rewriting the storage engine 
> for it. But I think we can support this relatively easily (emphasis on 
> "relatively" :)), which is probably the main reason why I like the approach.
> Namely, internally, we can store static columns as cells whose clustering 
> column values are empty. So in terms of cells, the partition of my example 
> would look like:
> {noformat}
> "k0" : [
>   (:"s" -> "I'm still shared"), // the static column
>   (0:"" -> "")  // row marker
>   (0:"

[jira] [Commented] (CASSANDRA-6590) Gossip does not heal after a temporary partition at startup

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898166#comment-13898166
 ] 

Brandon Williams commented on CASSANDRA-6590:
-

Hmm, so I went to test v4 and this time the ring weirdness came back, so 
perhaps it's just intermittent, but here's what it looks like.  There are three 
nodes, 10.208.8.123, 10.208.35.225, and 10.208.8.63.  123 is the seed, all 
nodes were started at the same time, and 63 is blocked from 225.  The log from 
123 looks normal:

{noformat}
 INFO 19:04:26,900 Handshaking version with /10.208.8.63
 INFO 19:04:26,913 Node /10.208.8.63 is now part of the cluster
 INFO 19:04:26,923 Handshaking version with /10.208.8.63
 INFO 19:04:26,963 Node bw-1/10.208.8.123 state jump to normal
 INFO 19:04:27,004 Startup completed! Now serving reads.
 INFO 19:04:27,076 Waiting for gossip to settle before accepting client 
requests...
 INFO 19:04:27,091 Handshaking version with /10.208.35.225
 INFO 19:04:27,097 Compacted 4 sstables to 
[/var/lib/cassandra/data/system/local/system-local-jb-5,].  5,846 bytes to 
5,684 (~97% of original) in 250ms = 0.021683MB/s.  4 total partitions merged to 
1.  Partition merge counts were {4:1, }
 INFO 19:04:27,100 Node /10.208.35.225 is now part of the cluster
 INFO 19:04:27,102 Handshaking version with /10.208.35.225
 INFO 19:04:35,190 Starting listening for CQL clients on 
bw-1/10.208.8.123:9042...
 INFO 19:04:35,252 Using TFramedTransport with a max frame size of 15728640 
bytes.
 INFO 19:04:35,253 Binding thrift service to bw-1/10.208.8.123:9160
 INFO 19:04:35,262 Using synchronous/threadpool thrift server on bw-1 : 9160
 INFO 19:04:35,262 Listening for thrift clients...
{noformat}

And it can see the other nodes in status, but can't tell their state:

{noformat}
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  AddressLoad   Tokens  Owns (effective)  Host ID 
  Rack
UN  10.208.8.123   40.9 KB256 68.1% 
fa02838d-c39b-4d44-90db-f21a359deb12  rack1
?N  10.208.8.6340.85 KB   256 63.0% 
90e71b90-9b41-4482-9521-71ba479c964e  rack1
?N  10.208.35.225  40.93 KB   256 68.9% 
e2fe818d-5d6c-47f9-8015-4580254cb91f  rack1
{noformat}


The other two nodes can't even see anything but themselves:

{noformat}
Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  AddressLoad   Tokens  Owns (effective)  Host ID 
  Rack
UN  10.208.35.225  40.93 KB   256 100.0%
e2fe818d-5d6c-47f9-8015-4580254cb91f  rack1

Datacenter: datacenter1
===
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address  Load   Tokens  Owns (effective)  Host ID   
Rack
UN  10.208.8.63  40.85 KB   256 100.0%
90e71b90-9b41-4482-9521-71ba479c964e  rack1
{noformat}

Even though both are fully connected to the seed:

{noformat}
tcp0  0 10.208.8.123:57215  10.208.8.63:7000ESTABLISHED 
16517/java  
tcp0  0 10.208.8.123:7000   10.208.8.63:37973   ESTABLISHED 
16517/java  
tcp0  0 10.208.8.123:7000   10.208.35.225:41926 ESTABLISHED 
16517/java  
tcp0  0 10.208.8.123:59308  10.208.35.225:7000  ESTABLISHED 
16517/java
{noformat}

I'm not sure what's going on here yet.

> Gossip does not heal after a temporary partition at startup
> ---
>
> Key: CASSANDRA-6590
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6590
> Project: Cassandra
>  Issue Type: Bug
>  Components: Core
>Reporter: Brandon Williams
>Assignee: Vijay
> Fix For: 2.0.6
>
> Attachments: 0001-CASSANDRA-6590.patch, 0001-logging-for-6590.patch, 
> 6590_disable_echo.txt
>
>
> See CASSANDRA-6571 for background.  If a node is partitioned on startup when 
> the echo command is sent, but then the partition heals, the halves of the 
> partition will never mark each other up despite being able to communicate.  
> This stems from CASSANDRA-3533.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6649) CQL: disallow counter update with "USING TIMESTAMP" and "USING TTL"

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898104#comment-13898104
 ] 

Sylvain Lebresne commented on CASSANDRA-6649:
-

For the 2.0 version, I really think we should have a longer, more explanatory 
message, since users will get that out of context in the log. Something like: 
"Detected use of 'USING TIMESTAMP' in counter UPDATE. This is invalid since 
counters do not use timestamps and the timestamp has been ignored. Such query 
will be rejected starting from Cassandra 2.1 so you should fix your query.". 
But lgtm otherwise.

Note: I would also commit that in 1.2 too because really, that's just a warning 
and it's never too early to warn user that it's not doing what they probably 
think it does (if we end up not doing any more 1.2 release, there is still not 
harm having committed it). I'm not extremely strong on that though.

> CQL: disallow counter update with "USING TIMESTAMP" and "USING TTL"
> ---
>
> Key: CASSANDRA-6649
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6649
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
>Priority: Trivial
> Fix For: 2.0.6, 2.1
>
> Attachments: 6649-2.0.txt, 6649-2.1.txt
>
>
> Timestamps are not used by counters and TTL are not supported, but it appears 
> we don't reject counter updates that have "USING TIMESTAMP X" or "USING TTL 
> X". We should since both are non-sensical (the value is completely ignored 
> currently).
> Note: we should also refuse "USING TIMESTAMP" on "DELETE" statements on 
> counters table: even though we kind of do use a timestamp internally, it's 
> more of an implementation detail and in fact may go away with CASSANDRA-6506 
> (there is also nothing clever you can do with it by providing it client side).
> Note bis: strictly speaking doing that could break a few users that where 
> setting those thinking it does something. I think that the lack of validation 
> is more of a bug and that user that think it's doing something probably ought 
> to know it's not sooner than later, but I could be fine with just warning in 
> the log file for 1.2 and 2.0, and only rejecting in 2.1 if someone thinks 
> it's safer.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6662) Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898101#comment-13898101
 ] 

Benedict commented on CASSANDRA-6662:
-

bq. It should have a significant enough effect on 
CollationController.collectTimeOrderedData()

I'm not totally convinced (I'm not totally unconvinced either, mind). 
1) it's likely we'll add columns that are not present many times as well, here, 
so we'll end up sorting anyway; and
2) I think the best thing to do here is to optimise addAll() so that if the 
source is an ABSC it uses the merge logic of sort() (which could be split out, 
and the columns being merged could be parameterized), and then the addAtom() 
calls - which will always be unique, and in-order, will always construct a 
sorted list we can efficiently merge. No binary search.

Not to say getting rid of binarySearch is still a good thing, but I think these 
optimisations would be better for this spot. 

I think we should also look into optimising the selection of names from the 
source. We shouldn't be constructing iterators, and shouldn't be performing 
binary search over the full range for every column. But that's another ticket.

> Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is 
> called
> 
>
> Key: CASSANDRA-6662
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6662
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 2.1
>
>
> To avoid poor performance with huge numbers of cells added out of order 
> (which should be rare, but *can* happen with certain batch scenarios) we 
> should make ABSC only sort/reconcile its cells when an actual accessor is 
> actually called, delaying sorting until the very end.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6649) CQL: disallow counter update with "USING TIMESTAMP" and "USING TTL"

[Aleksey Yeschenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-6649:
-

Priority: Trivial  (was: Minor)

> CQL: disallow counter update with "USING TIMESTAMP" and "USING TTL"
> ---
>
> Key: CASSANDRA-6649
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6649
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sylvain Lebresne
>Assignee: Aleksey Yeschenko
>Priority: Trivial
> Fix For: 2.0.6, 2.1
>
> Attachments: 6649-2.0.txt, 6649-2.1.txt
>
>
> Timestamps are not used by counters and TTL are not supported, but it appears 
> we don't reject counter updates that have "USING TIMESTAMP X" or "USING TTL 
> X". We should since both are non-sensical (the value is completely ignored 
> currently).
> Note: we should also refuse "USING TIMESTAMP" on "DELETE" statements on 
> counters table: even though we kind of do use a timestamp internally, it's 
> more of an implementation detail and in fact may go away with CASSANDRA-6506 
> (there is also nothing clever you can do with it by providing it client side).
> Note bis: strictly speaking doing that could break a few users that where 
> setting those thinking it does something. I think that the lack of validation 
> is more of a bug and that user that think it's doing something probably ought 
> to know it's not sooner than later, but I could be fine with just warning in 
> the log file for 1.2 and 2.0, and only rejecting in 2.1 if someone thinks 
> it's safer.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Comment Edited] (CASSANDRA-6662) Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898085#comment-13898085
 ] 

Aleksey Yeschenko edited comment on CASSANDRA-6662 at 2/11/14 6:06 PM:
---

Accepted nits #1, #2, #4, #5 and committed, thanks.

As for #3 - got rid of the binarySearch() there, but did leave the 
reconcile-with-previous-cell logic in place (for one, we've already done the 
comparison, so it makes sense to use the result; secondly, it does happen more 
often than you think - when merging results from memtabe+sstables on read in 
CollationController.collectAllData()).

Actually, returned the binarySearch() logic back as well. It should have a 
significant enough effect on CollationController.collectTimeOrderedData(), and 
only adds one more branch on top, anyway (in 
6de2fd9bfd4c20e47495d171da7175c1a9e92c68).


was (Author: iamaleksey):
Accepted nits #1, #2, #4 and #5 and committed, thanks.

As for #3 - got rid of the binarySearch() there, but did leave the 
reconcile-with-previous-cell logic in place (for one, we've already done the 
comparison, so it makes sense to use the result; secondly, it does happen more 
often than you think - when merging results from memtabe+sstables on read in 
CollationController.collectAllData).

Actually, returned the binarySearch() logic back as well. It should have a 
significant enough affect on CollationController.collectTimeOrderedData(), and 
only adds one more branch on top, anyway (in 
6de2fd9bfd4c20e47495d171da7175c1a9e92c68).

> Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is 
> called
> 
>
> Key: CASSANDRA-6662
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6662
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 2.1
>
>
> To avoid poor performance with huge numbers of cells added out of order 
> (which should be rare, but *can* happen with certain batch scenarios) we 
> should make ABSC only sort/reconcile its cells when an actual accessor is 
> actually called, delaying sorting until the very end.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Comment Edited] (CASSANDRA-6662) Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898085#comment-13898085
 ] 

Aleksey Yeschenko edited comment on CASSANDRA-6662 at 2/11/14 6:05 PM:
---

Accepted nits #1, #2, #4 and #5 and committed, thanks.

As for #3 - got rid of the binarySearch() there, but did leave the 
reconcile-with-previous-cell logic in place (for one, we've already done the 
comparison, so it makes sense to use the result; secondly, it does happen more 
often than you think - when merging results from memtabe+sstables on read in 
CollationController.collectAllData).

Actually, returned the binarySearch() logic back as well. It should have a 
significant enough affect on CollationController.collectTimeOrderedData(), and 
only adds one more branch on top, anyway (in 
6de2fd9bfd4c20e47495d171da7175c1a9e92c68).


was (Author: iamaleksey):
Accepted nits #1, #2, #4 and #5 and committed, thanks.

As for #4 - got rid of the binarySearch() there, but did leave the 
reconcile-with-previous-cell logic in place (for one, we've already done the 
comparison, so it makes sense to use the result; secondly, it does happen more 
often than you think - when merging results from memtabe+sstables on read in 
CollationController.collectAllData).

Actually, returned the binarySearch() logic back as well. It should have a 
significant enough affect on CollationController.collectTimeOrderedData(), and 
only adds one more branch on top, anyway (in 
6de2fd9bfd4c20e47495d171da7175c1a9e92c68).

> Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is 
> called
> 
>
> Key: CASSANDRA-6662
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6662
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 2.1
>
>
> To avoid poor performance with huge numbers of cells added out of order 
> (which should be rare, but *can* happen with certain batch scenarios) we 
> should make ABSC only sort/reconcile its cells when an actual accessor is 
> actually called, delaying sorting until the very end.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6662) Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898085#comment-13898085
 ] 

Aleksey Yeschenko commented on CASSANDRA-6662:
--

Accepted nits #1, #2, #4 and #5 and committed, thanks.

As for #4 - got rid of the binarySearch() there, but did leave the 
reconcile-with-previous-cell logic in place (for one, we've already done the 
comparison, so it makes sense to use the result; secondly, it does happen more 
often than you think - when merging results from memtabe+sstables on read in 
CollationController.collectAllData).

Actually, returned the binarySearch() logic back as well. It should have a 
significant enough affect on CollationController.collectTimeOrderedData(), and 
only adds one more branch on top, anyway (in 
6de2fd9bfd4c20e47495d171da7175c1a9e92c68).

> Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is 
> called
> 
>
> Key: CASSANDRA-6662
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6662
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 2.1
>
>
> To avoid poor performance with huge numbers of cells added out of order 
> (which should be rare, but *can* happen with certain batch scenarios) we 
> should make ABSC only sort/reconcile its cells when an actual accessor is 
> actually called, delaying sorting until the very end.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

git commit: Optimize CollationController.collectTimeOrderedData post CASSANDRA-6662

[aleksey]

Updated Branches:
  refs/heads/trunk 520457502 -> 6de2fd9bf


Optimize CollationController.collectTimeOrderedData post CASSANDRA-6662


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6de2fd9b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6de2fd9b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6de2fd9b

Branch: refs/heads/trunk
Commit: 6de2fd9bfd4c20e47495d171da7175c1a9e92c68
Parents: 5204575
Author: Aleksey Yeschenko 
Authored: Tue Feb 11 21:04:13 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Tue Feb 11 21:04:13 2014 +0300

--
 .../cassandra/db/ArrayBackedSortedColumns.java  |  7 --
 .../cassandra/db/CollationController.java   | 26 ++--
 2 files changed, 12 insertions(+), 21 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6de2fd9b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java 
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index a6153e4..8049d87 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -186,8 +186,11 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 }
 else
 {
-// Append to the end, making cells unsorted from now on
-internalAdd(cell);
+int pos = binarySearch(cell.name());
+if (pos >= 0) // Reconcile with an existing cell
+reconcileWith(pos, cell);
+else
+internalAdd(cell); // Append to the end, making cells unsorted 
from now on
 }
 }
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/6de2fd9b/src/java/org/apache/cassandra/db/CollationController.java
--
diff --git a/src/java/org/apache/cassandra/db/CollationController.java 
b/src/java/org/apache/cassandra/db/CollationController.java
index 3fd3e8b..e5d3495 100644
--- a/src/java/org/apache/cassandra/db/CollationController.java
+++ b/src/java/org/apache/cassandra/db/CollationController.java
@@ -62,16 +62,10 @@ public class CollationController
 private ColumnFamily collectTimeOrderedData()
 {
 final ColumnFamily container = 
ArrayBackedSortedColumns.factory.create(cfs.metadata, 
filter.filter.isReversed());
-List iterators = new 
ArrayList();
+List iterators = new ArrayList<>();
 Tracing.trace("Acquiring sstable references");
 ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key);
 
-// We use a temporary CF object per memtable or sstable source so we 
can accomodate this.factory being ABSC,
-// which requires addAtom to happen in sorted order.  Then we use 
addAll to merge into the final collection,
-// which allows a (sorted) set of columns to be merged even if they 
are not uniformly sorted after the existing
-// ones.
-ColumnFamily temp = 
ArrayBackedSortedColumns.factory.create(cfs.metadata, 
filter.filter.isReversed());
-
 try
 {
 Tracing.trace("Merging memtable contents");
@@ -81,13 +75,10 @@ public class CollationController
 if (iter != null)
 {
 iterators.add(iter);
-temp.delete(iter.getColumnFamily());
+container.delete(iter.getColumnFamily());
 while (iter.hasNext())
-temp.addAtom(iter.next());
+container.addAtom(iter.next());
 }
-
-container.addAll(temp);
-temp.clear();
 }
 
 // avoid changing the filter columns of the original filter
@@ -122,14 +113,11 @@ public class CollationController
 ColumnFamily cf = iter.getColumnFamily();
 if (cf.isMarkedForDelete())
 mostRecentRowTombstone = 
cf.deletionInfo().getTopLevelDeletion().markedForDeleteAt;
-temp.delete(cf);
+container.delete(cf);
 sstablesIterated++;
 while (iter.hasNext())
-temp.addAtom(iter.next());
+container.addAtom(iter.next());
 }
-
-container.addAll(temp);
-temp.clear();
 }
 
 // we need to distinguish between "there is no data at all for 
this row" (BF will let us rebuild that efficiently)
@@ -189,7 +177,7 @@ public class CollationController
 {
 Tracin

[jira] [Commented] (CASSANDRA-4911) Lift limitation that order by columns must be selected for IN queries

[Tyler Hobbs (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-4911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898081#comment-13898081
 ] 

Tyler Hobbs commented on CASSANDRA-4911:


If I define this table:

{noformat}
create table foo (a int, b int, c int, d int, e int, PRIMARY KEY(a, b, c, d));
{noformat}

and run this query:
{noformat}
SELECT e FROM foo WHERE a=1 AND b=1 AND c=1 AND d IN (1, 2) ORDER BY b asc;
{noformat}

I see this AssertionError in the Cassandra log:
{noformat}
java.lang.AssertionError: null
at 
org.apache.cassandra.cql3.statements.SelectStatement.orderResults(SelectStatement.java:948)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.process(SelectStatement.java:863)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.processResults(SelectStatement.java:222)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:202)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:172)
 ~[main/:na]
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:57)
 ~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:180)
 ~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:214) 
~[main/:na]
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:204) 
~[main/:na]
at 
org.apache.cassandra.thrift.CassandraServer.execute_cql3_query(CassandraServer.java:1973)
 ~[main/:na]
at 
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4486)
 ~[thrift/:na]
at 
org.apache.cassandra.thrift.Cassandra$Processor$execute_cql3_query.getResult(Cassandra.java:4470)
 ~[thrift/:na]
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
~[libthrift-0.9.1.jar:0.9.1]
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
~[libthrift-0.9.1.jar:0.9.1]
at 
org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:194)
 ~[main/:na]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 
~[na:1.7.0_40]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 
~[na:1.7.0_40]
at java.lang.Thread.run(Thread.java:724) ~[na:1.7.0_40]
{noformat}

> Lift limitation that order by columns must be selected for IN queries
> -
>
> Key: CASSANDRA-4911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4911
> Project: Cassandra
>  Issue Type: Improvement
>  Components: API
>Affects Versions: 1.2.0 beta 1
>Reporter: Sylvain Lebresne
>Assignee: Sylvain Lebresne
>Priority: Minor
> Fix For: 2.1
>
> Attachments: 4911.txt
>
>
> This is the followup of CASSANDRA-4645. We should remove the limitation that 
> for IN queries, you must have columns on which you have an ORDER BY in the 
> select clause.
> For that, we'll need to automatically add the columns on which we have an 
> ORDER BY to the one queried internally, and remove it afterwards (once the 
> sorting is done) from the resultSet.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[2/3] git commit: Optimize (rewrite) ArrayBackedSortedColumns

[aleksey]

Optimize (rewrite) ArrayBackedSortedColumns

patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-6662


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fb1c6b9c
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fb1c6b9c
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fb1c6b9c

Branch: refs/heads/trunk
Commit: fb1c6b9cded56e63dfcc765515edbc94ee9f67a0
Parents: 0e43885
Author: Aleksey Yeschenko 
Authored: Tue Feb 11 20:30:57 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Tue Feb 11 20:35:31 2014 +0300

--
 CHANGES.txt |   3 +-
 .../cassandra/db/ArrayBackedSortedColumns.java  | 333 +--
 .../db/ArrayBackedSortedColumnsTest.java|  69 
 3 files changed, 305 insertions(+), 100 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/fb1c6b9c/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7f9f9d2..d8478e9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -25,7 +25,8 @@
  * CF id is changed to be non-deterministic. Data dir/key cache are created
uniquely for CF id (CASSANDRA-5202)
  * New counters implementation (CASSANDRA-6504)
- * Replace UnsortedColumns usage with ArrayBackedSortedColumns (CASSANDRA-6630)
+ * Replace UnsortedColumns and TreeMapBackedSortedColumns with rewritten
+   ArrayBackedSortedColumns (CASSANDRA-6630, CASSANDRA-6662)
  * Add option to use row cache with a given amount of rows (CASSANDRA-5357)
  * Avoid repairing already repaired data (CASSANDRA-5351)
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/fb1c6b9c/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java 
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index ba082e9..a6153e4 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -31,16 +31,23 @@ import org.apache.cassandra.db.composites.Composite;
 import org.apache.cassandra.db.filter.ColumnSlice;
 
 /**
- * A ColumnFamily backed by an ArrayList.
+ * A ColumnFamily backed by an array.
  * This implementation is not synchronized and should only be used when
  * thread-safety is not required. This implementation makes sense when the
- * main operations performed are iterating over the map and adding cells
+ * main operations performed are iterating over the cells and adding cells
  * (especially if insertion is in sorted order).
  */
 public class ArrayBackedSortedColumns extends AbstractThreadUnsafeSortedColumns
 {
+private static final int INITIAL_CAPACITY = 10;
+
+private static final Cell[] EMPTY_ARRAY = new Cell[0];
+
 private final boolean reversed;
-private final ArrayList cells;
+
+private Cell[] cells;
+private int size;
+private int sortedSize;
 
 public static final ColumnFamily.Factory factory 
= new Factory()
 {
@@ -54,14 +61,18 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 {
 super(metadata);
 this.reversed = reversed;
-this.cells = new ArrayList<>();
+this.cells = new Cell[INITIAL_CAPACITY];
+this.size = 0;
+this.sortedSize = 0;
 }
 
-private ArrayBackedSortedColumns(Collection cells, CFMetaData 
metadata, boolean reversed)
+private ArrayBackedSortedColumns(ArrayBackedSortedColumns original)
 {
-super(metadata);
-this.reversed = reversed;
-this.cells = new ArrayList<>(cells);
+super(original.metadata);
+this.reversed = original.reversed;
+this.cells = Arrays.copyOf(original.cells, original.size);
+this.size = original.size;
+this.sortedSize = original.sortedSize;
 }
 
 public ColumnFamily.Factory getFactory()
@@ -71,7 +82,7 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 
 public ColumnFamily cloneMe()
 {
-return new ArrayBackedSortedColumns(cells, metadata, reversed);
+return new ArrayBackedSortedColumns(this);
 }
 
 public boolean isInsertReversed()
@@ -84,72 +95,191 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 return reversed ? getComparator().reverseComparator() : 
getComparator();
 }
 
+private void maybeSortCells()
+{
+if (size != sortedSize)
+sortCells();
+}
+
+private void sortCells()
+{
+Comparator comparator = reversed
+? getComparator().columnReverseComparator()

[3/3] git commit: ColumnFamily-related cleanups

[aleksey]

ColumnFamily-related cleanups

patch by Aleksey Yeschenko; reviewed by Benedict Elliott Smith for
CASSANDRA-6662


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0e43885a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0e43885a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0e43885a

Branch: refs/heads/trunk
Commit: 0e43885ab24768177cc1e758e1ab69b6dc1b23db
Parents: 5a3ffc0
Author: Aleksey Yeschenko 
Authored: Sun Feb 9 06:22:47 2014 +0300
Committer: Aleksey Yeschenko 
Committed: Tue Feb 11 20:35:31 2014 +0300

--
 .../cassandra/db/ArrayBackedSortedColumns.java  | 56 +---
 .../apache/cassandra/db/AtomicBTreeColumns.java | 15 +-
 .../cassandra/db/CollationController.java   |  5 +-
 .../org/apache/cassandra/db/ColumnFamily.java   | 50 ++---
 .../apache/cassandra/db/ColumnFamilyStore.java  | 23 
 .../org/apache/cassandra/db/EmptyColumns.java   | 18 ++-
 .../db/TreeMapBackedSortedColumns.java  | 47 ++--
 .../cassandra/db/index/keys/KeysSearcher.java   |  6 +--
 .../db/ArrayBackedSortedColumnsTest.java| 25 -
 .../apache/cassandra/db/ColumnFamilyTest.java   | 15 +++---
 10 files changed, 58 insertions(+), 202 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0e43885a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
--
diff --git a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java 
b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
index b81e403..ba082e9 100644
--- a/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
+++ b/src/java/org/apache/cassandra/db/ArrayBackedSortedColumns.java
@@ -29,7 +29,6 @@ import org.apache.cassandra.db.composites.CellName;
 import org.apache.cassandra.db.composites.CellNameType;
 import org.apache.cassandra.db.composites.Composite;
 import org.apache.cassandra.db.filter.ColumnSlice;
-import org.apache.cassandra.utils.memory.AbstractAllocator;
 
 /**
  * A ColumnFamily backed by an ArrayList.
@@ -55,14 +54,14 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 {
 super(metadata);
 this.reversed = reversed;
-this.cells = new ArrayList();
+this.cells = new ArrayList<>();
 }
 
 private ArrayBackedSortedColumns(Collection cells, CFMetaData 
metadata, boolean reversed)
 {
 super(metadata);
 this.reversed = reversed;
-this.cells = new ArrayList(cells);
+this.cells = new ArrayList<>(cells);
 }
 
 public ColumnFamily.Factory getFactory()
@@ -91,7 +90,7 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 return pos >= 0 ? cells.get(pos) : null;
 }
 
-public void addColumn(Cell cell, AbstractAllocator allocator)
+public void addColumn(Cell cell)
 {
 if (cells.isEmpty())
 {
@@ -109,13 +108,13 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 else if (c == 0)
 {
 // Resolve against last
-resolveAgainst(getColumnCount() - 1, cell, allocator);
+resolveAgainst(getColumnCount() - 1, cell);
 }
 else
 {
 int pos = binarySearch(cell.name());
 if (pos >= 0)
-resolveAgainst(pos, cell, allocator);
+resolveAgainst(pos, cell);
 else
 cells.add(-pos - 1, cell);
 }
@@ -125,13 +124,9 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
  * Resolve against element at position i.
  * Assume that i is a valid position.
  */
-private void resolveAgainst(int i, Cell cell, AbstractAllocator allocator)
+private void resolveAgainst(int i, Cell cell)
 {
-Cell oldCell = cells.get(i);
-
-// calculate reconciled col from old (existing) col and new col
-Cell reconciledCell = cell.reconcile(oldCell, allocator);
-cells.set(i, reconciledCell);
+cells.set(i, cell.reconcile(cells.get(i)));
 }
 
 private int binarySearch(CellName name)
@@ -155,22 +150,16 @@ public class ArrayBackedSortedColumns extends 
AbstractThreadUnsafeSortedColumns
 {
 mid = (low + high) >> 1;
 if ((result = comparator.compare(name, cells.get(mid).name())) > 0)
-{
 low = mid + 1;
-}
 else if (result == 0)
-{
 return mid;
-}
 else
-{
 high = mid - 1;
-}
 }
 return -mid - (result < 0 ? 1 : 2);
 }
 
-public void add

git commit: who forgets to add files? I do.

[brandonwilliams]

Updated Branches:
  refs/heads/trunk 7a6e5f043 -> 5a3ffc090


who forgets to add files? I do.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/5a3ffc09
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/5a3ffc09
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/5a3ffc09

Branch: refs/heads/trunk
Commit: 5a3ffc0904e75152eb3e8fafd0b84ae499a94c77
Parents: 7a6e5f0
Author: Brandon Williams 
Authored: Tue Feb 11 11:30:02 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 11:32:02 2014 -0600

--
 .../cassandra/hadoop/ReporterWrapper.java   | 57 
 1 file changed, 57 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/5a3ffc09/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java 
b/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
new file mode 100644
index 000..9940ba4
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
@@ -0,0 +1,57 @@
+package org.apache.cassandra.hadoop;
+
+import org.apache.hadoop.mapred.Counters;
+import org.apache.hadoop.mapred.InputSplit;
+import org.apache.hadoop.mapred.Reporter;
+import org.apache.hadoop.mapreduce.StatusReporter;
+
+/**
+ * A reporter that works with both mapred and mapreduce APIs.
+ */
+public class ReporterWrapper extends StatusReporter implements Reporter {
+private Reporter wrappedReporter;
+
+public ReporterWrapper(Reporter reporter) {
+wrappedReporter = reporter;
+}
+
+@Override
+public Counters.Counter getCounter(Enum anEnum) {
+return wrappedReporter.getCounter(anEnum);
+}
+
+@Override
+public Counters.Counter getCounter(String s, String s1) {
+return wrappedReporter.getCounter(s, s1);
+}
+
+@Override
+public void incrCounter(Enum anEnum, long l) {
+wrappedReporter.incrCounter(anEnum, l);
+}
+
+@Override
+public void incrCounter(String s, String s1, long l) {
+wrappedReporter.incrCounter(s, s1, l);
+}
+
+@Override
+public InputSplit getInputSplit() throws UnsupportedOperationException {
+return wrappedReporter.getInputSplit();
+}
+
+@Override
+public void progress() {
+wrappedReporter.progress();
+}
+
+// @Override
+public float getProgress() {
+throw new UnsupportedOperationException();
+}
+
+@Override
+public void setStatus(String s) {
+wrappedReporter.setStatus(s);
+}
+}
\ No newline at end of file

git commit: who forgets to add files? I do.

[brandonwilliams]

Updated Branches:
  refs/heads/cassandra-2.0 b9aece45e -> 662f546aa


who forgets to add files? I do.


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/662f546a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/662f546a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/662f546a

Branch: refs/heads/cassandra-2.0
Commit: 662f546aa76b7f6473b8da8b9ea675a9fd9757e1
Parents: b9aece4
Author: Brandon Williams 
Authored: Tue Feb 11 11:30:02 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 11:30:02 2014 -0600

--
 .../cassandra/hadoop/ReporterWrapper.java   | 57 
 1 file changed, 57 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/662f546a/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java 
b/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
new file mode 100644
index 000..9940ba4
--- /dev/null
+++ b/src/java/org/apache/cassandra/hadoop/ReporterWrapper.java
@@ -0,0 +1,57 @@
+package org.apache.cassandra.hadoop;
+
+import org.apache.hadoop.mapred.Counters;
+import org.apache.hadoop.mapred.InputSplit;
+import org.apache.hadoop.mapred.Reporter;
+import org.apache.hadoop.mapreduce.StatusReporter;
+
+/**
+ * A reporter that works with both mapred and mapreduce APIs.
+ */
+public class ReporterWrapper extends StatusReporter implements Reporter {
+private Reporter wrappedReporter;
+
+public ReporterWrapper(Reporter reporter) {
+wrappedReporter = reporter;
+}
+
+@Override
+public Counters.Counter getCounter(Enum anEnum) {
+return wrappedReporter.getCounter(anEnum);
+}
+
+@Override
+public Counters.Counter getCounter(String s, String s1) {
+return wrappedReporter.getCounter(s, s1);
+}
+
+@Override
+public void incrCounter(Enum anEnum, long l) {
+wrappedReporter.incrCounter(anEnum, l);
+}
+
+@Override
+public void incrCounter(String s, String s1, long l) {
+wrappedReporter.incrCounter(s, s1, l);
+}
+
+@Override
+public InputSplit getInputSplit() throws UnsupportedOperationException {
+return wrappedReporter.getInputSplit();
+}
+
+@Override
+public void progress() {
+wrappedReporter.progress();
+}
+
+// @Override
+public float getProgress() {
+throw new UnsupportedOperationException();
+}
+
+@Override
+public void setStatus(String s) {
+wrappedReporter.setStatus(s);
+}
+}
\ No newline at end of file

git commit: fix AntiCompacitonTest

[yukim]

Updated Branches:
  refs/heads/trunk e58737fe0 -> 7a6e5f043


fix AntiCompacitonTest


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7a6e5f04
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7a6e5f04
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7a6e5f04

Branch: refs/heads/trunk
Commit: 7a6e5f043670353848fa1373656445c5ca8e4671
Parents: e58737f
Author: Yuki Morishita 
Authored: Tue Feb 11 11:16:54 2014 -0600
Committer: Yuki Morishita 
Committed: Tue Feb 11 11:16:54 2014 -0600

--
 src/java/org/apache/cassandra/db/compaction/CompactionManager.java | 2 +-
 .../org/apache/cassandra/db/compaction/AntiCompactionTest.java | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/7a6e5f04/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java 
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index b964c5b..effaf1c 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -302,7 +302,7 @@ public class CompactionManager implements 
CompactionManagerMBean
  *
  * @param cfs
  * @param ranges Ranges that the repair was carried out on
- * @param validatedForRepair SSTables containing the repaired ranges
+ * @param validatedForRepair SSTables containing the repaired ranges. 
Should be referenced before passing them.
  * @throws InterruptedException, ExecutionException, IOException
  */
 public void performAnticompaction(ColumnFamilyStore cfs,

http://git-wip-us.apache.org/repos/asf/cassandra/blob/7a6e5f04/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java
--
diff --git 
a/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java 
b/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java
index 164dd62..fb87f80 100644
--- a/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java
+++ b/test/unit/org/apache/cassandra/db/compaction/AntiCompactionTest.java
@@ -70,6 +70,7 @@ public class AntiCompactionTest extends SchemaLoader
 Range range = new Range(new BytesToken("0".getBytes()), 
new BytesToken("4".getBytes()));
 List> ranges = Arrays.asList(range);
 
+SSTableReader.acquireReferences(sstables);
 long repairedAt = 1000;
 CompactionManager.instance.performAnticompaction(store, ranges, 
sstables, repairedAt);
 

[jira] [Comment Edited] (CASSANDRA-6662) Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897989#comment-13897989
 ] 

Benedict edited comment on CASSANDRA-6662 at 2/11/14 4:35 PM:
--

On the whole LGTM. A few suggestions you're free to ignore:

- internalAppend/AndReconcile should be colocated with internalAdd etc
- in maybeSort(), leftCopy could be optimised to assign a static EMPTY array 
for the typical case of being called when the whole thing is unsorted
- I don't think there much to be gained by binary search in addColumn; I would 
simply attempt a fast sorted append (no reconcile), and if that fails just 
perform an unsorted append and let it clean up next time. It may be that we 
sometimes overwrite the same values in a CF, but I'm not sure that it happens 
frequently enough to warrant the complexity.
- May be worth extracting the actual sorting from maybeSort() into sort() and 
calling it conditionally, as it is currently too long to be inlined. This way 
the conditional should be inlined so we don't incur the method invoke cost 
except when we have to. I wouldn't ordinarily worry about this, but we call 
these methods extensively.
- I have a slight preference, when merging two arrays, to use the three-loop 
method instead of the one loop with three branches of an if statement. I think 
it is clearer what is happening. Just a suggestion though.

I've pushed a version with my suggestions to 
[iss-6662|https://github.com/belliottsmith/cassandra/tree/iss-6662-2]




was (Author: benedict):
On the whole LGTM. A few suggestions you're free to ignore:

- internalAppend/AndReconcile should be colocated with internalAdd etc
- in maybeSort(), leftCopy could be optimised to assign a static EMPTY array 
for the typical case of being called when the whole thing is unsorted
- I don't think there much to be gained by binary search in addColumn; I would 
simply attempt a fast sorted append (no reconcile), and if that fails just 
perform an unsorted append and let it clean up next time. It may be that we 
sometimes overwrite the same values in a CF, but I'm not sure that it happens 
frequently enough to warrant the complexity.
- addColumn can use internalAppend instead of internalAdd
- May be worth extracting the actual sorting from maybeSort() into sort() and 
calling it conditionally, as it is currently too long to be inlined. This way 
the conditional should be inlined so we don't incur the method invoke cost 
except when we have to. I wouldn't ordinarily worry about this, but we call 
these methods extensively.
- I have a slight preference, when merging two arrays, to use the three-loop 
method instead of the one loop with three branches of an if statement. I think 
it is clearer what is happening. Just a suggestion though.

I've pushed a version with my suggestions to 
[iss-6662|https://github.com/belliottsmith/cassandra/tree/iss-6662-2]



> Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is 
> called
> 
>
> Key: CASSANDRA-6662
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6662
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 2.1
>
>
> To avoid poor performance with huge numbers of cells added out of order 
> (which should be rare, but *can* happen with certain batch scenarios) we 
> should make ABSC only sort/reconcile its cells when an actual accessor is 
> actually called, delaying sorting until the very end.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6662) Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is called

[Benedict (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897989#comment-13897989
 ] 

Benedict commented on CASSANDRA-6662:
-

On the whole LGTM. A few suggestions you're free to ignore:

- internalAppend/AndReconcile should be colocated with internalAdd etc
- in maybeSort(), leftCopy could be optimised to assign a static EMPTY array 
for the typical case of being called when the whole thing is unsorted
- I don't think there much to be gained by binary search in addColumn; I would 
simply attempt a fast sorted append (no reconcile), and if that fails just 
perform an unsorted append and let it clean up next time. It may be that we 
sometimes overwrite the same values in a CF, but I'm not sure that it happens 
frequently enough to warrant the complexity.
- addColumn can use internalAppend instead of internalAdd
- May be worth extracting the actual sorting from maybeSort() into sort() and 
calling it conditionally, as it is currently too long to be inlined. This way 
the conditional should be inlined so we don't incur the method invoke cost 
except when we have to. I wouldn't ordinarily worry about this, but we call 
these methods extensively.
- I have a slight preference, when merging two arrays, to use the three-loop 
method instead of the one loop with three branches of an if statement. I think 
it is clearer what is happening. Just a suggestion though.

I've pushed a version with my suggestions to 
[iss-6662|https://github.com/belliottsmith/cassandra/tree/iss-6662-2]



> Sort/reconcile cells in ArrayBackedSortedColumns only when an accessor is 
> called
> 
>
> Key: CASSANDRA-6662
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6662
> Project: Cassandra
>  Issue Type: Improvement
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
> Fix For: 2.1
>
>
> To avoid poor performance with huge numbers of cells added out of order 
> (which should be rare, but *can* happen with certain batch scenarios) we 
> should make ABSC only sort/reconcile its cells when an actual accessor is 
> actually called, delaying sorting until the very end.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6622) Streaming session failures during node replace of same address

[Brandon Williams (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-6622:


Attachment: 6622-v2.txt

So the problem isn't any gossip event breaking the stream, it's that streaming 
is subscribed to the FD directly in order to wait  for double the phi, but this 
notifies every second regardless of up/down state and ends up racing the stream 
request before the hibernate state has reached the source node.  We won't be 
able to fix that without CASSANDRA-3569, so we'll add the RING_DELAY sleep.  v2 
conditionally determines sleep length based on whether the IP is the same or 
not.

> Streaming session failures during node replace of same address
> --
>
> Key: CASSANDRA-6622
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6622
> Project: Cassandra
>  Issue Type: Bug
> Environment: RHEL6, cassandra-2.0.4
>Reporter: Ravi Prasad
>Assignee: Brandon Williams
> Fix For: 1.2.16, 2.0.6
>
> Attachments: 0001-don-t-signal-restart-of-dead-states.txt, 
> 6622-2.0.txt, 6622-v2.txt, 6622_logs.tgz, logs.tgz
>
>
> When using replace_address, Gossiper ApplicationState is set to hibernate, 
> which is a down state. We are seeing that the peer nodes are seeing streaming 
> plan request even before the Gossiper on them marks the replacing node as 
> dead. As a result, streaming on peer nodes convicts the replacing node by 
> closing the stream handler.  
> I think, making the StorageService thread on the replacing node, sleep for 
> BROADCAST_INTERVAL before bootstrapping, would avoid this scenario.
> Relevant logs from peer node (see that the Gossiper on peer node mark the 
> replacing node as down, 2 secs after  the streaming init request):
> {noformat}
>  INFO [STREAM-INIT-/x.x.x.x:46436] 2014-01-26 20:42:24,388 
> StreamResultFuture.java (line 116) [Stream 
> #5c6cd940-86ca-11e3-90a0-411b913c0e88] Received streaming plan for Bootstrap
> 
>  INFO [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with /x.x.x.x is 
> complete
>  WARN [GossipTasks:1] 2014-01-26 20:42:25,240 StreamResultFuture.java (line 
> 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
>  INFO [GossipStage:1] 2014-01-26 20:42:25,242 Gossiper.java (line 850) 
> InetAddress /x.x.x.x is now DOWN
> ERROR [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,766 StreamSession.java (line 
> 410) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Streaming error occurred
> java.lang.RuntimeException: Outgoing stream handler has been closed
> at 
> org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:175)
> at 
> org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:436)
> at 
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:358)
> at 
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
> at java.lang.Thread.run(Thread.java:722)
>  INFO [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 181) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Session with 
> /x.x.x.x is complete
>  WARN [STREAM-IN-/x.x.x.x] 2014-01-26 20:42:25,768 StreamResultFuture.java 
> (line 210) [Stream #5c6cd940-86ca-11e3-90a0-411b913c0e88] Stream failed
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-5839) Save repair data to system table

[Aleksey Yeschenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897971#comment-13897971
 ] 

Aleksey Yeschenko commented on CASSANDRA-5839:
--

or maybe system_shared

> Save repair data to system table
> 
>
> Key: CASSANDRA-5839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5839
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core, Tools
>Reporter: Jonathan Ellis
>Assignee: Jimmy Mårdell
>Priority: Minor
> Fix For: 2.0.6
>
> Attachments: 2.0.4-5839-draft.patch, 2.0.6-5839-v2.patch
>
>
> As noted in CASSANDRA-2405, it would be useful to store repair results, 
> particularly with sub-range repair available (CASSANDRA-5280).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6688) Avoid possible sstable overlaps with leveled compaction

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897960#comment-13897960
 ] 

Jonathan Ellis commented on CASSANDRA-6688:
---

that's definitely the simplest solution

> Avoid possible sstable overlaps with leveled compaction
> ---
>
> Key: CASSANDRA-6688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6688
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Attachments: 0001-6688.patch, 6688-v2.txt
>
>
> Two cases where we can end up with overlapping sstables in the leveled 
> manifest;
> FIrst one is when we skip levels during compaction. Here we need to make sure 
> we are not compacting in newLevel - 1 since if, for example, we are doing a 
> L1 -> L2 compaction and then start a new L0 compaction where we decide to 
> skip L1, we could have overlapping sstables in L2 when the compactions are 
> done. This case is new in 2.0 since we check if we skip levels before the 
> compaction starts.
> Second case is where we try to include as many overlapping L0 sstables as 
> possible, here we could add sstables that are not compacting, but overlap 
> sstables that are.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-6688) Avoid possible sstable overlaps with leveled compaction

[Jonathan Ellis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-6688:
--

Attachment: 6688-v2.txt

> Avoid possible sstable overlaps with leveled compaction
> ---
>
> Key: CASSANDRA-6688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6688
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Attachments: 0001-6688.patch, 6688-v2.txt
>
>
> Two cases where we can end up with overlapping sstables in the leveled 
> manifest;
> FIrst one is when we skip levels during compaction. Here we need to make sure 
> we are not compacting in newLevel - 1 since if, for example, we are doing a 
> L1 -> L2 compaction and then start a new L0 compaction where we decide to 
> skip L1, we could have overlapping sstables in L2 when the compactions are 
> done. This case is new in 2.0 since we check if we skip levels before the 
> compaction starts.
> Second case is where we try to include as many overlapping L0 sstables as 
> possible, here we could add sstables that are not compacting, but overlap 
> sstables that are.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6688) Avoid possible sstable overlaps with leveled compaction

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897959#comment-13897959
 ] 

Marcus Eriksson commented on CASSANDRA-6688:


.bq This affects 1.2 as well right?
yeah, looks like it

Should we perhaps stop skipping levels? Can't really see it happening more than 
once in the lifetime of a cluster, right after bootstrap (or after dropping all 
the sstables to L0), first compaction might skip L1.

> Avoid possible sstable overlaps with leveled compaction
> ---
>
> Key: CASSANDRA-6688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6688
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Attachments: 0001-6688.patch, 6688-v2.txt
>
>
> Two cases where we can end up with overlapping sstables in the leveled 
> manifest;
> FIrst one is when we skip levels during compaction. Here we need to make sure 
> we are not compacting in newLevel - 1 since if, for example, we are doing a 
> L1 -> L2 compaction and then start a new L0 compaction where we decide to 
> skip L1, we could have overlapping sstables in L2 when the compactions are 
> done. This case is new in 2.0 since we check if we skip levels before the 
> compaction starts.
> Second case is where we try to include as many overlapping L0 sstables as 
> possible, here we could add sstables that are not compacting, but overlap 
> sstables that are.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Comment Edited] (CASSANDRA-6688) Avoid possible sstable overlaps with leveled compaction

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897959#comment-13897959
 ] 

Marcus Eriksson edited comment on CASSANDRA-6688 at 2/11/14 4:02 PM:
-

bq. This affects 1.2 as well right?
yeah, looks like it

Should we perhaps stop skipping levels? Can't really see it happening more than 
once in the lifetime of a cluster, right after bootstrap (or after dropping all 
the sstables to L0), first compaction might skip L1.


was (Author: krummas):
.bq This affects 1.2 as well right?
yeah, looks like it

Should we perhaps stop skipping levels? Can't really see it happening more than 
once in the lifetime of a cluster, right after bootstrap (or after dropping all 
the sstables to L0), first compaction might skip L1.

> Avoid possible sstable overlaps with leveled compaction
> ---
>
> Key: CASSANDRA-6688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6688
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Attachments: 0001-6688.patch, 6688-v2.txt
>
>
> Two cases where we can end up with overlapping sstables in the leveled 
> manifest;
> FIrst one is when we skip levels during compaction. Here we need to make sure 
> we are not compacting in newLevel - 1 since if, for example, we are doing a 
> L1 -> L2 compaction and then start a new L0 compaction where we decide to 
> skip L1, we could have overlapping sstables in L2 when the compactions are 
> done. This case is new in 2.0 since we check if we skip levels before the 
> compaction starts.
> Second case is where we try to include as many overlapping L0 sstables as 
> possible, here we could add sstables that are not compacting, but overlap 
> sstables that are.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Updated] (CASSANDRA-5839) Save repair data to system table

[JIRA]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimmy Mårdell updated CASSANDRA-5839:
-

Attachment: 2.0.6-5839-v2.patch

> Save repair data to system table
> 
>
> Key: CASSANDRA-5839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5839
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core, Tools
>Reporter: Jonathan Ellis
>Assignee: Jimmy Mårdell
>Priority: Minor
> Fix For: 2.0.6
>
> Attachments: 2.0.4-5839-draft.patch, 2.0.6-5839-v2.patch
>
>
> As noted in CASSANDRA-2405, it would be useful to store repair results, 
> particularly with sub-range repair available (CASSANDRA-5280).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-5839) Save repair data to system table

[JIRA]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897944#comment-13897944
 ] 

Jimmy Mårdell commented on CASSANDRA-5839:
--

There you go, new patch ready for review. Sorry for the delay!

> Save repair data to system table
> 
>
> Key: CASSANDRA-5839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5839
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Core, Tools
>Reporter: Jonathan Ellis
>Assignee: Jimmy Mårdell
>Priority: Minor
> Fix For: 2.0.6
>
> Attachments: 2.0.4-5839-draft.patch, 2.0.6-5839-v2.patch
>
>
> As noted in CASSANDRA-2405, it would be useful to store repair results, 
> particularly with sub-range repair available (CASSANDRA-5280).



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6688) Avoid possible sstable overlaps with leveled compaction

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897932#comment-13897932
 ] 

Jonathan Ellis commented on CASSANDRA-6688:
---

v2 attached that generalizes the skipLevel approach a bit, but I think we have 
a problem: what we really want to know is (as I edited the comment) whether 
sstables being compacted *into* newLevel might overlap, but I don't think we 
actually track that anywhere.  (Maybe we could walk the in-progress 
CompactionTasks but that is tricky since once we're out of 
getCompactionCandidates we don't have {{synchronized}} to keep things 
race-free.)  So, if we had two compactions skipping levels, we could still get 
into trouble if we're just checking newlevel-1.

> Avoid possible sstable overlaps with leveled compaction
> ---
>
> Key: CASSANDRA-6688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6688
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Attachments: 0001-6688.patch
>
>
> Two cases where we can end up with overlapping sstables in the leveled 
> manifest;
> FIrst one is when we skip levels during compaction. Here we need to make sure 
> we are not compacting in newLevel - 1 since if, for example, we are doing a 
> L1 -> L2 compaction and then start a new L0 compaction where we decide to 
> skip L1, we could have overlapping sstables in L2 when the compactions are 
> done. This case is new in 2.0 since we check if we skip levels before the 
> compaction starts.
> Second case is where we try to include as many overlapping L0 sstables as 
> possible, here we could add sstables that are not compacting, but overlap 
> sstables that are.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6688) Avoid possible sstable overlaps with leveled compaction

[Jonathan Ellis (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897933#comment-13897933
 ] 

Jonathan Ellis commented on CASSANDRA-6688:
---

bq. Second case is where we try to include as many overlapping L0 sstables as 
possible, here we could add sstables that are not compacting, but overlap 
sstables that are.

This affects 1.2 as well right?

> Avoid possible sstable overlaps with leveled compaction
> ---
>
> Key: CASSANDRA-6688
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6688
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Attachments: 0001-6688.patch
>
>
> Two cases where we can end up with overlapping sstables in the leveled 
> manifest;
> FIrst one is when we skip levels during compaction. Here we need to make sure 
> we are not compacting in newLevel - 1 since if, for example, we are doing a 
> L1 -> L2 compaction and then start a new L0 compaction where we decide to 
> skip L1, we could have overlapping sstables in L2 when the compactions are 
> done. This case is new in 2.0 since we check if we skip levels before the 
> compaction starts.
> Second case is where we try to include as many overlapping L0 sstables as 
> possible, here we could add sstables that are not compacting, but overlap 
> sstables that are.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-5201) Cassandra/Hadoop does not support current Hadoop releases

[Brandon Williams (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-5201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897929#comment-13897929
 ] 

Brandon Williams commented on CASSANDRA-5201:
-

Committed to 2.0.  [~bcoverston] can you rebase a patch for trunk?

> Cassandra/Hadoop does not support current Hadoop releases
> -
>
> Key: CASSANDRA-5201
> URL: https://issues.apache.org/jira/browse/CASSANDRA-5201
> Project: Cassandra
>  Issue Type: Bug
>  Components: Hadoop
>Affects Versions: 1.2.0
>Reporter: Brian Jeltema
>Assignee: Benjamin Coverston
> Attachments: 5201_a.txt, hadoopCompat.patch
>
>
> Using Hadoop 0.22.0 with Cassandra results in the stack trace below.
> It appears that version 0.21+ changed org.apache.hadoop.mapreduce.JobContext
> from a class to an interface.
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found 
> interface org.apache.hadoop.mapreduce.JobContext, but class was expected
>   at 
> org.apache.cassandra.hadoop.ColumnFamilyInputFormat.getSplits(ColumnFamilyInputFormat.java:103)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:445)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462)
>   at 
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:357)
>   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1045)
>   at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1042)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:415)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1153)
>   at org.apache.hadoop.mapreduce.Job.submit(Job.java:1042)
>   at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1062)
>   at MyHadoopApp.run(MyHadoopApp.java:163)
>   at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
>   at MyHadoopApp.main(MyHadoopApp.java:82)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:601)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:192)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[3/3] git commit: Merge branch 'cassandra-2.0' into trunk

[brandonwilliams]

Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e58737fe
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e58737fe
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e58737fe

Branch: refs/heads/trunk
Commit: e58737fe0809e82596e890bf58b2c4b657062143
Parents: a6e7d14 b9aece4
Author: Brandon Williams 
Authored: Tue Feb 11 09:36:10 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 09:36:10 2014 -0600

--

--

[1/3] git commit: Use EB HadoopCompat for compat with Hadoop 0.2.x Patch by Ben Coversaton, reviewed by brandonwilliams for CASSANDRA-5201

[brandonwilliams]

Updated Branches:
  refs/heads/cassandra-2.0 9381b8d56 -> b9aece45e
  refs/heads/trunk a6e7d1405 -> e58737fe0


Use EB HadoopCompat for compat with Hadoop 0.2.x
Patch by Ben Coversaton, reviewed by brandonwilliams for CASSANDRA-5201


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b9aece45
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b9aece45
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b9aece45

Branch: refs/heads/cassandra-2.0
Commit: b9aece45e913fb20f9e04f3c5fdd02622ebb7898
Parents: 9381b8d
Author: Brandon Williams 
Authored: Tue Feb 11 09:34:49 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 09:34:49 2014 -0600

--
 CHANGES.txt   |  1 +
 build.xml |  3 +++
 .../hadoop/AbstractColumnFamilyInputFormat.java   | 11 ++-
 .../hadoop/AbstractColumnFamilyOutputFormat.java  |  3 ++-
 .../apache/cassandra/hadoop/BulkOutputFormat.java |  3 ++-
 .../apache/cassandra/hadoop/BulkRecordWriter.java |  3 ++-
 .../cassandra/hadoop/ColumnFamilyInputFormat.java | 17 +
 .../hadoop/ColumnFamilyRecordReader.java  |  3 ++-
 .../hadoop/ColumnFamilyRecordWriter.java  |  3 ++-
 .../hadoop/cql3/CqlPagingInputFormat.java | 18 ++
 .../hadoop/cql3/CqlPagingRecordReader.java|  5 +++--
 .../cassandra/hadoop/cql3/CqlRecordWriter.java|  3 ++-
 .../cassandra/hadoop/pig/CassandraStorage.java|  5 +++--
 .../apache/cassandra/hadoop/pig/CqlStorage.java   |  5 +++--
 14 files changed, 50 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b9aece45/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a8114a3..16cbd0a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.6
+ * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
  * Fix EstimatedHistogram races (CASSANDRA-6682)
  * Failure detector correctly converts initial value to nanos (CASSANDRA-6658)
  * Add nodetool taketoken to relocate vnodes (CASSANDRA-4445)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b9aece45/build.xml
--
diff --git a/build.xml b/build.xml
index bfbd8b2..fb774f6 100644
--- a/build.xml
+++ b/build.xml
@@ -367,6 +367,7 @@
   
   
   
+  
   
   
 
@@ -408,6 +409,7 @@
 
 

+
 
 
 
@@ -470,6 +472,7 @@
 
 
 
+
 
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b9aece45/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java 
b/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
index 6c45a35..f547fd0 100644
--- a/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
@@ -29,6 +29,7 @@ import java.util.concurrent.TimeUnit;
 
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.Lists;
+import com.twitter.elephantbird.util.HadoopCompat;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -117,16 +118,16 @@ public abstract class AbstractColumnFamilyInputFormat extends InputFormat<
 
 public List getSplits(JobContext context) throws IOException
 {
-Configuration conf = context.getConfiguration();
+Configuration conf = HadoopCompat.getConfiguration(context);;
 
 validateConfiguration(conf);
 
 // cannonical ranges and nodes holding replicas
 List masterRangeNodes = getRangeMap(conf);
 
-keyspace = ConfigHelper.getInputKeyspace(context.getConfiguration());
-cfName = ConfigHelper.getInputColumnFamily(context.getConfiguration());
-partitioner = 
ConfigHelper.getInputPartitioner(context.getConfiguration());
+keyspace = ConfigHelper.getInputKeyspace(conf);
+cfName = ConfigHelper.getInputColumnFamily(conf);
+partitioner = ConfigHelper.getInputPartitioner(conf);
 logger.debug("partitioner is " + partitioner);
 
 
@@ -344,7 +345,7 @@ public abstract class AbstractColumnFamilyInputFormat 
extends InputFormat<
 //
 public org.apache.hadoop.mapred.InputSplit[] getSplits(JobConf jobConf, 
int numSplits) throws IOException
 {
-TaskAttemptContext tac = new TaskAttemptContext(jobConf, new 
TaskAttemptID());
+TaskAttemptContext tac = HadoopCompat.newTaskAttemptContext(jobCon

[2/3] git commit: Use EB HadoopCompat for compat with Hadoop 0.2.x Patch by Ben Coversaton, reviewed by brandonwilliams for CASSANDRA-5201

[brandonwilliams]

Use EB HadoopCompat for compat with Hadoop 0.2.x
Patch by Ben Coversaton, reviewed by brandonwilliams for CASSANDRA-5201


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b9aece45
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b9aece45
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b9aece45

Branch: refs/heads/trunk
Commit: b9aece45e913fb20f9e04f3c5fdd02622ebb7898
Parents: 9381b8d
Author: Brandon Williams 
Authored: Tue Feb 11 09:34:49 2014 -0600
Committer: Brandon Williams 
Committed: Tue Feb 11 09:34:49 2014 -0600

--
 CHANGES.txt   |  1 +
 build.xml |  3 +++
 .../hadoop/AbstractColumnFamilyInputFormat.java   | 11 ++-
 .../hadoop/AbstractColumnFamilyOutputFormat.java  |  3 ++-
 .../apache/cassandra/hadoop/BulkOutputFormat.java |  3 ++-
 .../apache/cassandra/hadoop/BulkRecordWriter.java |  3 ++-
 .../cassandra/hadoop/ColumnFamilyInputFormat.java | 17 +
 .../hadoop/ColumnFamilyRecordReader.java  |  3 ++-
 .../hadoop/ColumnFamilyRecordWriter.java  |  3 ++-
 .../hadoop/cql3/CqlPagingInputFormat.java | 18 ++
 .../hadoop/cql3/CqlPagingRecordReader.java|  5 +++--
 .../cassandra/hadoop/cql3/CqlRecordWriter.java|  3 ++-
 .../cassandra/hadoop/pig/CassandraStorage.java|  5 +++--
 .../apache/cassandra/hadoop/pig/CqlStorage.java   |  5 +++--
 14 files changed, 50 insertions(+), 33 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b9aece45/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index a8114a3..16cbd0a 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 2.0.6
+ * Add compatibility for Hadoop 0.2.x (CASSANDRA-5201)
  * Fix EstimatedHistogram races (CASSANDRA-6682)
  * Failure detector correctly converts initial value to nanos (CASSANDRA-6658)
  * Add nodetool taketoken to relocate vnodes (CASSANDRA-4445)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b9aece45/build.xml
--
diff --git a/build.xml b/build.xml
index bfbd8b2..fb774f6 100644
--- a/build.xml
+++ b/build.xml
@@ -367,6 +367,7 @@
   
   
   
+  
   
   
 
@@ -408,6 +409,7 @@
 
 

+
 
 
 
@@ -470,6 +472,7 @@
 
 
 
+
 
 
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b9aece45/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
--
diff --git 
a/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java 
b/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
index 6c45a35..f547fd0 100644
--- a/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
+++ b/src/java/org/apache/cassandra/hadoop/AbstractColumnFamilyInputFormat.java
@@ -29,6 +29,7 @@ import java.util.concurrent.TimeUnit;
 
 import com.google.common.collect.ImmutableList;
 import com.google.common.collect.Lists;
+import com.twitter.elephantbird.util.HadoopCompat;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -117,16 +118,16 @@ public abstract class AbstractColumnFamilyInputFormat extends InputFormat<
 
 public List getSplits(JobContext context) throws IOException
 {
-Configuration conf = context.getConfiguration();
+Configuration conf = HadoopCompat.getConfiguration(context);;
 
 validateConfiguration(conf);
 
 // cannonical ranges and nodes holding replicas
 List masterRangeNodes = getRangeMap(conf);
 
-keyspace = ConfigHelper.getInputKeyspace(context.getConfiguration());
-cfName = ConfigHelper.getInputColumnFamily(context.getConfiguration());
-partitioner = 
ConfigHelper.getInputPartitioner(context.getConfiguration());
+keyspace = ConfigHelper.getInputKeyspace(conf);
+cfName = ConfigHelper.getInputColumnFamily(conf);
+partitioner = ConfigHelper.getInputPartitioner(conf);
 logger.debug("partitioner is " + partitioner);
 
 
@@ -344,7 +345,7 @@ public abstract class AbstractColumnFamilyInputFormat 
extends InputFormat<
 //
 public org.apache.hadoop.mapred.InputSplit[] getSplits(JobConf jobConf, 
int numSplits) throws IOException
 {
-TaskAttemptContext tac = new TaskAttemptContext(jobConf, new 
TaskAttemptID());
+TaskAttemptContext tac = HadoopCompat.newTaskAttemptContext(jobConf, 
new TaskAttemptID());
 List newInputSplits = 
this.getSplits(tac);
 org.apache.hadoop.mapred.InputSp

[jira] [Commented] (CASSANDRA-6404) Tools emit ERRORs and WARNINGs about missing javaagent

[Daniel Smedegaard Buus (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897798#comment-13897798
 ] 

Daniel Smedegaard Buus commented on CASSANDRA-6404:
---

Righty-O, thanks :)

> Tools emit ERRORs and WARNINGs about missing javaagent 
> ---
>
> Key: CASSANDRA-6404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6404
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tools
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Minor
> Fix For: 1.2.14, 2.0.4
>
> Attachments: 0001-Set-javaagent-when-running-tools-in-bin.patch
>
>
> The combination of CASSANDRA-6107 & CASSANDRA-6293 has lead to a number of 
> the tools shipped in bin/ to display the following warnings when run:
> {code}
> ERROR 15:21:47,337 Unable to initialize MemoryMeter (jamm not specified as 
> javaagent).  This means Cassandra will be unable to measure object sizes 
> accurately and may consequently OOM.
>  WARN 15:21:47,506 MemoryMeter uninitialized (jamm not specified as java 
> agent); KeyCache size in JVM Heap will not be calculated accurately. Usually 
> this means cassandra-env.sh disabled jamm because you are u
> {code}
> Although harmless, these are a bit disconcerting. The simplest fix seems to 
> be to set the javaagent switch as we do for the main C* launch.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6364) There should be different disk_failure_policies for data and commit volumes or commit volume failure should always cause node exit

[Marcus Eriksson (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897791#comment-13897791
 ] 

Marcus Eriksson commented on CASSANDRA-6364:


+1 on the 2.0-patch, committed

> There should be different disk_failure_policies for data and commit volumes 
> or commit volume failure should always cause node exit
> --
>
> Key: CASSANDRA-6364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6364
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: JBOD, single dedicated commit disk
>Reporter: J. Ryan Earl
>Assignee: Benedict
> Fix For: 2.0.6
>
> Attachments: tmp-2.0.patch
>
>
> We're doing fault testing on a pre-production Cassandra cluster.  One of the 
> tests was to simulation failure of the commit volume/disk, which in our case 
> is on a dedicated disk.  We expected failure of the commit volume to be 
> handled somehow, but what we found was that no action was taken by Cassandra 
> when the commit volume fail.  We simulated this simply by pulling the 
> physical disk that backed the commit volume, which resulted in filesystem I/O 
> errors on the mount point.
> What then happened was that the Cassandra Heap filled up to the point that it 
> was spending 90% of its time doing garbage collection.  No errors were logged 
> in regards to the failed commit volume.  Gossip on other nodes in the cluster 
> eventually flagged the node as down.  Gossip on the local node showed itself 
> as up, and all other nodes as down.
> The most serious problem was that connections to the coordinator on this node 
> became very slow due to the on-going GC, as I assume uncommitted writes piled 
> up on the JVM heap.  What we believe should have happened is that Cassandra 
> should have caught the I/O error and exited with a useful log message, or 
> otherwise done some sort of useful cleanup.  Otherwise the node goes into a 
> sort of Zombie state, spending most of its time in GC, and thus slowing down 
> any transactions that happen to use the coordinator on said node.
> A limit on in-memory, unflushed writes before refusing requests may also 
> work.  Point being, something should be done to handle the commit volume 
> dying as doing nothing results in affecting the entire cluster.  I should 
> note, we are using: disk_failure_policy: best_effort



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[1/2] git commit: Add commit_failure_policy.

[marcuse]

Updated Branches:
  refs/heads/trunk 7604935ef -> a6e7d1405


Add commit_failure_policy.

Patch by belliottsmith, reviewed by marcuse for CASSANDRA-6364

CASSANDRA-6364


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9381b8d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9381b8d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9381b8d5

Branch: refs/heads/trunk
Commit: 9381b8d569ae17cf2760bca266b5253e4bcd6ac2
Parents: 55b5605
Author: Marcus Eriksson 
Authored: Tue Feb 11 13:13:37 2014 +0100
Committer: Marcus Eriksson 
Committed: Tue Feb 11 13:13:37 2014 +0100

--
 CHANGES.txt |  2 +-
 conf/cassandra.yaml |  8 +
 .../org/apache/cassandra/config/Config.java |  8 +
 .../cassandra/config/DatabaseDescriptor.java| 11 ++
 .../BatchCommitLogExecutorService.java  | 17 +++--
 .../cassandra/db/commitlog/CommitLog.java   | 24 +
 .../db/commitlog/CommitLogAllocator.java| 37 +---
 .../PeriodicCommitLogExecutorService.java   | 26 --
 .../org/apache/cassandra/io/util/FileUtils.java | 20 ++-
 .../cassandra/service/StorageService.java   | 19 ++
 .../org/apache/cassandra/db/CommitLogTest.java  | 32 +
 11 files changed, 169 insertions(+), 35 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 93552ef..a8114a3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -9,13 +9,13 @@
  * Account for range/row tombstones in tombstone drop
time histogram (CASSANDRA-6522)
  * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
+ * Make commitlog failure handling configurable (CASSANDRA-6364)
 Merged from 1.2:
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
  * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
 
-
 2.0.5
  * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609)
  * Add ks.cf names to tombstone logging (CASSANDRA-6597)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index db924bb..bfe60c4 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -121,6 +121,14 @@ commitlog_directory: /var/lib/cassandra/commitlog
 # ignore: ignore fatal errors and let requests fail, as in pre-1.2 Cassandra
 disk_failure_policy: stop
 
+# policy for commit disk failures:
+# stop: shut down gossip and Thrift, leaving the node effectively dead, but
+#   can still be inspected via JMX.
+# stop_commit: shutdown the commit log, letting writes collect but 
+#  continuing to service reads, as in pre-2.0.5 Cassandra
+# ignore: ignore fatal errors and let the batches fail
+commit_failure_policy: stop
+
 # Maximum size of the key cache in memory.
 #
 # Each key cache hit saves 1 seek and each row cache hit saves 2 seeks at the

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index a4e4e92..2fa49f3 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -45,6 +45,7 @@ public class Config
 public DiskAccessMode disk_access_mode = DiskAccessMode.auto;
 
 public DiskFailurePolicy disk_failure_policy = DiskFailurePolicy.ignore;
+public CommitFailurePolicy commit_failure_policy = 
CommitFailurePolicy.stop;
 
 /* initial token in the ring */
 public String initial_token;
@@ -230,6 +231,13 @@ public class Config
 ignore,
 }
 
+public static enum CommitFailurePolicy
+{
+stop,
+stop_commit,
+ignore,
+}
+
 public static enum RequestSchedulerId
 {
 keyspace

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index bd5db69..e1a95ab 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/or

git commit: Add commit_failure_policy.

[marcuse]

Updated Branches:
  refs/heads/cassandra-2.0 55b5605b7 -> 9381b8d56


Add commit_failure_policy.

Patch by belliottsmith, reviewed by marcuse for CASSANDRA-6364

CASSANDRA-6364


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9381b8d5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9381b8d5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9381b8d5

Branch: refs/heads/cassandra-2.0
Commit: 9381b8d569ae17cf2760bca266b5253e4bcd6ac2
Parents: 55b5605
Author: Marcus Eriksson 
Authored: Tue Feb 11 13:13:37 2014 +0100
Committer: Marcus Eriksson 
Committed: Tue Feb 11 13:13:37 2014 +0100

--
 CHANGES.txt |  2 +-
 conf/cassandra.yaml |  8 +
 .../org/apache/cassandra/config/Config.java |  8 +
 .../cassandra/config/DatabaseDescriptor.java| 11 ++
 .../BatchCommitLogExecutorService.java  | 17 +++--
 .../cassandra/db/commitlog/CommitLog.java   | 24 +
 .../db/commitlog/CommitLogAllocator.java| 37 +---
 .../PeriodicCommitLogExecutorService.java   | 26 --
 .../org/apache/cassandra/io/util/FileUtils.java | 20 ++-
 .../cassandra/service/StorageService.java   | 19 ++
 .../org/apache/cassandra/db/CommitLogTest.java  | 32 +
 11 files changed, 169 insertions(+), 35 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 93552ef..a8114a3 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -9,13 +9,13 @@
  * Account for range/row tombstones in tombstone drop
time histogram (CASSANDRA-6522)
  * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
+ * Make commitlog failure handling configurable (CASSANDRA-6364)
 Merged from 1.2:
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)
  * Fix mean cells and mean row size per sstable calculations (CASSANDRA-6667)
  * Compact hints after partial replay to clean out tombstones (CASSANDRA-)
 
-
 2.0.5
  * Reduce garbage generated by bloom filter lookups (CASSANDRA-6609)
  * Add ks.cf names to tombstone logging (CASSANDRA-6597)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/conf/cassandra.yaml
--
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index db924bb..bfe60c4 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -121,6 +121,14 @@ commitlog_directory: /var/lib/cassandra/commitlog
 # ignore: ignore fatal errors and let requests fail, as in pre-1.2 Cassandra
 disk_failure_policy: stop
 
+# policy for commit disk failures:
+# stop: shut down gossip and Thrift, leaving the node effectively dead, but
+#   can still be inspected via JMX.
+# stop_commit: shutdown the commit log, letting writes collect but 
+#  continuing to service reads, as in pre-2.0.5 Cassandra
+# ignore: ignore fatal errors and let the batches fail
+commit_failure_policy: stop
+
 # Maximum size of the key cache in memory.
 #
 # Each key cache hit saves 1 seek and each row cache hit saves 2 seeks at the

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/src/java/org/apache/cassandra/config/Config.java
--
diff --git a/src/java/org/apache/cassandra/config/Config.java 
b/src/java/org/apache/cassandra/config/Config.java
index a4e4e92..2fa49f3 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -45,6 +45,7 @@ public class Config
 public DiskAccessMode disk_access_mode = DiskAccessMode.auto;
 
 public DiskFailurePolicy disk_failure_policy = DiskFailurePolicy.ignore;
+public CommitFailurePolicy commit_failure_policy = 
CommitFailurePolicy.stop;
 
 /* initial token in the ring */
 public String initial_token;
@@ -230,6 +231,13 @@ public class Config
 ignore,
 }
 
+public static enum CommitFailurePolicy
+{
+stop,
+stop_commit,
+ignore,
+}
+
 public static enum RequestSchedulerId
 {
 keyspace

http://git-wip-us.apache.org/repos/asf/cassandra/blob/9381b8d5/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
--
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java 
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index bd5db69..e1a95ab 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+

[2/2] git commit: Merge branch 'cassandra-2.0' into trunk

[marcuse]

Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/a6e7d140
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/a6e7d140
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/a6e7d140

Branch: refs/heads/trunk
Commit: a6e7d1405ad34127948538a866ef6fd027bf940d
Parents: 7604935 9381b8d
Author: Marcus Eriksson 
Authored: Tue Feb 11 13:14:28 2014 +0100
Committer: Marcus Eriksson 
Committed: Tue Feb 11 13:14:28 2014 +0100

--

--

[jira] [Updated] (CASSANDRA-6364) There should be different disk_failure_policies for data and commit volumes or commit volume failure should always cause node exit

[Benedict (JIRA)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-6364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benedict updated CASSANDRA-6364:


Attachment: tmp-2.0.patch

Attaching a patch for 2.0 that switches the default policy to stop, and on 
policy ignore will pause the erroring thread for 1s before continuing.

Still not sure this is definitely the correct behaviour for ignore, but it will 
do for now, since ignore is hardly going to be the most recommended setting.

> There should be different disk_failure_policies for data and commit volumes 
> or commit volume failure should always cause node exit
> --
>
> Key: CASSANDRA-6364
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6364
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Core
> Environment: JBOD, single dedicated commit disk
>Reporter: J. Ryan Earl
>Assignee: Benedict
> Fix For: 2.0.6
>
> Attachments: tmp-2.0.patch
>
>
> We're doing fault testing on a pre-production Cassandra cluster.  One of the 
> tests was to simulation failure of the commit volume/disk, which in our case 
> is on a dedicated disk.  We expected failure of the commit volume to be 
> handled somehow, but what we found was that no action was taken by Cassandra 
> when the commit volume fail.  We simulated this simply by pulling the 
> physical disk that backed the commit volume, which resulted in filesystem I/O 
> errors on the mount point.
> What then happened was that the Cassandra Heap filled up to the point that it 
> was spending 90% of its time doing garbage collection.  No errors were logged 
> in regards to the failed commit volume.  Gossip on other nodes in the cluster 
> eventually flagged the node as down.  Gossip on the local node showed itself 
> as up, and all other nodes as down.
> The most serious problem was that connections to the coordinator on this node 
> became very slow due to the on-going GC, as I assume uncommitted writes piled 
> up on the JVM heap.  What we believe should have happened is that Cassandra 
> should have caught the I/O error and exited with a useful log message, or 
> otherwise done some sort of useful cleanup.  Otherwise the node goes into a 
> sort of Zombie state, spending most of its time in GC, and thus slowing down 
> any transactions that happen to use the coordinator on said node.
> A limit on in-memory, unflushed writes before refusing requests may also 
> work.  Point being, something should be done to handle the commit volume 
> dying as doing nothing results in affecting the entire cluster.  I should 
> note, we are using: disk_failure_policy: best_effort



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Commented] (CASSANDRA-6663) Connecting to a Raspberry PI Cassandra Cluster crashes the node being connected to

[Sylvain Lebresne (JIRA)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-6663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13897633#comment-13897633
 ] 

Sylvain Lebresne commented on CASSANDRA-6663:
-

Drivers, including the python driver, are handled separately, they are not part 
of the Cassandra project per-se and thus drivers problems are not tracked here. 
So if it is a python driver problem, which doesn't sound unlikely given you 
don't seem to have problems with DevCenter, you should report it to the 
relevant driver project. Maybe the driver mistakenly asks for Snappy 
compression to the server even though you have asked not too for instance.

> Connecting to a Raspberry PI Cassandra Cluster crashes the node being 
> connected to
> --
>
> Key: CASSANDRA-6663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6663
> Project: Cassandra
>  Issue Type: Bug
>  Components: Drivers (now out of tree)
> Environment: 4x node Raspberry PI cluster
> Macbook running Idle 2.7
>Reporter: ian mccrae
> Attachments: Python Client Log.txt, hs_err_pid6327.log
>
>
> I have a working 4x node Raspberry Pi cluster and
> # DevCenter happily connects to this (...which has an option to turn Snappy 
> compression off)
> # ...however the Python Driver fails to connect and crashes the node being 
> connected to with the errors in the error-log below.
> There appears to be a problem with Snappy compression (not supported on the 
> Raspberry Pi).  So I also tried "compression = None" with the same result.
> How might I fix this?
> *Python Code*
> {noformat}
> >>> from cassandra.cluster import Cluster
> >>> cluster = Cluster(['192.168.200.151'], compression = None)
> >>> session = cluster.connect()
> {noformat}
> *Error Log*
> {noformat}
> Traceback (most recent call last):
>   File "", line 1, in 
> session = cluster.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 471, in connect
> self.control_connection.connect()
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1351, in connect
> self._set_new_connection(self._reconnect_internal())
>   File 
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/cassandra/cluster.py",
>  line 1386, in _reconnect_internal
> raise NoHostAvailable("Unable to connect to any servers", errors)
> NoHostAvailable: ('Unable to connect to any servers', {'192.168.200.151': 
> ConnectionShutdown('Connection to 192.168.200.151 is closed',))
> {noformat}
> *A Dump of the "cluster" class attributes*
> {noformat}
> >>> pprint(vars(cluster))
> {'_core_connections_per_host': {0: 2, 1: 1},
>  '_is_setup': True,
>  '_is_shutdown': True,
>  '_listener_lock': ,
>  '_listeners': set([]),
>  '_lock': <_RLock owner=None count=0>,
>  '_max_connections_per_host': {0: 8, 1: 2},
>  '_max_requests_per_connection': {0: 100, 1: 100},
>  '_min_requests_per_connection': {0: 5, 1: 5},
>  '_prepared_statements': ,
>  'compression': None,
>  'contact_points': ['192.168.200.151'],
>  'control_connection':  0x106168cd0>,
>  'control_connection_timeout': 2.0,
>  'cql_version': None,
>  'executor':  0x106148410>,
>  'load_balancing_policy':  0x104adae50>,
>  'max_schema_agreement_wait': 10,
>  'metadata': ,
>  'metrics_enabled': False,
>  'port': 9042,
>  'scheduler': ,
>  'sessions': <_weakrefset.WeakSet object at 0x106148750>,
>  'sockopts': None,
>  'ssl_options': None}
> >>>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[1/2] git commit: Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer.

[marcuse]

Updated Branches:
  refs/heads/trunk f05870ee8 -> 7604935ef


Stop CommitLogSegment.close() from unnecessarily calling sync() prior to 
cleaning the buffer.

Patch by belliotsmith, reviewed by marcuse for CASSANDRA-6652


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/55b5605b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/55b5605b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/55b5605b

Branch: refs/heads/trunk
Commit: 55b5605b7afcb7ae9bcb9b61959b91502f769db4
Parents: 87aca60
Author: Marcus Eriksson 
Authored: Tue Feb 11 09:18:09 2014 +0100
Committer: Marcus Eriksson 
Committed: Tue Feb 11 09:21:22 2014 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLog.java   |  2 +-
 .../db/commitlog/CommitLogSegment.java  | 74 ++--
 3 files changed, 55 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/55b5605b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b98dec7..93552ef 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -8,6 +8,7 @@
  * Correctly handle null with IF conditions and TTL (CASSANDRA-6623)
  * Account for range/row tombstones in tombstone drop
time histogram (CASSANDRA-6522)
+ * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
 Merged from 1.2:
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/55b5605b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 0e9a3f1..e9507da 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -56,7 +56,7 @@ public class CommitLog implements CommitLogMBean
 public static final int END_OF_SEGMENT_MARKER = 0;  // this is 
written out at the end of a segment
 public static final int END_OF_SEGMENT_MARKER_SIZE = 4; // number of 
bytes of ^^^
 
-public CommitLogSegment activeSegment;
+public volatile CommitLogSegment activeSegment;
 
 private final CommitLogMetrics metrics;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/55b5605b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
index df3d257..25658ed 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
@@ -19,8 +19,10 @@ package org.apache.cassandra.db.commitlog;
 
 import java.io.DataOutputStream;
 import java.io.File;
+import java.io.FileOutputStream;
 import java.io.IOException;
 import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
 import java.nio.MappedByteBuffer;
 import java.nio.channels.FileChannel;
 import java.util.Collection;
@@ -130,8 +132,8 @@ public class CommitLogSegment
 bufferStream = new DataOutputStream(new 
ChecksummedOutputStream(new ByteBufferOutputStream(buffer), checksum));
 buffer.putInt(CommitLog.END_OF_SEGMENT_MARKER);
 buffer.position(0);
-
 needsSync = true;
+sync();
 }
 catch (IOException e)
 {
@@ -146,8 +148,54 @@ public class CommitLogSegment
 {
 // TODO shouldn't we close the file when we're done writing to it, 
which comes (potentially) much earlier than it's eligible for recyling?
 close();
+// it's safe to simply try (and maybe fail) to delete the log file 
because we should only ever close()/discard() once
+// the global ReplayPosition is past the current log file position, so 
we will never replay it; however to be on the
+// safe side we attempt to rename/zero it if delete fails
 if (deleteFile)
-FileUtils.deleteWithConfirm(logFile);
+{
+try
+{
+FileUtils.deleteWithConfirm(logFile);
+}
+catch (FSWriteError e)
+{
+// attempt to rename the file and zero its start, if possible, 
before throwing the error
+File file = logFile;
+try
+{
+File newFile = new File(file.getPath() + ".discarded");
+FileUtils.r

[2/2] git commit: Merge branch 'cassandra-2.0' into trunk

[marcuse]

Merge branch 'cassandra-2.0' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/7604935e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/7604935e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/7604935e

Branch: refs/heads/trunk
Commit: 7604935efe871e9bd458389c4116c492e9cbc7d7
Parents: f05870e 55b5605
Author: Marcus Eriksson 
Authored: Tue Feb 11 09:23:08 2014 +0100
Committer: Marcus Eriksson 
Committed: Tue Feb 11 09:23:08 2014 +0100

--

--

git commit: Stop CommitLogSegment.close() from unnecessarily calling sync() prior to cleaning the buffer.

[marcuse]

Updated Branches:
  refs/heads/cassandra-2.0 87aca600f -> 55b5605b7


Stop CommitLogSegment.close() from unnecessarily calling sync() prior to 
cleaning the buffer.

Patch by belliotsmith, reviewed by marcuse for CASSANDRA-6652


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/55b5605b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/55b5605b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/55b5605b

Branch: refs/heads/cassandra-2.0
Commit: 55b5605b7afcb7ae9bcb9b61959b91502f769db4
Parents: 87aca60
Author: Marcus Eriksson 
Authored: Tue Feb 11 09:18:09 2014 +0100
Committer: Marcus Eriksson 
Committed: Tue Feb 11 09:21:22 2014 +0100

--
 CHANGES.txt |  1 +
 .../cassandra/db/commitlog/CommitLog.java   |  2 +-
 .../db/commitlog/CommitLogSegment.java  | 74 ++--
 3 files changed, 55 insertions(+), 22 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/55b5605b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index b98dec7..93552ef 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -8,6 +8,7 @@
  * Correctly handle null with IF conditions and TTL (CASSANDRA-6623)
  * Account for range/row tombstones in tombstone drop
time histogram (CASSANDRA-6522)
+ * Stop CommitLogSegment.close() from calling sync() (CASSANDRA-6652)
 Merged from 1.2:
  * Fix upgradesstables NPE for non-CF-based indexes (CASSANDRA-6645)
  * Fix partition and range deletes not triggering flush (CASSANDRA-6655)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/55b5605b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
index 0e9a3f1..e9507da 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLog.java
@@ -56,7 +56,7 @@ public class CommitLog implements CommitLogMBean
 public static final int END_OF_SEGMENT_MARKER = 0;  // this is 
written out at the end of a segment
 public static final int END_OF_SEGMENT_MARKER_SIZE = 4; // number of 
bytes of ^^^
 
-public CommitLogSegment activeSegment;
+public volatile CommitLogSegment activeSegment;
 
 private final CommitLogMetrics metrics;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/55b5605b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
index df3d257..25658ed 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java
@@ -19,8 +19,10 @@ package org.apache.cassandra.db.commitlog;
 
 import java.io.DataOutputStream;
 import java.io.File;
+import java.io.FileOutputStream;
 import java.io.IOException;
 import java.io.RandomAccessFile;
+import java.nio.ByteBuffer;
 import java.nio.MappedByteBuffer;
 import java.nio.channels.FileChannel;
 import java.util.Collection;
@@ -130,8 +132,8 @@ public class CommitLogSegment
 bufferStream = new DataOutputStream(new 
ChecksummedOutputStream(new ByteBufferOutputStream(buffer), checksum));
 buffer.putInt(CommitLog.END_OF_SEGMENT_MARKER);
 buffer.position(0);
-
 needsSync = true;
+sync();
 }
 catch (IOException e)
 {
@@ -146,8 +148,54 @@ public class CommitLogSegment
 {
 // TODO shouldn't we close the file when we're done writing to it, 
which comes (potentially) much earlier than it's eligible for recyling?
 close();
+// it's safe to simply try (and maybe fail) to delete the log file 
because we should only ever close()/discard() once
+// the global ReplayPosition is past the current log file position, so 
we will never replay it; however to be on the
+// safe side we attempt to rename/zero it if delete fails
 if (deleteFile)
-FileUtils.deleteWithConfirm(logFile);
+{
+try
+{
+FileUtils.deleteWithConfirm(logFile);
+}
+catch (FSWriteError e)
+{
+// attempt to rename the file and zero its start, if possible, 
before throwing the error
+File file = logFile;
+try
+{
+File newFile = new File(file.getPath() + ".discarded");
+