date:20130523

Updated Branches:
  refs/heads/cassandra-1.2 c48c7ef16 - e39462557
  refs/heads/trunk a583123e8 - 0680372d8


comment


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e3946255
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e3946255
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e3946255

Branch: refs/heads/cassandra-1.2
Commit: e39462557c65d82e22644dbebfceef98ce522483
Parents: c48c7ef
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 09:24:59 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 09:24:59 2013 -0500

--
 .../org/apache/cassandra/net/CallbackInfo.java |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e3946255/src/java/org/apache/cassandra/net/CallbackInfo.java
--
diff --git a/src/java/org/apache/cassandra/net/CallbackInfo.java 
b/src/java/org/apache/cassandra/net/CallbackInfo.java
index a5fc8ad..f0e48e9 100644
--- a/src/java/org/apache/cassandra/net/CallbackInfo.java
+++ b/src/java/org/apache/cassandra/net/CallbackInfo.java
@@ -55,10 +55,10 @@ public class CallbackInfo
 }
 
 /**
- * @return TRUE if a hint should be written for this target and if the CL 
was achieved. FALSE otherwise.
+ * @return TRUE iff a hint should be written for this target.
  *
  * NOTE:
- * Assumes it is only called after the write of message to target has 
timed out.
+ * Assumes it is only called after the write of sentMessage to target 
has timed out.
  */
 public boolean shouldHint()
 {

[2/3] git commit: comment

comment


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e3946255
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e3946255
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e3946255

Branch: refs/heads/trunk
Commit: e39462557c65d82e22644dbebfceef98ce522483
Parents: c48c7ef
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 09:24:59 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 09:24:59 2013 -0500

--
 .../org/apache/cassandra/net/CallbackInfo.java |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/e3946255/src/java/org/apache/cassandra/net/CallbackInfo.java
--
diff --git a/src/java/org/apache/cassandra/net/CallbackInfo.java 
b/src/java/org/apache/cassandra/net/CallbackInfo.java
index a5fc8ad..f0e48e9 100644
--- a/src/java/org/apache/cassandra/net/CallbackInfo.java
+++ b/src/java/org/apache/cassandra/net/CallbackInfo.java
@@ -55,10 +55,10 @@ public class CallbackInfo
 }
 
 /**
- * @return TRUE if a hint should be written for this target and if the CL 
was achieved. FALSE otherwise.
+ * @return TRUE iff a hint should be written for this target.
  *
  * NOTE:
- * Assumes it is only called after the write of message to target has 
timed out.
+ * Assumes it is only called after the write of sentMessage to target 
has timed out.
  */
 public boolean shouldHint()
 {

[3/3] git commit: Merge branch 'cassandra-1.2' into trunk

Merge branch 'cassandra-1.2' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0680372d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0680372d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0680372d

Branch: refs/heads/trunk
Commit: 0680372d8d35db485a47602d9838c28baa1ced17
Parents: a583123 e394625
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 09:25:10 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 09:25:10 2013 -0500

--
 .../org/apache/cassandra/net/CallbackInfo.java |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0680372d/src/java/org/apache/cassandra/net/CallbackInfo.java
--

git commit: Remove the unused defaultConsistency from ModificationStatement

Updated Branches:
  refs/heads/cassandra-1.2 e39462557 - 49f220cbe


Remove the unused defaultConsistency from ModificationStatement


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49f220cb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49f220cb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49f220cb

Branch: refs/heads/cassandra-1.2
Commit: 49f220cbe0a1774066a048cdfd8682d214a6df68
Parents: e394625
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu May 23 17:31:52 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu May 23 17:31:52 2013 +0300

--
 .../cql3/statements/ModificationStatement.java |2 --
 1 files changed, 0 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49f220cb/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 28a003e..322ce7f 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -40,8 +40,6 @@ import org.apache.cassandra.transport.messages.ResultMessage;
  */
 public abstract class ModificationStatement extends CFStatement implements 
CQLStatement
 {
-public static final ConsistencyLevel defaultConsistency = 
ConsistencyLevel.ONE;
-
 public static enum Type
 {
 LOGGED, UNLOGGED, COUNTER

[1/2] git commit: Remove the unused defaultConsistency from ModificationStatement

Updated Branches:
  refs/heads/trunk 0680372d8 - 0d5ed281f


Remove the unused defaultConsistency from ModificationStatement


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/49f220cb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/49f220cb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/49f220cb

Branch: refs/heads/trunk
Commit: 49f220cbe0a1774066a048cdfd8682d214a6df68
Parents: e394625
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu May 23 17:31:52 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu May 23 17:31:52 2013 +0300

--
 .../cql3/statements/ModificationStatement.java |2 --
 1 files changed, 0 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/49f220cb/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git 
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java 
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 28a003e..322ce7f 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -40,8 +40,6 @@ import org.apache.cassandra.transport.messages.ResultMessage;
  */
 public abstract class ModificationStatement extends CFStatement implements 
CQLStatement
 {
-public static final ConsistencyLevel defaultConsistency = 
ConsistencyLevel.ONE;
-
 public static enum Type
 {
 LOGGED, UNLOGGED, COUNTER

[2/2] git commit: Merge branch 'cassandra-1.2' into trunk

Merge branch 'cassandra-1.2' into trunk

Conflicts:
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/0d5ed281
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/0d5ed281
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/0d5ed281

Branch: refs/heads/trunk
Commit: 0d5ed281f323f8a653017289825e8dc391564b0c
Parents: 0680372 49f220c
Author: Aleksey Yeschenko alek...@apache.org
Authored: Thu May 23 17:55:38 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Thu May 23 17:55:38 2013 +0300

--
 .../cql3/statements/ModificationStatement.java |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/0d5ed281/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --cc 
src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index 5b3e718,322ce7f..133a8ff
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@@ -34,71 -31,35 +34,70 @@@ import org.apache.cassandra.exceptions.
  import org.apache.cassandra.service.ClientState;
  import org.apache.cassandra.service.QueryState;
  import org.apache.cassandra.service.StorageProxy;
 +import org.apache.cassandra.thrift.ThriftValidation;
  import org.apache.cassandra.transport.messages.ResultMessage;
 +import org.apache.cassandra.utils.Pair;
  
 -/**
 - * Abstract class for statements that apply on a given column family.
 +/*
 + * Abstract parent class of individual modifications, i.e. INSERT, UPDATE and 
DELETE.
   */
 -public abstract class ModificationStatement extends CFStatement implements 
CQLStatement
 +public abstract class ModificationStatement implements CQLStatement
  {
- public static final ConsistencyLevel defaultConsistency = 
ConsistencyLevel.ONE;
 -public static enum Type
 +private static final ColumnIdentifier RESULT_COLUMN = new 
ColumnIdentifier(result, false);
 +
 +private final int boundTerms;
 +public final CFMetaData cfm;
 +private final Attributes attrs;
 +
 +private final MapColumnIdentifier, ListTerm processedKeys = new 
HashMapColumnIdentifier, ListTerm();
 +private final ListOperation columnOperations = new 
ArrayListOperation();
 +
 +private ListOperation columnConditions;
 +private boolean ifNotExists;
 +
 +public ModificationStatement(int boundTerms, CFMetaData cfm, Attributes 
attrs)
  {
 -LOGGED, UNLOGGED, COUNTER
 +this.boundTerms = boundTerms;
 +this.cfm = cfm;
 +this.attrs = attrs;
  }
  
 -protected Type type;
 +protected abstract boolean requireFullClusteringKey();
 +public abstract ColumnFamily updateForKey(ByteBuffer key, 
ColumnNameBuilder builder, UpdateParameters params) throws 
InvalidRequestException;
  
 -private Long timestamp;
 -private final int timeToLive;
 +public int getBoundsTerms()
 +{
 +return boundTerms;
 +}
  
 -public ModificationStatement(CFName name, Attributes attrs)
 +public String keyspace()
  {
 -this(name, attrs.timestamp, attrs.timeToLive);
 +return cfm.ksName;
  }
  
 -public ModificationStatement(CFName name, Long timestamp, int timeToLive)
 +public String columnFamily()
  {
 -super(name);
 -this.timestamp = timestamp;
 -this.timeToLive = timeToLive;
 +return cfm.cfName;
 +}
 +
 +public boolean isCounter()
 +{
 +return cfm.getDefaultValidator().isCommutative();
 +}
 +
 +public int getTimeToLive()
 +{
 +return attrs.timeToLive;
 +}
 +
 +public long getTimestamp(long now)
 +{
 +return attrs.timestamp == null ? now : attrs.timestamp;
 +}
 +
 +public boolean isSetTimestamp()
 +{
 +return attrs.timestamp != null;
  }
  
  public void checkAccess(ClientState state) throws 
InvalidRequestException, UnauthorizedException

[1/4] git commit: Ignore pre-truncate hints patch by Alexey Zotov and jbellis; reviewed by vijay for CASSANDRA-4655

Updated Branches:
  refs/heads/cassandra-1.2 49f220cbe - c4c9626f8
  refs/heads/trunk 0d5ed281f - 66aadda2e


Ignore pre-truncate hints
patch by Alexey Zotov and jbellis; reviewed by vijay for CASSANDRA-4655


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b7063918
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b7063918
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b7063918

Branch: refs/heads/trunk
Commit: b70639187e1ee5e3aebb489baa07aaa2771fb4ed
Parents: e394625
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 10:04:04 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 10:04:04 2013 -0500

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/db/ColumnFamilyStore.java |7 +++
 .../apache/cassandra/db/HintedHandOffManager.java  |   25 ++
 src/java/org/apache/cassandra/db/RowMutation.java  |9 
 src/java/org/apache/cassandra/db/SystemTable.java  |   35 ++-
 .../cassandra/db/commitlog/CommitLogReplayer.java  |   10 ++---
 .../cassandra/db/compaction/CompactionManager.java |2 +-
 7 files changed, 71 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7063918/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6f1127a..66c5f04 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.6
+ * Ignore pre-truncate hints (CASSANDRA-4655)
  * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
  * Write row markers when serializing schema (CASSANDRA-5572)
  * Check only SSTables for the requested range when streaming (CASSANDRA-5569)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7063918/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 055c415..429859e 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -298,6 +298,7 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 valid = false;
 unregisterMBean();
 
+SystemTable.removeTruncationRecord(metadata.cfId);
 data.unreferenceSSTables();
 indexManager.invalidate();
 }
@@ -2077,4 +2078,10 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 {
 return getDataTracker().getDroppableTombstoneRatio();
 }
+
+public long getTruncationTime()
+{
+PairReplayPosition, Long truncationRecord = 
SystemTable.getTruncationRecords().get(metadata.cfId);
+return truncationRecord == null ? Long.MIN_VALUE : 
truncationRecord.right;
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/b7063918/src/java/org/apache/cassandra/db/HintedHandOffManager.java
--
diff --git a/src/java/org/apache/cassandra/db/HintedHandOffManager.java 
b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
index 53411f5..9346fb3 100644
--- a/src/java/org/apache/cassandra/db/HintedHandOffManager.java
+++ b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
@@ -30,6 +30,7 @@ import javax.management.MBeanServer;
 import javax.management.ObjectName;
 
 import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.ImmutableSortedSet;
 import com.google.common.collect.Lists;
 import com.google.common.util.concurrent.RateLimiter;
@@ -365,6 +366,30 @@ public class HintedHandOffManager implements 
HintedHandOffManagerMBean
 throw new AssertionError(e);
 }
 
+MapUUID, Long truncationTimesCache = new HashMapUUID, 
Long();
+for (UUID cfId : 
ImmutableSet.copyOf((rm.getColumnFamilyIds(
+{
+Long truncatedAt = truncationTimesCache.get(cfId);
+if (truncatedAt == null)
+{
+ColumnFamilyStore cfs = 
Table.open(rm.getTable()).getColumnFamilyStore(cfId);
+truncatedAt = cfs.getTruncationTime();
+truncationTimesCache.put(cfId, truncatedAt);
+}
+
+if (hint.maxTimestamp()  truncatedAt)
+{
+logger.debug(Skipping delivery of hint for truncated 
columnfamily {} + cfId);
+rm = rm.without(cfId);
+}
+}
+
+

[3/4] git commit: Ignore pre-truncate hints patch by Alexey Zotov and jbellis; reviewed by vijay for CASSANDRA-4655

Ignore pre-truncate hints
patch by Alexey Zotov and jbellis; reviewed by vijay for CASSANDRA-4655


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c4c9626f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c4c9626f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c4c9626f

Branch: refs/heads/cassandra-1.2
Commit: c4c9626f8b3ba941aa51860a7d10a6e686362f85
Parents: 49f220c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 10:04:04 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 10:08:17 2013 -0500

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/db/ColumnFamilyStore.java |7 +++
 .../apache/cassandra/db/HintedHandOffManager.java  |   25 ++
 src/java/org/apache/cassandra/db/RowMutation.java  |9 
 src/java/org/apache/cassandra/db/SystemTable.java  |   35 ++-
 .../cassandra/db/commitlog/CommitLogReplayer.java  |   10 ++---
 .../cassandra/db/compaction/CompactionManager.java |2 +-
 7 files changed, 71 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4c9626f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6f1127a..66c5f04 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.6
+ * Ignore pre-truncate hints (CASSANDRA-4655)
  * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
  * Write row markers when serializing schema (CASSANDRA-5572)
  * Check only SSTables for the requested range when streaming (CASSANDRA-5569)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4c9626f/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 055c415..429859e 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -298,6 +298,7 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 valid = false;
 unregisterMBean();
 
+SystemTable.removeTruncationRecord(metadata.cfId);
 data.unreferenceSSTables();
 indexManager.invalidate();
 }
@@ -2077,4 +2078,10 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 {
 return getDataTracker().getDroppableTombstoneRatio();
 }
+
+public long getTruncationTime()
+{
+PairReplayPosition, Long truncationRecord = 
SystemTable.getTruncationRecords().get(metadata.cfId);
+return truncationRecord == null ? Long.MIN_VALUE : 
truncationRecord.right;
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4c9626f/src/java/org/apache/cassandra/db/HintedHandOffManager.java
--
diff --git a/src/java/org/apache/cassandra/db/HintedHandOffManager.java 
b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
index 53411f5..9346fb3 100644
--- a/src/java/org/apache/cassandra/db/HintedHandOffManager.java
+++ b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
@@ -30,6 +30,7 @@ import javax.management.MBeanServer;
 import javax.management.ObjectName;
 
 import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.ImmutableSortedSet;
 import com.google.common.collect.Lists;
 import com.google.common.util.concurrent.RateLimiter;
@@ -365,6 +366,30 @@ public class HintedHandOffManager implements 
HintedHandOffManagerMBean
 throw new AssertionError(e);
 }
 
+MapUUID, Long truncationTimesCache = new HashMapUUID, 
Long();
+for (UUID cfId : 
ImmutableSet.copyOf((rm.getColumnFamilyIds(
+{
+Long truncatedAt = truncationTimesCache.get(cfId);
+if (truncatedAt == null)
+{
+ColumnFamilyStore cfs = 
Table.open(rm.getTable()).getColumnFamilyStore(cfId);
+truncatedAt = cfs.getTruncationTime();
+truncationTimesCache.put(cfId, truncatedAt);
+}
+
+if (hint.maxTimestamp()  truncatedAt)
+{
+logger.debug(Skipping delivery of hint for truncated 
columnfamily {} + cfId);
+rm = rm.without(cfId);
+}
+}
+
+if (rm.isEmpty())
+{
+deleteHint(hostIdBytes,

[2/4] git commit: merge from 1.2

merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f620b348
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f620b348
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f620b348

Branch: refs/heads/trunk
Commit: f620b348a064a55749ffcbc4d25e08c3fe71f1be
Parents: 0680372 b706391
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 10:08:06 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 10:08:06 2013 -0500

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/db/ColumnFamilyStore.java |9 -
 .../apache/cassandra/db/HintedHandOffManager.java  |   25 +++
 src/java/org/apache/cassandra/db/RowMutation.java  |9 
 src/java/org/apache/cassandra/db/SystemTable.java  |   32 ++-
 .../cassandra/db/commitlog/CommitLogReplayer.java  |   10 ++---
 6 files changed, 68 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f620b348/CHANGES.txt
--
diff --cc CHANGES.txt
index b765896,66c5f04..d283b72
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,58 -1,5 +1,59 @@@
 +2.0
 + * Removed on-heap row cache (CASSANDRA-5348)
 + * use nanotime consistently for node-local timeouts (CASSANDRA-5581)
 + * Avoid unnecessary second pass on name-based queries (CASSANDRA-5577)
 + * Experimental triggers (CASSANDRA-1311)
 + * JEMalloc support for off-heap allocation (CASSANDRA-3997)
 + * Single-pass compaction (CASSANDRA-4180)
 + * Removed token range bisection (CASSANDRA-5518)
 + * Removed compatibility with pre-1.2.5 sstables and network messages
 +   (CASSANDRA-5511)
 + * removed PBSPredictor (CASSANDRA-5455)
 + * CAS support (CASSANDRA-5062, 5441, 5443)
 + * Leveled compaction performs size-tiered compactions in L0 
 +   (CASSANDRA-5371, 5439)
 + * Add yaml network topology snitch for mixed ec2/other envs (CASSANDRA-5339)
 + * Log when a node is down longer than the hint window (CASSANDRA-4554)
 + * Optimize tombstone creation for ExpiringColumns (CASSANDRA-4917)
 + * Improve LeveledScanner work estimation (CASSANDRA-5250, 5407)
 + * Replace compaction lock with runWithCompactionsDisabled (CASSANDRA-3430)
 + * Change Message IDs to ints (CASSANDRA-5307)
 + * Move sstable level information into the Stats component, removing the
 +   need for a separate Manifest file (CASSANDRA-4872)
 + * avoid serializing to byte[] on commitlog append (CASSANDRA-5199)
 + * make index_interval configurable per columnfamily (CASSANDRA-3961)
 + * add default_time_to_live (CASSANDRA-3974)
 + * add memtable_flush_period_in_ms (CASSANDRA-4237)
 + * replace supercolumns internally by composites (CASSANDRA-3237, 5123)
 + * upgrade thrift to 0.9.0 (CASSANDRA-3719)
 + * drop unnecessary keyspace parameter from user-defined compaction API 
 +   (CASSANDRA-5139)
 + * more robust solution to incomplete compactions + counters (CASSANDRA-5151)
 + * Change order of directory searching for c*.in.sh (CASSANDRA-3983)
 + * Add tool to reset SSTable compaction level for LCS (CASSANDRA-5271)
 + * Allow custom configuration loader (CASSANDRA-5045)
 + * Remove memory emergency pressure valve logic (CASSANDRA-3534)
 + * Reduce request latency with eager retry (CASSANDRA-4705)
 + * cqlsh: Remove ASSUME command (CASSANDRA-5331)
 + * Rebuild BF when loading sstables if bloom_filter_fp_chance
 +   has changed since compaction (CASSANDRA-5015)
 + * remove row-level bloom filters (CASSANDRA-4885)
 + * Change Kernel Page Cache skipping into row preheating (disabled by default)
 +   (CASSANDRA-4937)
 + * Improve repair by deciding on a gcBefore before sending
 +   out TreeRequests (CASSANDRA-4932)
 + * Add an official way to disable compactions (CASSANDRA-5074)
 + * Reenable ALTER TABLE DROP with new semantics (CASSANDRA-3919)
 + * Add binary protocol versioning (CASSANDRA-5436)
 + * Swap THshaServer for TThreadedSelectorServer (CASSANDRA-5530)
 + * Add alias support to SELECT statement (CASSANDRA-5075)
 + * Don't create empty RowMutations in CommitLogReplayer (CASSANDRA-5541)
 + * Use range tombstones when dropping cfs/columns from schema (CASSANDRA-5579)
 + * cqlsh: drop CQL2/CQL3-beta support (CASSANDRA-5585)
 +
 +
  1.2.6
+  * Ignore pre-truncate hints (CASSANDRA-4655)
   * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
   * Write row markers when serializing schema (CASSANDRA-5572)
   * Check only SSTables for the requested range when streaming (CASSANDRA-5569)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/f620b348/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --cc src/java/org/apache/cassandra/db/ColumnFamilyStore.java

[4/4] git commit: Merge remote-tracking branch 'origin/trunk' into trunk

Merge remote-tracking branch 'origin/trunk' into trunk


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66aadda2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66aadda2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66aadda2

Branch: refs/heads/trunk
Commit: 66aadda2efeefebabb50128091378531b7c864c7
Parents: f620b34 0d5ed28
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 10:08:25 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 10:08:25 2013 -0500

--
 .../cql3/statements/ModificationStatement.java |1 -
 1 files changed, 0 insertions(+), 1 deletions(-)
--

[jira] [Updated] (CASSANDRA-5589) ArrayIndexOutOfBoundsException in LeveledManifest


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5589:
--

 Reviewer: krummas
 Priority: Minor  (was: Major)
Affects Version/s: (was: 1.2.4)
   1.0.0
Fix Version/s: 1.2.6
 Assignee: Jonathan Ellis
   Labels: compaction  (was: )

 ArrayIndexOutOfBoundsException in LeveledManifest
 -

 Key: CASSANDRA-5589
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5589
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.0.0
Reporter: Jeremy Hanna
Assignee: Jonathan Ellis
Priority: Minor
  Labels: compaction
 Fix For: 1.2.6

 Attachments: 5589.txt


 The following stack trace was in the system.log:
 {quote}
 ERROR [CompactionExecutor:2] 2013-05-22 16:19:32,402 CassandraDaemon.java 
 (line 174) Exception in thread Thread[CompactionExecutor:2,1,main]
  java.lang.ArrayIndexOutOfBoundsException: 5
   at 
 org.apache.cassandra.db.compaction.LeveledManifest.skipLevels(LeveledManifest.java:176)
   at 
 org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:215)
   at 
 org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:155)
   at 
 org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:410)
   at 
 org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:223)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:991)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:230)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:188)
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5589) ArrayIndexOutOfBoundsException in LeveledManifest


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5589:
--

Attachment: 5589.txt

The manifest is assuming that when the sstable size is increased, existing 
sstables are magically resized to that, which is not the case.

Fix attached, with an additional warning to not increase sstable size above 1GB 
(which is what the user has done here).

 ArrayIndexOutOfBoundsException in LeveledManifest
 -

 Key: CASSANDRA-5589
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5589
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
Reporter: Jeremy Hanna
 Attachments: 5589.txt


 The following stack trace was in the system.log:
 {quote}
 ERROR [CompactionExecutor:2] 2013-05-22 16:19:32,402 CassandraDaemon.java 
 (line 174) Exception in thread Thread[CompactionExecutor:2,1,main]
  java.lang.ArrayIndexOutOfBoundsException: 5
   at 
 org.apache.cassandra.db.compaction.LeveledManifest.skipLevels(LeveledManifest.java:176)
   at 
 org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:215)
   at 
 org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:155)
   at 
 org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:410)
   at 
 org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:223)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:991)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:230)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:188)
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[4/4] git commit: merge from 1.2

merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19047b89
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19047b89
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19047b89

Branch: refs/heads/trunk
Commit: 19047b8943ed4b4ea5925c7c8d94025d26f076f3
Parents: 66aadda 44c462c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 11:05:53 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 11:05:53 2013 -0500

--
 src/java/org/apache/cassandra/db/RowMutation.java |9 +
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 2 files changed, 10 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/19047b89/src/java/org/apache/cassandra/db/RowMutation.java
--
diff --cc src/java/org/apache/cassandra/db/RowMutation.java
index d78247b,b85cfcd..64069c8
--- a/src/java/org/apache/cassandra/db/RowMutation.java
+++ b/src/java/org/apache/cassandra/db/RowMutation.java
@@@ -238,23 -311,65 +238,32 @@@ public class RowMutation implements IMu
  return buff.append(])).toString();
  }
  
 -public void addColumnOrSuperColumn(String cfName, ColumnOrSuperColumn 
cosc)
 -{
 -if (cosc.super_column != null)
 -{
 -for (org.apache.cassandra.thrift.Column column : 
cosc.super_column.columns)
 -{
 -add(new QueryPath(cfName, cosc.super_column.name, 
column.name), column.value, column.timestamp, column.ttl);
 -}
 -}
 -else if (cosc.column != null)
 -{
 -add(new QueryPath(cfName, null, cosc.column.name), 
cosc.column.value, cosc.column.timestamp, cosc.column.ttl);
 -}
 -else if (cosc.counter_super_column != null)
 -{
 -for (org.apache.cassandra.thrift.CounterColumn column : 
cosc.counter_super_column.columns)
 -{
 -addCounter(new QueryPath(cfName, 
cosc.counter_super_column.name, column.name), column.value);
 -}
 -}
 -else // cosc.counter_column != null
 -{
 -addCounter(new QueryPath(cfName, null, cosc.counter_column.name), 
cosc.counter_column.value);
 -}
 -}
 -
 -public void deleteColumnOrSuperColumn(String cfName, Deletion del)
 +public RowMutation without(UUID cfId)
  {
 -if (del.predicate != null  del.predicate.column_names != null)
 -{
 -for(ByteBuffer c : del.predicate.column_names)
 -{
 -if (del.super_column == null  
Schema.instance.getColumnFamilyType(table, cfName) == ColumnFamilyType.Super)
 -delete(new QueryPath(cfName, c), del.timestamp);
 -else
 -delete(new QueryPath(cfName, del.super_column, c), 
del.timestamp);
 -}
 -}
 -else
 -{
 -delete(new QueryPath(cfName, del.super_column), del.timestamp);
 -}
 +RowMutation rm = new RowMutation(table, key);
 +for (Map.EntryUUID, ColumnFamily entry : modifications.entrySet())
 +if (!entry.getKey().equals(cfId))
 +rm.add(entry.getValue());
 +return rm;
  }
  
+ public RowMutation without(UUID cfId)
+ {
+ RowMutation rm = new RowMutation(table, key);
+ for (Map.EntryUUID, ColumnFamily entry : modifications.entrySet())
+ if (!entry.getKey().equals(cfId))
+ rm.add(entry.getValue());
+ return rm;
+ }
+ 
  public static class RowMutationSerializer implements 
IVersionedSerializerRowMutation
  {
 -public void serialize(RowMutation rm, DataOutput dos, int version) 
throws IOException
 +public void serialize(RowMutation rm, DataOutput out, int version) 
throws IOException
  {
 -dos.writeUTF(rm.getTable());
 -ByteBufferUtil.writeWithShortLength(rm.key(), dos);
 +if (version  MessagingService.VERSION_20)
 +out.writeUTF(rm.getTable());
 +
 +ByteBufferUtil.writeWithShortLength(rm.key(), out);
  
  /* serialize the modifications in the mutation */
  int size = rm.modifications.size();

http://git-wip-us.apache.org/repos/asf/cassandra/blob/19047b89/src/java/org/apache/cassandra/db/SystemTable.java
--

[2/4] git commit: fix typo in removeTruncationRecord

fix typo in removeTruncationRecord


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44c462cb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44c462cb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44c462cb

Branch: refs/heads/trunk
Commit: 44c462cb7a231c1e99226e4a3cac3f05a3a99669
Parents: c4c9626
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 11:04:18 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 11:04:18 2013 -0500

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44c462cb/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java 
b/src/java/org/apache/cassandra/db/SystemTable.java
index 327f01b..ecdad94 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -193,7 +193,7 @@ public class SystemTable
  */
 public static void removeTruncationRecord(UUID cfId)
 {
-String req = DELETE truncation_time['%s'] from system.%s WHERE key = 
'%s';
+String req = DELETE truncated_at['%s'] from system.%s WHERE key = 
'%s';
 processInternal(String.format(req, cfId, LOCAL_CF, LOCAL_KEY));
 forceBlockingFlush(LOCAL_CF);
 }

[3/4] git commit: fix typo in removeTruncationRecord

fix typo in removeTruncationRecord


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/44c462cb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/44c462cb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/44c462cb

Branch: refs/heads/cassandra-1.2
Commit: 44c462cb7a231c1e99226e4a3cac3f05a3a99669
Parents: c4c9626
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 11:04:18 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 11:04:18 2013 -0500

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/44c462cb/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java 
b/src/java/org/apache/cassandra/db/SystemTable.java
index 327f01b..ecdad94 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -193,7 +193,7 @@ public class SystemTable
  */
 public static void removeTruncationRecord(UUID cfId)
 {
-String req = DELETE truncation_time['%s'] from system.%s WHERE key = 
'%s';
+String req = DELETE truncated_at['%s'] from system.%s WHERE key = 
'%s';
 processInternal(String.format(req, cfId, LOCAL_CF, LOCAL_KEY));
 forceBlockingFlush(LOCAL_CF);
 }

[1/4] git commit: Ignore pre-truncate hints patch by Alexey Zotov and jbellis; reviewed by vijay for CASSANDRA-4655

Updated Branches:
  refs/heads/cassandra-1.2 c4c9626f8 - 44c462cb7
  refs/heads/trunk 66aadda2e - 19047b894


Ignore pre-truncate hints
patch by Alexey Zotov and jbellis; reviewed by vijay for CASSANDRA-4655


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c4c9626f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c4c9626f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c4c9626f

Branch: refs/heads/trunk
Commit: c4c9626f8b3ba941aa51860a7d10a6e686362f85
Parents: 49f220c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 10:04:04 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 10:08:17 2013 -0500

--
 CHANGES.txt|1 +
 .../org/apache/cassandra/db/ColumnFamilyStore.java |7 +++
 .../apache/cassandra/db/HintedHandOffManager.java  |   25 ++
 src/java/org/apache/cassandra/db/RowMutation.java  |9 
 src/java/org/apache/cassandra/db/SystemTable.java  |   35 ++-
 .../cassandra/db/commitlog/CommitLogReplayer.java  |   10 ++---
 .../cassandra/db/compaction/CompactionManager.java |2 +-
 7 files changed, 71 insertions(+), 18 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4c9626f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 6f1127a..66c5f04 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.6
+ * Ignore pre-truncate hints (CASSANDRA-4655)
  * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
  * Write row markers when serializing schema (CASSANDRA-5572)
  * Check only SSTables for the requested range when streaming (CASSANDRA-5569)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4c9626f/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
--
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java 
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 055c415..429859e 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -298,6 +298,7 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 valid = false;
 unregisterMBean();
 
+SystemTable.removeTruncationRecord(metadata.cfId);
 data.unreferenceSSTables();
 indexManager.invalidate();
 }
@@ -2077,4 +2078,10 @@ public class ColumnFamilyStore implements 
ColumnFamilyStoreMBean
 {
 return getDataTracker().getDroppableTombstoneRatio();
 }
+
+public long getTruncationTime()
+{
+PairReplayPosition, Long truncationRecord = 
SystemTable.getTruncationRecords().get(metadata.cfId);
+return truncationRecord == null ? Long.MIN_VALUE : 
truncationRecord.right;
+}
 }

http://git-wip-us.apache.org/repos/asf/cassandra/blob/c4c9626f/src/java/org/apache/cassandra/db/HintedHandOffManager.java
--
diff --git a/src/java/org/apache/cassandra/db/HintedHandOffManager.java 
b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
index 53411f5..9346fb3 100644
--- a/src/java/org/apache/cassandra/db/HintedHandOffManager.java
+++ b/src/java/org/apache/cassandra/db/HintedHandOffManager.java
@@ -30,6 +30,7 @@ import javax.management.MBeanServer;
 import javax.management.ObjectName;
 
 import com.google.common.annotations.VisibleForTesting;
+import com.google.common.collect.ImmutableSet;
 import com.google.common.collect.ImmutableSortedSet;
 import com.google.common.collect.Lists;
 import com.google.common.util.concurrent.RateLimiter;
@@ -365,6 +366,30 @@ public class HintedHandOffManager implements 
HintedHandOffManagerMBean
 throw new AssertionError(e);
 }
 
+MapUUID, Long truncationTimesCache = new HashMapUUID, 
Long();
+for (UUID cfId : 
ImmutableSet.copyOf((rm.getColumnFamilyIds(
+{
+Long truncatedAt = truncationTimesCache.get(cfId);
+if (truncatedAt == null)
+{
+ColumnFamilyStore cfs = 
Table.open(rm.getTable()).getColumnFamilyStore(cfId);
+truncatedAt = cfs.getTruncationTime();
+truncationTimesCache.put(cfId, truncatedAt);
+}
+
+if (hint.maxTimestamp()  truncatedAt)
+{
+logger.debug(Skipping delivery of hint for truncated 
columnfamily {} + cfId);
+rm = rm.without(cfId);
+}
+}
+
+

[jira] [Commented] (CASSANDRA-5588) Add get commands to nodetool for things with set

2013-05-23 Thread Brandon Williams (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665309#comment-13665309
 ] 

Brandon Williams commented on CASSANDRA-5588:
-

That's not a bad idea, but we actually already have getcompactionthroughput, so 
simply adding getstreamthroughput would be most consistent.

 Add get commands to nodetool for things with set
 

 Key: CASSANDRA-5588
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Jeremiah Jordan
Assignee: Michał Michalski
Priority: Minor
  Labels: lhf
 Fix For: 1.2.6


 Can we add:
 nodetool getcompactionthroughput
 nodetool getstreamthroughput
 To go with the set commands?  You currently have to fire up a JMX client to 
 know what the current values are.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[1/5] git commit: improve cell index tracing

Updated Branches:
  refs/heads/cassandra-1.2 44c462cb7 - 57e6888e6
  refs/heads/trunk 19047b894 - 1c8e627a6


improve cell index tracing


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d0dff7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d0dff7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d0dff7a

Branch: refs/heads/cassandra-1.2
Commit: 2d0dff7acec0c36f43536c92480892d9c92bd27f
Parents: 44c462c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 11:23:18 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 11:24:45 2013 -0500

--
 .../apache/cassandra/io/sstable/SSTableReader.java |5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d0dff7a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index 6b71223..ea9c451 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -732,7 +732,7 @@ public class SSTableReader extends SSTable
 assert key instanceof DecoratedKey; // EQ only make sense if the 
key is a valid row key
 if (!bf.isPresent(((DecoratedKey)key).key))
 {
-logger.debug(Bloom filter allows skipping sstable {}, 
descriptor.generation);
+Tracing.trace(Bloom filter allows skipping sstable {}, 
descriptor.generation);
 return null;
 }
 }
@@ -745,7 +745,6 @@ public class SSTableReader extends SSTable
 RowIndexEntry cachedPosition = getCachedPosition(cacheKey, 
updateCacheAndStats);
 if (cachedPosition != null)
 {
-logger.trace(Cache hit for {} - {}, cacheKey, 
cachedPosition);
 Tracing.trace(Key cache hit for sstable {}, 
descriptor.generation);
 return cachedPosition;
 }
@@ -835,7 +834,7 @@ public class SSTableReader extends SSTable
 }
 if (op == Operator.EQ  updateCacheAndStats)
 bloomFilterTracker.addTruePositive();
-Tracing.trace(Partition index lookup complete for 
sstable {}, descriptor.generation);
+Tracing.trace(Partition index with {} entries found 
for sstable {}, indexEntry.columnsIndex().size(), descriptor.generation);
 return indexEntry;
 }

[2/5] git commit: improve cell index tracing

improve cell index tracing


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2d0dff7a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2d0dff7a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2d0dff7a

Branch: refs/heads/trunk
Commit: 2d0dff7acec0c36f43536c92480892d9c92bd27f
Parents: 44c462c
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 11:23:18 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 11:24:45 2013 -0500

--
 .../apache/cassandra/io/sstable/SSTableReader.java |5 ++---
 1 files changed, 2 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/2d0dff7a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
index 6b71223..ea9c451 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableReader.java
@@ -732,7 +732,7 @@ public class SSTableReader extends SSTable
 assert key instanceof DecoratedKey; // EQ only make sense if the 
key is a valid row key
 if (!bf.isPresent(((DecoratedKey)key).key))
 {
-logger.debug(Bloom filter allows skipping sstable {}, 
descriptor.generation);
+Tracing.trace(Bloom filter allows skipping sstable {}, 
descriptor.generation);
 return null;
 }
 }
@@ -745,7 +745,6 @@ public class SSTableReader extends SSTable
 RowIndexEntry cachedPosition = getCachedPosition(cacheKey, 
updateCacheAndStats);
 if (cachedPosition != null)
 {
-logger.trace(Cache hit for {} - {}, cacheKey, 
cachedPosition);
 Tracing.trace(Key cache hit for sstable {}, 
descriptor.generation);
 return cachedPosition;
 }
@@ -835,7 +834,7 @@ public class SSTableReader extends SSTable
 }
 if (op == Operator.EQ  updateCacheAndStats)
 bloomFilterTracker.addTruePositive();
-Tracing.trace(Partition index lookup complete for 
sstable {}, descriptor.generation);
+Tracing.trace(Partition index with {} entries found 
for sstable {}, indexEntry.columnsIndex().size(), descriptor.generation);
 return indexEntry;
 }

[3/5] git commit: fix NPE

fix NPE


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57e6888e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57e6888e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57e6888e

Branch: refs/heads/trunk
Commit: 57e6888e6aa8cc726fadb4087fe0cb77853c12eb
Parents: 2d0dff7
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 11:24:35 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 11:24:46 2013 -0500

--
 .../cassandra/db/commitlog/CommitLogReplayer.java  |3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/57e6888e/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
index e1fefa1..6b401fb 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
@@ -80,7 +80,8 @@ public class CommitLogReplayer
 ReplayPosition rp = 
ReplayPosition.getReplayPosition(cfs.getSSTables());
 
 // but, if we've truncted the cf in question, then we need to need 
to start replay after the truncation
-ReplayPosition truncatedAt = 
truncationPositions.get(cfs.metadata.cfId).left;
+PairReplayPosition, Long truncateRecord = 
truncationPositions.get(cfs.metadata.cfId);
+ReplayPosition truncatedAt = truncateRecord == null ? null : 
truncateRecord.left;
 if (truncatedAt != null)
 rp = replayPositionOrdering.max(Arrays.asList(rp, 
truncatedAt));

[4/5] git commit: fix NPE

fix NPE


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/57e6888e
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/57e6888e
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/57e6888e

Branch: refs/heads/cassandra-1.2
Commit: 57e6888e6aa8cc726fadb4087fe0cb77853c12eb
Parents: 2d0dff7
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 11:24:35 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 11:24:46 2013 -0500

--
 .../cassandra/db/commitlog/CommitLogReplayer.java  |3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/57e6888e/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
--
diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java 
b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
index e1fefa1..6b401fb 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogReplayer.java
@@ -80,7 +80,8 @@ public class CommitLogReplayer
 ReplayPosition rp = 
ReplayPosition.getReplayPosition(cfs.getSSTables());
 
 // but, if we've truncted the cf in question, then we need to need 
to start replay after the truncation
-ReplayPosition truncatedAt = 
truncationPositions.get(cfs.metadata.cfId).left;
+PairReplayPosition, Long truncateRecord = 
truncationPositions.get(cfs.metadata.cfId);
+ReplayPosition truncatedAt = truncateRecord == null ? null : 
truncateRecord.left;
 if (truncatedAt != null)
 rp = replayPositionOrdering.max(Arrays.asList(rp, 
truncatedAt));

[jira] [Resolved] (CASSANDRA-2308) add tracing of cell name index effectiveness


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-2308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis resolved CASSANDRA-2308.
---

   Resolution: Fixed
Fix Version/s: (was: 2.0)
   1.2.6

done in 2d0dff7acec0c36f43536c92480892d9c92bd27f

 add tracing of cell name index effectiveness
 

 Key: CASSANDRA-2308
 URL: https://issues.apache.org/jira/browse/CASSANDRA-2308
 Project: Cassandra
  Issue Type: New Feature
  Components: Core
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
 Fix For: 1.2.6


 Exposing row index sizes would give us the information to tune 
 column_index_size correctly (see CASSANDRA-2297).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5589) ArrayIndexOutOfBoundsException in LeveledManifest

2013-05-23 Thread Marcus Eriksson (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665325#comment-13665325
 ] 

Marcus Eriksson commented on CASSANDRA-5589:


lgtm

 ArrayIndexOutOfBoundsException in LeveledManifest
 -

 Key: CASSANDRA-5589
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5589
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.0.0
Reporter: Jeremy Hanna
Assignee: Jonathan Ellis
Priority: Minor
  Labels: compaction
 Fix For: 1.2.6

 Attachments: 5589.txt


 The following stack trace was in the system.log:
 {quote}
 ERROR [CompactionExecutor:2] 2013-05-22 16:19:32,402 CassandraDaemon.java 
 (line 174) Exception in thread Thread[CompactionExecutor:2,1,main]
  java.lang.ArrayIndexOutOfBoundsException: 5
   at 
 org.apache.cassandra.db.compaction.LeveledManifest.skipLevels(LeveledManifest.java:176)
   at 
 org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:215)
   at 
 org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:155)
   at 
 org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:410)
   at 
 org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:223)
   at 
 org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:991)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:230)
   at 
 org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
   at 
 org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
   at 
 org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
   at 
 org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
   at 
 org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:188)
 {quote}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5588) Add get commands to nodetool for things with set

2013-05-23 Thread JIRA


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665327#comment-13665327
 ] 

Michał Michalski commented on CASSANDRA-5588:
-

Yes, but please note that getcompactionthreshold (I guess you meant it) is 
called with two additional params (KS and CF), so it wouldn't make sense to mix 
with node-wide info command. 
Anyway, I'm fine with both options, so I'll do it as stated in description if 
you prefer so :-)

 Add get commands to nodetool for things with set
 

 Key: CASSANDRA-5588
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Jeremiah Jordan
Assignee: Michał Michalski
Priority: Minor
  Labels: lhf
 Fix For: 1.2.6


 Can we add:
 nodetool getcompactionthroughput
 nodetool getstreamthroughput
 To go with the set commands?  You currently have to fire up a JMX client to 
 know what the current values are.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5588) Add get commands to nodetool for things with set


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665333#comment-13665333
 ] 

Jeremiah Jordan commented on CASSANDRA-5588:


We don't have getcompactionthroughput, we have getcompactionthreshold.

We do have a call to getCompactionThroughput from the time estimation code, but 
the getcompactionthroughput command is not exposed to the end user.

 Add get commands to nodetool for things with set
 

 Key: CASSANDRA-5588
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Jeremiah Jordan
Assignee: Michał Michalski
Priority: Minor
  Labels: lhf
 Fix For: 1.2.6


 Can we add:
 nodetool getcompactionthroughput
 nodetool getstreamthroughput
 To go with the set commands?  You currently have to fire up a JMX client to 
 know what the current values are.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5588) Add get commands to nodetool for things with set


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665333#comment-13665333
 ] 

Jeremiah Jordan edited comment on CASSANDRA-5588 at 5/23/13 4:40 PM:
-

We don't have getcompactionthroughput, we have getcompactionthreshold.

We do have a call to getCompactionThroughput from the time estimation code, but 
the getcompactionthroughput command is not exposed to the end user.

Putting them in info works for me, we already have the global cache stuff 
there. 

  was (Author: jjordan):
We don't have getcompactionthroughput, we have getcompactionthreshold.

We do have a call to getCompactionThroughput from the time estimation code, but 
the getcompactionthroughput command is not exposed to the end user.
  
 Add get commands to nodetool for things with set
 

 Key: CASSANDRA-5588
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5588
 Project: Cassandra
  Issue Type: Bug
  Components: Tools
Reporter: Jeremiah Jordan
Assignee: Michał Michalski
Priority: Minor
  Labels: lhf
 Fix For: 1.2.6


 Can we add:
 nodetool getcompactionthroughput
 nodetool getstreamthroughput
 To go with the set commands?  You currently have to fire up a JMX client to 
 know what the current values are.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5590) User defined types for CQL3

2013-05-23 Thread Sylvain Lebresne (JIRA)

Sylvain Lebresne created CASSANDRA-5590:
---

 Summary: User defined types for CQL3
 Key: CASSANDRA-5590
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5590
 Project: Cassandra
  Issue Type: New Feature
Reporter: Sylvain Lebresne


A typical use case for a collection could be to store a bunch of addresses in a 
user profile. An address could typically be composed of a few properties: say a 
street, a city, a postal code and maybe a few phone numbers associated to it.

To model that currently with collections, you might use a {{mapstring, 
blob}}, where the map key could be a string identifying the address, and the 
value would be all the infos of an address serialized manually (you can use 
{{text}} instead of {{blob}} and shove everything in a string if you prefer but 
the principle is the same).

This ticket suggests to make this more user friendly by allowing:
{noformat}
CREATE TYPE address (
  street text,
  city text,
  zip_code int,
  phones settext
)

CREATE TABLE users (
  id uuid PRIMARY KEY,
  name text,
  addresses mapstring, address
)
{noformat}

Under the hood, that type declaration would just be metadata on top of 
CompositeType (which does mean a limitation would be that we wouldn't allow 
re-ordering or removal of fields in a custom TYPE). Namely, the {{address}} 
type would be in practice a {{CompositeType(UTF8Type, UTF8Type, Int32Type, 
SetType(UTF8Type))}} + some metadata that records the name of each component.  
In other words, this would mostly be user-friendly syntactic sugar to create 
composite blobs.

I'll note that this would also be useful outside collections, as it might 
sometimes be more efficient/useful to have such simple composite blob. For 
instance, you could imagine to have a:
{noformat}
CREATE TYPE fullname (
  firstname text,
  lastname text
)
{noformat}
and to rewrite the {{users}} table above as
{noformat}
CREATE TABLE users (
  id uuid PRIMARY KEY,
  name fullname,
  addresses mapstring, address
)
{noformat}

In terms of inserts we'd need a syntax for those new struct. Could be:
{noformat}
INSERT INTO users (id, name)
   VALUES (2ad..., { firstname: 'Paul', lastname: 'smith'});
UPDATE users
   SET addresses = address + { 'home': { street: '...', city: 'SF', zip_code: 
94102, phones: {} } }
   WHERE id=2ad...;
{noformat}
where the difference with a map is that the key would be a column name (in 
the CQL3 sense), not a value/literal. Though we might find that a bit confusing 
and find some other syntax.

On the query side, we could optionally allow things like:
{noformat}
SELECT name.firstname, name.lastname FROM users WHERE id=2ad...;
{noformat}

One open question however is what type do we send back in the result set
for a query like:
{noformat}
SELECT name FROM users WHERE id=2ad...;
{noformat}
We could:
# return just that it's the user defined type named {{address}}, but that imply 
the client has to query the cluster metadata to find out the definition of the 
type.
# return the full definition of the type every time.

I also note that client side, it might be a tad harder to support such types 
cleanly in statically type languages than in dynamically typed ones, but that's 
not the end of the world either. 


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[1/3] git commit: unquote uuid in cql string

Updated Branches:
  refs/heads/cassandra-1.2 57e6888e6 - 6abb1d499
  refs/heads/trunk 1c8e627a6 - 342050fcd


unquote uuid in cql string


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6abb1d49
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6abb1d49
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6abb1d49

Branch: refs/heads/cassandra-1.2
Commit: 6abb1d499e850ebc7d3d5ee140a868d743f9f441
Parents: 57e6888
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 12:41:58 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 12:41:58 2013 -0500

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6abb1d49/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java 
b/src/java/org/apache/cassandra/db/SystemTable.java
index ecdad94..187aac5 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -193,7 +193,7 @@ public class SystemTable
  */
 public static void removeTruncationRecord(UUID cfId)
 {
-String req = DELETE truncated_at['%s'] from system.%s WHERE key = 
'%s';
+String req = DELETE truncated_at[%s] from system.%s WHERE key = '%s';
 processInternal(String.format(req, cfId, LOCAL_CF, LOCAL_KEY));
 forceBlockingFlush(LOCAL_CF);
 }

[2/3] git commit: unquote uuid in cql string

unquote uuid in cql string


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/6abb1d49
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/6abb1d49
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/6abb1d49

Branch: refs/heads/trunk
Commit: 6abb1d499e850ebc7d3d5ee140a868d743f9f441
Parents: 57e6888
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 12:41:58 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 12:41:58 2013 -0500

--
 src/java/org/apache/cassandra/db/SystemTable.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/6abb1d49/src/java/org/apache/cassandra/db/SystemTable.java
--
diff --git a/src/java/org/apache/cassandra/db/SystemTable.java 
b/src/java/org/apache/cassandra/db/SystemTable.java
index ecdad94..187aac5 100644
--- a/src/java/org/apache/cassandra/db/SystemTable.java
+++ b/src/java/org/apache/cassandra/db/SystemTable.java
@@ -193,7 +193,7 @@ public class SystemTable
  */
 public static void removeTruncationRecord(UUID cfId)
 {
-String req = DELETE truncated_at['%s'] from system.%s WHERE key = 
'%s';
+String req = DELETE truncated_at[%s] from system.%s WHERE key = '%s';
 processInternal(String.format(req, cfId, LOCAL_CF, LOCAL_KEY));
 forceBlockingFlush(LOCAL_CF);
 }

git commit: fix bad merge

Updated Branches:
  refs/heads/trunk 342050fcd - f5cc4760b


fix bad merge


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f5cc4760
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f5cc4760
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f5cc4760

Branch: refs/heads/trunk
Commit: f5cc4760b993355c695253dca925aa58fffb582f
Parents: 342050f
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 12:46:38 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 12:46:38 2013 -0500

--
 src/java/org/apache/cassandra/db/RowMutation.java |9 -
 1 files changed, 0 insertions(+), 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f5cc4760/src/java/org/apache/cassandra/db/RowMutation.java
--
diff --git a/src/java/org/apache/cassandra/db/RowMutation.java 
b/src/java/org/apache/cassandra/db/RowMutation.java
index 64069c8..d78247b 100644
--- a/src/java/org/apache/cassandra/db/RowMutation.java
+++ b/src/java/org/apache/cassandra/db/RowMutation.java
@@ -247,15 +247,6 @@ public class RowMutation implements IMutation
 return rm;
 }
 
-public RowMutation without(UUID cfId)
-{
-RowMutation rm = new RowMutation(table, key);
-for (Map.EntryUUID, ColumnFamily entry : modifications.entrySet())
-if (!entry.getKey().equals(cfId))
-rm.add(entry.getValue());
-return rm;
-}
-
 public static class RowMutationSerializer implements 
IVersionedSerializerRowMutation
 {
 public void serialize(RowMutation rm, DataOutput out, int version) 
throws IOException

[3/3] git commit: merge from 1.2

merge from 1.2


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/71a7aeae
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/71a7aeae
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/71a7aeae

Branch: refs/heads/trunk
Commit: 71a7aeae5d6a9c2489ad80e0be9b7422c6ac6c43
Parents: f5cc476 cc301de
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 13:16:42 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 13:16:42 2013 -0500

--
 .../apache/cassandra/db/CollationController.java   |4 +---
 .../db/columniterator/IndexedSliceReader.java  |2 ++
 .../db/columniterator/SimpleSliceReader.java   |3 ++-
 3 files changed, 5 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/71a7aeae/src/java/org/apache/cassandra/db/CollationController.java
--
diff --cc src/java/org/apache/cassandra/db/CollationController.java
index eb5192e,908df3b..4bba554
--- a/src/java/org/apache/cassandra/db/CollationController.java
+++ b/src/java/org/apache/cassandra/db/CollationController.java
@@@ -67,8 -76,7 +67,7 @@@ public class CollationControlle
   */
  private ColumnFamily collectTimeOrderedData()
  {
- logger.trace(collectTimeOrderedData);
 -ColumnFamily container = ColumnFamily.create(cfs.metadata, factory, 
filter.filter.isReversed());
 +final ColumnFamily container = 
ArrayBackedSortedColumns.factory.create(cfs.metadata, 
filter.filter.isReversed());
  ListOnDiskAtomIterator iterators = new 
ArrayListOnDiskAtomIterator();
  Tracing.trace(Acquiring sstable references);
  ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/71a7aeae/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
--

http://git-wip-us.apache.org/repos/asf/cassandra/blob/71a7aeae/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
--
diff --cc src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
index 5711fa2,19a73df..6d7055c
--- a/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
+++ b/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
@@@ -29,8 -30,11 +29,9 @@@ import org.apache.cassandra.db.*
  import org.apache.cassandra.db.marshal.AbstractType;
  import org.apache.cassandra.io.sstable.CorruptSSTableException;
  import org.apache.cassandra.io.sstable.Descriptor;
 -import org.apache.cassandra.io.sstable.IndexHelper;
  import org.apache.cassandra.io.sstable.SSTableReader;
  import org.apache.cassandra.io.util.FileDataInput;
 -import org.apache.cassandra.io.util.FileMark;
+ import org.apache.cassandra.tracing.Tracing;
  import org.apache.cassandra.utils.ByteBufferUtil;
  
  class SimpleSliceReader extends AbstractIteratorOnDiskAtom implements 
OnDiskAtomIterator
@@@ -46,7 -52,8 +47,7 @@@
  
  public SimpleSliceReader(SSTableReader sstable, RowIndexEntry indexEntry, 
FileDataInput input, ByteBuffer finishColumn)
  {
- logger.debug(Slicing {}, sstable);
+ Tracing.trace(Seeking to partition beginning in data file);
 -this.sstable = sstable;
  this.finishColumn = finishColumn;
  this.comparator = sstable.metadata.comparator;
  try

[1/3] git commit: tracing improvements

Updated Branches:
  refs/heads/cassandra-1.2 6abb1d499 - cc301de4f
  refs/heads/trunk f5cc4760b - 71a7aeae5


tracing improvements


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc301de4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc301de4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc301de4

Branch: refs/heads/cassandra-1.2
Commit: cc301de4f7236f411a621566f749ff53ad962ca9
Parents: 6abb1d4
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 13:08:52 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 13:08:52 2013 -0500

--
 .../apache/cassandra/db/CollationController.java   |4 +---
 .../db/columniterator/IndexedSliceReader.java  |2 ++
 .../db/columniterator/SimpleSliceReader.java   |2 ++
 3 files changed, 5 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc301de4/src/java/org/apache/cassandra/db/CollationController.java
--
diff --git a/src/java/org/apache/cassandra/db/CollationController.java 
b/src/java/org/apache/cassandra/db/CollationController.java
index dafcdd3..908df3b 100644
--- a/src/java/org/apache/cassandra/db/CollationController.java
+++ b/src/java/org/apache/cassandra/db/CollationController.java
@@ -76,7 +76,6 @@ public class CollationController
  */
 private ColumnFamily collectTimeOrderedData()
 {
-logger.trace(collectTimeOrderedData);
 ColumnFamily container = ColumnFamily.create(cfs.metadata, factory, 
filter.filter.isReversed());
 ListOnDiskAtomIterator iterators = new 
ArrayListOnDiskAtomIterator();
 Tracing.trace(Acquiring sstable references);
@@ -230,7 +229,6 @@ public class CollationController
  */
 private ColumnFamily collectAllData()
 {
-logger.trace(collectAllData);
 Tracing.trace(Acquiring sstable references);
 ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key);
 ListOnDiskAtomIterator iterators = new 
ArrayListOnDiskAtomIterator(Iterables.size(view.memtables) + 
view.sstables.size());
@@ -238,7 +236,7 @@ public class CollationController
 
 try
 {
-Tracing.trace(Merging memtable contents);
+Tracing.trace(Merging memtable tombstones);
 for (Memtable memtable : view.memtables)
 {
 OnDiskAtomIterator iter = 
filter.getMemtableColumnIterator(memtable);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc301de4/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
--
diff --git 
a/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java 
b/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
index 0493819..4ca0ea5 100644
--- a/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
+++ b/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
@@ -39,6 +39,7 @@ import org.apache.cassandra.io.sstable.IndexHelper.IndexInfo;
 import org.apache.cassandra.io.sstable.SSTableReader;
 import org.apache.cassandra.io.util.FileDataInput;
 import org.apache.cassandra.io.util.FileMark;
+import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
@@ -67,6 +68,7 @@ class IndexedSliceReader extends AbstractIteratorOnDiskAtom 
implements OnDiskA
  */
 public IndexedSliceReader(SSTableReader sstable, RowIndexEntry indexEntry, 
FileDataInput input, ColumnSlice[] slices, boolean reversed)
 {
+Tracing.trace(Seeking to partition indexed section in data file);
 this.sstable = sstable;
 this.originalInput = input;
 this.reversed = reversed;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc301de4/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
--
diff --git 
a/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java 
b/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
index 83cdeaa..19a73df 100644
--- a/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
+++ b/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.io.sstable.IndexHelper;
 import org.apache.cassandra.io.sstable.SSTableReader;
 import org.apache.cassandra.io.util.FileDataInput;
 import org.apache.cassandra.io.util.FileMark;
+import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.ByteBufferUtil;
 
 class SimpleSliceReader extends AbstractIteratorOnDiskAtom implements

[2/3] git commit: tracing improvements

tracing improvements


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc301de4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc301de4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc301de4

Branch: refs/heads/trunk
Commit: cc301de4f7236f411a621566f749ff53ad962ca9
Parents: 6abb1d4
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 13:08:52 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 13:08:52 2013 -0500

--
 .../apache/cassandra/db/CollationController.java   |4 +---
 .../db/columniterator/IndexedSliceReader.java  |2 ++
 .../db/columniterator/SimpleSliceReader.java   |2 ++
 3 files changed, 5 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc301de4/src/java/org/apache/cassandra/db/CollationController.java
--
diff --git a/src/java/org/apache/cassandra/db/CollationController.java 
b/src/java/org/apache/cassandra/db/CollationController.java
index dafcdd3..908df3b 100644
--- a/src/java/org/apache/cassandra/db/CollationController.java
+++ b/src/java/org/apache/cassandra/db/CollationController.java
@@ -76,7 +76,6 @@ public class CollationController
  */
 private ColumnFamily collectTimeOrderedData()
 {
-logger.trace(collectTimeOrderedData);
 ColumnFamily container = ColumnFamily.create(cfs.metadata, factory, 
filter.filter.isReversed());
 ListOnDiskAtomIterator iterators = new 
ArrayListOnDiskAtomIterator();
 Tracing.trace(Acquiring sstable references);
@@ -230,7 +229,6 @@ public class CollationController
  */
 private ColumnFamily collectAllData()
 {
-logger.trace(collectAllData);
 Tracing.trace(Acquiring sstable references);
 ColumnFamilyStore.ViewFragment view = cfs.markReferenced(filter.key);
 ListOnDiskAtomIterator iterators = new 
ArrayListOnDiskAtomIterator(Iterables.size(view.memtables) + 
view.sstables.size());
@@ -238,7 +236,7 @@ public class CollationController
 
 try
 {
-Tracing.trace(Merging memtable contents);
+Tracing.trace(Merging memtable tombstones);
 for (Memtable memtable : view.memtables)
 {
 OnDiskAtomIterator iter = 
filter.getMemtableColumnIterator(memtable);

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc301de4/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
--
diff --git 
a/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java 
b/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
index 0493819..4ca0ea5 100644
--- a/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
+++ b/src/java/org/apache/cassandra/db/columniterator/IndexedSliceReader.java
@@ -39,6 +39,7 @@ import org.apache.cassandra.io.sstable.IndexHelper.IndexInfo;
 import org.apache.cassandra.io.sstable.SSTableReader;
 import org.apache.cassandra.io.util.FileDataInput;
 import org.apache.cassandra.io.util.FileMark;
+import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.ByteBufferUtil;
 
 /**
@@ -67,6 +68,7 @@ class IndexedSliceReader extends AbstractIteratorOnDiskAtom 
implements OnDiskA
  */
 public IndexedSliceReader(SSTableReader sstable, RowIndexEntry indexEntry, 
FileDataInput input, ColumnSlice[] slices, boolean reversed)
 {
+Tracing.trace(Seeking to partition indexed section in data file);
 this.sstable = sstable;
 this.originalInput = input;
 this.reversed = reversed;

http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc301de4/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
--
diff --git 
a/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java 
b/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
index 83cdeaa..19a73df 100644
--- a/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
+++ b/src/java/org/apache/cassandra/db/columniterator/SimpleSliceReader.java
@@ -34,6 +34,7 @@ import org.apache.cassandra.io.sstable.IndexHelper;
 import org.apache.cassandra.io.sstable.SSTableReader;
 import org.apache.cassandra.io.util.FileDataInput;
 import org.apache.cassandra.io.util.FileMark;
+import org.apache.cassandra.tracing.Tracing;
 import org.apache.cassandra.utils.ByteBufferUtil;
 
 class SimpleSliceReader extends AbstractIteratorOnDiskAtom implements 
OnDiskAtomIterator
@@ -51,6 +52,7 @@ class SimpleSliceReader extends AbstractIteratorOnDiskAtom 
implements OnDiskAt
 
 public

[jira] [Updated] (CASSANDRA-5590) User defined types for CQL3


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-5590:
--

  Component/s: Core
   API
Fix Version/s: 2.1

 User defined types for CQL3
 ---

 Key: CASSANDRA-5590
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5590
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Sylvain Lebresne
 Fix For: 2.1


 A typical use case for a collection could be to store a bunch of addresses in 
 a user profile. An address could typically be composed of a few properties: 
 say a street, a city, a postal code and maybe a few phone numbers associated 
 to it.
 To model that currently with collections, you might use a {{mapstring, 
 blob}}, where the map key could be a string identifying the address, and the 
 value would be all the infos of an address serialized manually (you can use 
 {{text}} instead of {{blob}} and shove everything in a string if you prefer 
 but the principle is the same).
 This ticket suggests to make this more user friendly by allowing:
 {noformat}
 CREATE TYPE address (
   street text,
   city text,
   zip_code int,
   phones settext
 )
 CREATE TABLE users (
   id uuid PRIMARY KEY,
   name text,
   addresses mapstring, address
 )
 {noformat}
 Under the hood, that type declaration would just be metadata on top of 
 CompositeType (which does mean a limitation would be that we wouldn't allow 
 re-ordering or removal of fields in a custom TYPE). Namely, the {{address}} 
 type would be in practice a {{CompositeType(UTF8Type, UTF8Type, Int32Type, 
 SetType(UTF8Type))}} + some metadata that records the name of each component. 
  In other words, this would mostly be user-friendly syntactic sugar to create 
 composite blobs.
 I'll note that this would also be useful outside collections, as it might 
 sometimes be more efficient/useful to have such simple composite blob. For 
 instance, you could imagine to have a:
 {noformat}
 CREATE TYPE fullname (
   firstname text,
   lastname text
 )
 {noformat}
 and to rewrite the {{users}} table above as
 {noformat}
 CREATE TABLE users (
   id uuid PRIMARY KEY,
   name fullname,
   addresses mapstring, address
 )
 {noformat}
 In terms of inserts we'd need a syntax for those new struct. Could be:
 {noformat}
 INSERT INTO users (id, name)
VALUES (2ad..., { firstname: 'Paul', lastname: 'smith'});
 UPDATE users
SET addresses = address + { 'home': { street: '...', city: 'SF', zip_code: 
 94102, phones: {} } }
WHERE id=2ad...;
 {noformat}
 where the difference with a map is that the key would be a column name (in 
 the CQL3 sense), not a value/literal. Though we might find that a bit 
 confusing and find some other syntax.
 On the query side, we could optionally allow things like:
 {noformat}
 SELECT name.firstname, name.lastname FROM users WHERE id=2ad...;
 {noformat}
 One open question however is what type do we send back in the result set
 for a query like:
 {noformat}
 SELECT name FROM users WHERE id=2ad...;
 {noformat}
 We could:
 # return just that it's the user defined type named {{address}}, but that 
 imply the client has to query the cluster metadata to find out the definition 
 of the type.
 # return the full definition of the type every time.
 I also note that client side, it might be a tad harder to support such types 
 cleanly in statically type languages than in dynamically typed ones, but 
 that's not the end of the world either. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: fix test-clientutil-jar target

Updated Branches:
  refs/heads/trunk 71a7aeae5 - f3ae3


fix test-clientutil-jar target


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3ae
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3ae
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3ae

Branch: refs/heads/trunk
Commit: f3ae3d6b4ded9b649a096f2dc22acfcdad72
Parents: 71a7aea
Author: Dave Brosius dbros...@apache.org
Authored: Thu May 23 15:09:31 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu May 23 15:09:31 2013 -0400

--
 build.xml |   11 ++-
 1 files changed, 10 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/f3ae/build.xml
--
diff --git a/build.xml b/build.xml
index 025f9b7..e8b536c 100644
--- a/build.xml
+++ b/build.xml
@@ -808,6 +808,9 @@
   include name=org/apache/cassandra/utils/ByteBufferUtil*.class /
   include name=org/apache/cassandra/utils/Hex.class /
   include name=org/apache/cassandra/utils/UUIDGen*.class /
+  include name=org/apache/cassandra/utils/FBUtilities*.class /
+  include name=org/apache/cassandra/exceptions/*.class /
+  include name=org/apache/cassandra/utils/CloseableIterator.class /
 /fileset
 manifest
   attribute name=Implementation-Title value=Cassandra/
@@ -1114,7 +1117,13 @@
   classpath
 pathelement location=${test.classes} /
 pathelement 
location=${build.dir}/${ant.project.name}-clientutil-${version}.jar /
-
+pathelement 
location=${build.dir}/${ant.project.name}-thrift-${version}.jar /
+pathelement location=${build.lib}/libthrift-0.9.0.jar /
+pathelement location=${build.lib}/slf4j-api-1.7.2.jar /
+pathelement location=${build.lib}/slf4j-log4j12-1.7.2.jar /
+pathelement location=${build.lib}/log4j-1.2.16.jar /
+pathelement location=${build.lib}/jackson-core-asl-1.9.2.jar /
+pathelement location=${build.lib}/jackson-mapper-asl-1.9.2.jar /
 fileset dir=${build.dir.lib}
   include name=**/junit*.jar /
 /fileset

git commit: Fix dealing with ridiculously large max sstable sizes in LCS patch by jbellis; reviewed by marcuse for CASSANDRA-5589

Updated Branches:
  refs/heads/cassandra-1.2 cc301de4f - 03f011661


Fix dealing with ridiculously large max sstable sizes in LCS
patch by jbellis; reviewed by marcuse for CASSANDRA-5589


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03f01166
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03f01166
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03f01166

Branch: refs/heads/cassandra-1.2
Commit: 03f0116611795836009ea488dd8035b70641ecb3
Parents: cc301de
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 14:50:49 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 14:50:49 2013 -0500

--
 CHANGES.txt|1 +
 .../db/compaction/LeveledCompactionStrategy.java   |5 +
 .../cassandra/db/compaction/LeveledManifest.java   |6 --
 3 files changed, 10 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f01166/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 66c5f04..b687570 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.6
+ * Fix dealing with ridiculously large max sstable sizes in LCS 
(CASSANDRA-5589)
  * Ignore pre-truncate hints (CASSANDRA-4655)
  * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
  * Write row markers when serializing schema (CASSANDRA-5572)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f01166/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
index f964297..52c03fc 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
@@ -57,6 +57,11 @@ public class LeveledCompactionStrategy extends 
AbstractCompactionStrategy implem
 {
 String value = options.containsKey(SSTABLE_SIZE_OPTION) ? 
options.get(SSTABLE_SIZE_OPTION) : 5;
 configuredMaxSSTableSize = Integer.parseInt(value);
+if (configuredMaxSSTableSize = 1000)
+{
+// Yes, people have done this
+logger.warn(Max sstable size of {}MB is configured; having a 
unit of compaction this large is probably a bad idea, 
configuredMaxSSTableSize);
+}
 }
 maxSSTableSizeInMB = configuredMaxSSTableSize;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f01166/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index 864e70c..f62c796 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -70,8 +70,10 @@ public class LeveledManifest
 this.cfs = cfs;
 this.maxSSTableSizeInBytes = maxSSTableSizeInMB * 1024 * 1024;
 
-// allocate enough generations for a PB of data
-int n = (int) Math.log10(1000 * 1000 * 1000 / maxSSTableSizeInMB);
+// allocate enough generations for a PB of data, with a 1-MB sstable 
size.  (Note that if maxSSTableSize is
+// updated, we will still have sstables of the older, potentially 
smaller size.  So don't make this
+// dependent on maxSSTableSize.)
+int n = (int) Math.log10(1000 * 1000 * 1000);
 generations = new List[n];
 lastCompactedKeys = new RowPosition[n];
 for (int i = 0; i  generations.length; i++)

[jira] [Commented] (CASSANDRA-5525) Adding nodes to 1.2 cluster w/ vnodes streamed more data than average node load


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665560#comment-13665560
 ] 

Jeremiah Jordan commented on CASSANDRA-5525:


The disk space thing looks like the case [~rlow] guessed at.  If you enable 
vnodes, you have to run shuffle before you add more nodes.

 Adding nodes to 1.2 cluster w/ vnodes streamed more data than average node 
 load
 ---

 Key: CASSANDRA-5525
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5525
 Project: Cassandra
  Issue Type: Bug
Reporter: John Watson
 Attachments: cass-ring.txt, Screen Shot 2013-04-25 at 12.35.24 PM.png


 12 node cluster upgraded from 1.1.9 to 1.2.3, enabled 'num_tokens: 256', 
 restarted and ran upgradesstables and cleanup.
 Tried to join 2 additional nodes into the ring.
 However, 1 of the new nodes ran out of disk space. This started causing 'no 
 host id' alerts in the live cluster when attempting to store hints for that 
 node.
 {noformat}
 ERROR 10:12:02,408 Exception in thread Thread[MutationStage:190,5,main]
 java.lang.AssertionError: Missing host ID 
 {noformat}
 The other node I killed to stop it from continuing to join. Since the live 
 cluster was now in some sort of broken state dropping mutation messages on 3 
 nodes. This was fixed by restarting them, however 1 node never stopped, so 
 had to decomm it (leaving the original cluster at 11 nodes.)
 Ring pre-join:
 {noformat}
 Load   Tokens  Owns (effective)  Host ID 
 147.55 GB  256 16.7% 754f9f4c-4ba7-4495-97e7-1f5b6755cb27
 124.99 GB  256 16.7% 93f4400a-09d9-4ca0-b6a6-9bcca2427450
 136.63 GB  256 16.7% ff821e8e-b2ca-48a9-ac3f-8234b16329ce
 141.78 GB  253 100.0%339c474f-cf19-4ada-9a47-8b10912d5eb3
 137.74 GB  256 16.7% 6d726cbf-147d-426e-a735-e14928c95e45
 135.9 GB   256 16.7% e59a02b3-8b91-4abd-990e-b3cb2a494950
 165.96 GB  256 16.7% 83ca527c-60c5-4ea0-89a8-de53b92b99c8
 135.41 GB  256 16.7% c3ea4026-551b-4a14-a346-480e8c1fe283
 143.38 GB  256 16.7% df7ba879-74ad-400b-b371-91b45dcbed37
 178.05 GB  256 25.0% 78192d73-be0b-4d49-a129-9bec0770efed
 194.92 GB  256 25.0% 361d7e31-b155-4ce1-8890-451b3ddf46cf
 150.5 GB   256 16.7% 9889280a-1433-439e-bb84-6b7e7f44d761
 {noformat}
 Ring after decomm bad node:
 {noformat}
 Load   Tokens  Owns (effective)  Host ID
 80.95 GB   256 16.7% 754f9f4c-4ba7-4495-97e7-1f5b6755cb27
 87.15 GB   256 16.7% 93f4400a-09d9-4ca0-b6a6-9bcca2427450
 98.16 GB   256 16.7% ff821e8e-b2ca-48a9-ac3f-8234b16329ce
 142.6 GB   253 100.0%339c474f-cf19-4ada-9a47-8b10912d5eb3
 77.64 GB   256 16.7% e59a02b3-8b91-4abd-990e-b3cb2a494950
 194.31 GB  256 25.0% 6d726cbf-147d-426e-a735-e14928c95e45
 221.94 GB  256 33.3% 83ca527c-60c5-4ea0-89a8-de53b92b99c8
 87.61 GB   256 16.7% c3ea4026-551b-4a14-a346-480e8c1fe283
 101.02 GB  256 16.7% df7ba879-74ad-400b-b371-91b45dcbed37
 172.44 GB  256 25.0% 78192d73-be0b-4d49-a129-9bec0770efed
 108.5 GB   256 16.7% 9889280a-1433-439e-bb84-6b7e7f44d761
 {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Fix dealing with ridiculously large max sstable sizes in LCS patch by jbellis; reviewed by marcuse for CASSANDRA-5589

Updated Branches:
  refs/heads/trunk f3ae3 - 98b9cb47f


Fix dealing with ridiculously large max sstable sizes in LCS
patch by jbellis; reviewed by marcuse for CASSANDRA-5589


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/98b9cb47
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/98b9cb47
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/98b9cb47

Branch: refs/heads/trunk
Commit: 98b9cb47f991937b5e92178856bb6017ff6c13c0
Parents: f3a
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 14:50:49 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 15:03:31 2013 -0500

--
 CHANGES.txt|1 +
 .../db/compaction/LeveledCompactionStrategy.java   |5 +
 .../cassandra/db/compaction/LeveledManifest.java   |6 --
 3 files changed, 10 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/98b9cb47/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index d283b72..2b6e38f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -53,6 +53,7 @@
 
 
 1.2.6
+ * Fix dealing with ridiculously large max sstable sizes in LCS 
(CASSANDRA-5589)
  * Ignore pre-truncate hints (CASSANDRA-4655)
  * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
  * Write row markers when serializing schema (CASSANDRA-5572)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/98b9cb47/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
index 2f1c1e8..d29dead 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
@@ -58,6 +58,11 @@ public class LeveledCompactionStrategy extends 
AbstractCompactionStrategy implem
 {
 String value = options.containsKey(SSTABLE_SIZE_OPTION) ? 
options.get(SSTABLE_SIZE_OPTION) : 5;
 configuredMaxSSTableSize = Integer.parseInt(value);
+if (configuredMaxSSTableSize = 1000)
+{
+// Yes, people have done this
+logger.warn(Max sstable size of {}MB is configured; having a 
unit of compaction this large is probably a bad idea, 
configuredMaxSSTableSize);
+}
 }
 maxSSTableSizeInMB = configuredMaxSSTableSize;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/98b9cb47/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index fb4244d..f43c13e 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -65,8 +65,10 @@ public class LeveledManifest
 this.maxSSTableSizeInBytes = maxSSTableSizeInMB * 1024 * 1024;
 this.options = options;
 
-// allocate enough generations for a PB of data
-int n = (int) Math.log10(1000 * 1000 * 1000 / maxSSTableSizeInMB);
+// allocate enough generations for a PB of data, with a 1-MB sstable 
size.  (Note that if maxSSTableSize is
+// updated, we will still have sstables of the older, potentially 
smaller size.  So don't make this
+// dependent on maxSSTableSize.)
+int n = (int) Math.log10(1000 * 1000 * 1000);
 generations = new List[n];
 lastCompactedKeys = new RowPosition[n];
 for (int i = 0; i  generations.length; i++)

[jira] [Commented] (CASSANDRA-5514) Allow timestamp hints


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665696#comment-13665696
 ] 

Jonathan Ellis commented on CASSANDRA-5514:
---

LGTM in principle.  Made some tweaks in 
https://github.com/jbellis/cassandra/commits/5514.  Still a bit unclear about 
DCT -- added a hack in the last commit to always return true, but do we need to 
do anything on the collector side?

 Allow timestamp hints
 -

 Key: CASSANDRA-5514
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5514
 Project: Cassandra
  Issue Type: New Feature
  Components: API, Core
Reporter: Jonathan Ellis
Assignee: Marcus Eriksson
 Fix For: 2.0

 Attachments: 0001-CASSANDRA-5514-v1.patch, 
 0001-CASSANDRA-5514-v2.patch


 Slice queries can't optimize based on timestamp except for rare cases 
 (CASSANDRA-4116).  However, many common queries involve an implicit time 
 component, where the application author knows that he is only interested in 
 data more recent than X, or older than Y.
 We could use the per-sstable max and min timestamps we track to avoid 
 touching cold data if we could pass a hint to Cassandra about the time range 
 we care about.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5591) Window failure renaming LCS json.

Jeremiah Jordan created CASSANDRA-5591:
--

 Summary: Window failure renaming LCS json.
 Key: CASSANDRA-5591
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5591
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Windows
Reporter: Jeremiah Jordan


Had someone report that on Windows, under load, the LCS json file sometimes 
fails to be renamed.

{noformat}
ERROR [CompactionExecutor:1] 2013-05-23 14:43:55,848 CassandraDaemon.java (line 
174) Exception in thread Thread[CompactionExecutor:1,1,main]
 java.lang.RuntimeException: Failed to rename C:\development\tools\DataStax 
Community\data\data\zzz\zzz\zzz.json to C:\development\tools\DataStax 
Community\data\data\zzz\zzz\zzz-old.json
at 
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:133)
at 
org.apache.cassandra.db.compaction.LeveledManifest.serialize(LeveledManifest.java:617)
at 
org.apache.cassandra.db.compaction.LeveledManifest.promote(LeveledManifest.java:229)
at 
org.apache.cassandra.db.compaction.LeveledCompactionStrategy.handleNotification(LeveledCompactionStrategy.java:155)
at 
org.apache.cassandra.db.DataTracker.notifySSTablesChanged(DataTracker.java:410)
at 
org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:223)
at 
org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:991)
at 
org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:230)
at 
org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at 
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at 
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:58)
at 
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
at 
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:188)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:662)
{noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5555) Allow sstableloader to handle a larger number of files


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-:
--

Assignee: Dave Brosius  (was: Tyler Hobbs)

How much of a rewrite would this be, [~dbrosius]?

 Allow sstableloader to handle a larger number of files
 --

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Dave Brosius

 With the default heap size, sstableloader will OOM when there are roughly 25k 
 files in the directory to load.  It's easy to reach this number of files in a 
 single LCS column family.
 By avoiding creating all SSTableReaders up front in SSTableLoader, we should 
 be able to increase the number of files that sstableloader can handle 
 considerably.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5555) Allow sstableloader to handle a larger number of files


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-:
--

Fix Version/s: 1.2.6

 Allow sstableloader to handle a larger number of files
 --

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Dave Brosius
 Fix For: 1.2.6


 With the default heap size, sstableloader will OOM when there are roughly 25k 
 files in the directory to load.  It's easy to reach this number of files in a 
 single LCS column family.
 By avoiding creating all SSTableReaders up front in SSTableLoader, we should 
 be able to increase the number of files that sstableloader can handle 
 considerably.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor

2013-05-23 Thread Pavel Yaskevich (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-5582:
---

Attachment: (was: CASSANDRA-5582.patch)

 Replace CustomHsHaServer with better optimized solution based on LMAX 
 Disruptor
 ---

 Key: CASSANDRA-5582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 2.0

 Attachments: CASSANDRA-5530-invoker-fix.patch, disruptor-3.0.1.jar, 
 Pavel's Patch.rtf


 I have been working on https://github.com/xedin/disruptor_thrift_server and 
 consider it as stable and performant enough for integration with Cassandra. 
 Proposed replacement can work in both on/off Heap modes (depending if JNA is 
 available) and doesn't blindly reallocate things, which allows to resolve 
 CASSANDRA-4265 as Won't Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor

2013-05-23 Thread Pavel Yaskevich (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-5582:
---

Attachment: (was: disruptor-thrift-0.1-SNAPSHOT.jar)

 Replace CustomHsHaServer with better optimized solution based on LMAX 
 Disruptor
 ---

 Key: CASSANDRA-5582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 2.0

 Attachments: CASSANDRA-5530-invoker-fix.patch, disruptor-3.0.1.jar, 
 Pavel's Patch.rtf


 I have been working on https://github.com/xedin/disruptor_thrift_server and 
 consider it as stable and performant enough for integration with Cassandra. 
 Proposed replacement can work in both on/off Heap modes (depending if JNA is 
 available) and doesn't blindly reallocate things, which allows to resolve 
 CASSANDRA-4265 as Won't Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (CASSANDRA-5582) Replace CustomHsHaServer with better optimized solution based on LMAX Disruptor

2013-05-23 Thread Pavel Yaskevich (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Yaskevich updated CASSANDRA-5582:
---

Attachment: CASSANDRA-5582.patch
disruptor-thrift-0.1-SNAPSHOT.jar

patch is rebased with latest trunk and server is defaults are changed. I'm 
pretty confident about everything right now, tested again on 4 node cluster 
(RF=3) and p999 latencies are close to p95 and stable.

 Replace CustomHsHaServer with better optimized solution based on LMAX 
 Disruptor
 ---

 Key: CASSANDRA-5582
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5582
 Project: Cassandra
  Issue Type: Improvement
  Components: API, Core
Reporter: Pavel Yaskevich
Assignee: Pavel Yaskevich
 Fix For: 2.0

 Attachments: CASSANDRA-5530-invoker-fix.patch, CASSANDRA-5582.patch, 
 disruptor-3.0.1.jar, disruptor-thrift-0.1-SNAPSHOT.jar, Pavel's Patch.rtf


 I have been working on https://github.com/xedin/disruptor_thrift_server and 
 consider it as stable and performant enough for integration with Cassandra. 
 Proposed replacement can work in both on/off Heap modes (depending if JNA is 
 available) and doesn't blindly reallocate things, which allows to resolve 
 CASSANDRA-4265 as Won't Fix.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (CASSANDRA-5592) Flaw in documentation

2013-05-23 Thread Matt Herold (JIRA)

Matt Herold created CASSANDRA-5592:
--

 Summary: Flaw in documentation
 Key: CASSANDRA-5592
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5592
 Project: Cassandra
  Issue Type: Task
  Components: Documentation  website
Affects Versions: 1.2.5
Reporter: Matt Herold
Priority: Trivial


On http://www.datastax.com/docs/1.2/cql_cli/using/collections_map, the 
following code snippet seems not to work as expected: 

 ALTER TABLE users ADD todo maptimestamp, reminder_text;
 Bad Request: line 1:42 no viable alternative at input 'reminder_text'
 



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-4905) Repair should exclude gcable tombstones from merkle-tree computation

2013-05-23 Thread Michael Theroux (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-4905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665806#comment-13665806
 ] 

Michael Theroux commented on CASSANDRA-4905:


To followup on my previous comment, I performed the test I described.  The 
results were quite incredible.  I brought up three nodes that represented one 
token range and its replicas.  These were brought up from very recent 
snapshots, so some inconsistency was expected.  I ran the test twice with and 
without the patch, and on the same data, and periodically monitored the number 
of pending compaction tasks.  Below you see a time that it was monitored, and 
the number of pending compactions during that time

Without the Patch:

start: 10:30
11:22 - 196
12:33 - 112
13:19 - 1558
14:03 - 1579
14:48 - 1356
15:25 - 1181
16:49 - 752
17:30 - 657
17:52 - 548
18:56 - 202
19:36 - 29
01:50 - 0

With the patch:

start: 3:47
4:34 - 1
4:40 - 1
4:50 - 32
5:01 - 209
5:54 - 1
6:50 - 3 (all streaming from compaction complete)
6:54 - Repair complete, no compactions

Not only was this a very efficient repair from the point of view of the number 
of compactions, it also completed in a little over 3 hours, which is equally as 
dramatic (validation typically lasts several hours for us).

 Repair should exclude gcable tombstones from merkle-tree computation
 

 Key: CASSANDRA-4905
 URL: https://issues.apache.org/jira/browse/CASSANDRA-4905
 Project: Cassandra
  Issue Type: Improvement
  Components: Core
Reporter: Christian Spriegel
Assignee: Sylvain Lebresne
 Fix For: 1.2.0 beta 3

 Attachments: 4905.txt


 Currently gcable tombstones get repaired if some replicas compacted already, 
 but some are not compacted.
 This could be avoided by ignoring all gcable tombstones during merkle tree 
 calculation.
 This was discussed with Sylvain on the mailing list:
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/repair-compaction-and-tombstone-rows-td7583481.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: Fix addColumn order in BM.getBatchlogMutationFor()

Updated Branches:
  refs/heads/trunk 98b9cb47f - 22d117302


Fix addColumn order in BM.getBatchlogMutationFor()


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/22d11730
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/22d11730
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/22d11730

Branch: refs/heads/trunk
Commit: 22d1173021772cfb445cd60c7d6b8f2ef5ca193b
Parents: 98b9cb4
Author: Aleksey Yeschenko alek...@apache.org
Authored: Fri May 24 02:24:16 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Fri May 24 02:24:16 2013 +0300

--
 .../org/apache/cassandra/db/BatchlogManager.java   |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/22d11730/src/java/org/apache/cassandra/db/BatchlogManager.java
--
diff --git a/src/java/org/apache/cassandra/db/BatchlogManager.java 
b/src/java/org/apache/cassandra/db/BatchlogManager.java
index 6f9cb35..48d86e3 100644
--- a/src/java/org/apache/cassandra/db/BatchlogManager.java
+++ b/src/java/org/apache/cassandra/db/BatchlogManager.java
@@ -122,8 +122,8 @@ public class BatchlogManager implements BatchlogManagerMBean
 
 ColumnFamily cf = 
ArrayBackedSortedColumns.factory.create(CFMetaData.BatchlogCf);
 cf.addColumn(new Column(columnName(), 
ByteBufferUtil.EMPTY_BYTE_BUFFER, timestamp));
-cf.addColumn(new Column(columnName(written_at), writtenAt, 
timestamp));
 cf.addColumn(new Column(columnName(data), data, timestamp));
+cf.addColumn(new Column(columnName(written_at), writtenAt, 
timestamp));
 
 return new RowMutation(Table.SYSTEM_KS, 
UUIDType.instance.decompose(uuid), cf);
 }

git commit: Fix ReadCallback.get() TimeUnit

Updated Branches:
  refs/heads/trunk 22d117302 - b2ba251ac


Fix ReadCallback.get() TimeUnit


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b2ba251a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b2ba251a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b2ba251a

Branch: refs/heads/trunk
Commit: b2ba251ac60952fb96f34253340cf111677d17c8
Parents: 22d1173
Author: Aleksey Yeschenko alek...@apache.org
Authored: Fri May 24 05:39:17 2013 +0300
Committer: Aleksey Yeschenko alek...@apache.org
Committed: Fri May 24 05:39:17 2013 +0300

--
 .../org/apache/cassandra/service/ReadCallback.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/b2ba251a/src/java/org/apache/cassandra/service/ReadCallback.java
--
diff --git a/src/java/org/apache/cassandra/service/ReadCallback.java 
b/src/java/org/apache/cassandra/service/ReadCallback.java
index 3e25aef..79e15b1 100644
--- a/src/java/org/apache/cassandra/service/ReadCallback.java
+++ b/src/java/org/apache/cassandra/service/ReadCallback.java
@@ -97,7 +97,7 @@ public class ReadCallbackTMessage, TResolved implements 
IAsyncCallbackTMessag
 
 public TResolved get() throws ReadTimeoutException, DigestMismatchException
 {
-if (!await(command.getTimeout(), TimeUnit.MICROSECONDS))
+if (!await(command.getTimeout(), TimeUnit.MILLISECONDS))
 {
 ReadTimeoutException ex = new 
ReadTimeoutException(consistencyLevel, received.get(), blockfor, 
resolver.isDataPresent());
 if (logger.isDebugEnabled())

[jira] [Created] (CASSANDRA-5593) Auth.isExistingUser is periodically throwing org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE

2013-05-23 Thread Justin Geiser (JIRA)

Justin Geiser created CASSANDRA-5593:


 Summary: Auth.isExistingUser is periodically throwing 
org.apache.cassandra.exceptions.UnavailableException: Cannot achieve 
consistency level ONE
 Key: CASSANDRA-5593
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5593
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Three node cluster
Reporter: Justin Geiser


When setting up authentication on a clustered setup we're periodically getting 
an UnavailableException: Cannot achieve consistency level ONE whenever one or 
two of the cluster nodes is down.

{code}
java.lang.RuntimeException: 
org.apache.cassandra.exceptions.UnavailableException: Cannot achieve 
consistency level ONE
at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:75)
at 
com.resolve.cassandra.auth.SimpleAuthenticator.setup(SimpleAuthenticator.java:273)
at org.apache.cassandra.auth.Auth.setup(Auth.java:139)
at 
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:781)
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
at 
org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve 
consistency level ONE
at 
org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:250)
at 
org.apache.cassandra.service.ReadCallback.assureSufficientLiveNodes(ReadCallback.java:152)
at 
org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:891)
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:829)
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:126)
at 
org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:1)
at 
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:132)
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:143)
at 
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:151)
at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:71)
... 8 more
{code}

Digging into the issue it looks like the problem is the 
SimpleStrategy.calculateNaturalEndpoints method is only returning 1 entry, 
because the replication factor for the system_auth column family is 1, and if 
this node happens to be one of the down nodes it gets removed by 
getLiveNaturalEndpoints in StorageService.  So by the time it reaches 
StorageProxy.fetchRows(StorageProxy.java:891) the endpoints list is empty, even 
though we have valid nodes running.

For a quick fix I removed the endpoints.size()  replicas check from the 
while loop in SimpleStrategy.calculateNaturalEndpoints:

{code}
public ListInetAddress calculateNaturalEndpoints(Token token, 
TokenMetadata metadata)
{
int replicas = getReplicationFactor();
ArrayListToken tokens = metadata.sortedTokens();
ListInetAddress endpoints = new ArrayListInetAddress(replicas);

if (tokens.isEmpty())
return endpoints;

// Add the token at the index by default
IteratorToken iter = TokenMetadata.ringIterator(tokens, token, false);
while (iter.hasNext())
{
InetAddress ep = metadata.getEndpoint(iter.next());
if (!endpoints.contains(ep))
{
endpoints.add(ep);
}
}
return endpoints;
}
{code}


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

git commit: add missing white space to message

Updated Branches:
  refs/heads/cassandra-1.2 03f011661 - 54c15d826


add missing white space to message


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54c15d82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54c15d82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54c15d82

Branch: refs/heads/cassandra-1.2
Commit: 54c15d826b4c414e0f22f5eedf023dfe53d2eae4
Parents: 03f0116
Author: Dave Brosius dbros...@apache.org
Authored: Thu May 23 22:54:50 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu May 23 22:54:50 2013 -0400

--
 .../apache/cassandra/io/sstable/SSTableLoader.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/54c15d82/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
index 32dbbc3..9965138 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
@@ -124,7 +124,7 @@ public class SSTableLoader
 }
 
 MapInetAddress, CollectionRangeToken endpointToRanges = 
client.getEndpointToRangesMap();
-outputHandler.output(String.format(Streaming revelant part of %sto 
%s, names(sstables), endpointToRanges.keySet()));
+outputHandler.output(String.format(Streaming revelant part of %s to 
%s, names(sstables), endpointToRanges.keySet()));
 
 // There will be one streaming session by endpoint
 LoaderFuture future = new LoaderFuture(endpointToRanges.size());

[2/3] git commit: add missing white space to message

add missing white space to message


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/54c15d82
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/54c15d82
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/54c15d82

Branch: refs/heads/trunk
Commit: 54c15d826b4c414e0f22f5eedf023dfe53d2eae4
Parents: 03f0116
Author: Dave Brosius dbros...@apache.org
Authored: Thu May 23 22:54:50 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Thu May 23 22:54:50 2013 -0400

--
 .../apache/cassandra/io/sstable/SSTableLoader.java |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/54c15d82/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java 
b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
index 32dbbc3..9965138 100644
--- a/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
+++ b/src/java/org/apache/cassandra/io/sstable/SSTableLoader.java
@@ -124,7 +124,7 @@ public class SSTableLoader
 }
 
 MapInetAddress, CollectionRangeToken endpointToRanges = 
client.getEndpointToRangesMap();
-outputHandler.output(String.format(Streaming revelant part of %sto 
%s, names(sstables), endpointToRanges.keySet()));
+outputHandler.output(String.format(Streaming revelant part of %s to 
%s, names(sstables), endpointToRanges.keySet()));
 
 // There will be one streaming session by endpoint
 LoaderFuture future = new LoaderFuture(endpointToRanges.size());

[1/3] git commit: Fix dealing with ridiculously large max sstable sizes in LCS patch by jbellis; reviewed by marcuse for CASSANDRA-5589

Updated Branches:
  refs/heads/trunk b2ba251ac - 778b6a0eb


Fix dealing with ridiculously large max sstable sizes in LCS
patch by jbellis; reviewed by marcuse for CASSANDRA-5589


Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03f01166
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03f01166
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03f01166

Branch: refs/heads/trunk
Commit: 03f0116611795836009ea488dd8035b70641ecb3
Parents: cc301de
Author: Jonathan Ellis jbel...@apache.org
Authored: Thu May 23 14:50:49 2013 -0500
Committer: Jonathan Ellis jbel...@apache.org
Committed: Thu May 23 14:50:49 2013 -0500

--
 CHANGES.txt|1 +
 .../db/compaction/LeveledCompactionStrategy.java   |5 +
 .../cassandra/db/compaction/LeveledManifest.java   |6 --
 3 files changed, 10 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f01166/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 66c5f04..b687570 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 1.2.6
+ * Fix dealing with ridiculously large max sstable sizes in LCS 
(CASSANDRA-5589)
  * Ignore pre-truncate hints (CASSANDRA-4655)
  * Move System.exit on OOM into a separate thread (CASSANDRA-5273)
  * Write row markers when serializing schema (CASSANDRA-5572)

http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f01166/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
--
diff --git 
a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
index f964297..52c03fc 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledCompactionStrategy.java
@@ -57,6 +57,11 @@ public class LeveledCompactionStrategy extends 
AbstractCompactionStrategy implem
 {
 String value = options.containsKey(SSTABLE_SIZE_OPTION) ? 
options.get(SSTABLE_SIZE_OPTION) : 5;
 configuredMaxSSTableSize = Integer.parseInt(value);
+if (configuredMaxSSTableSize = 1000)
+{
+// Yes, people have done this
+logger.warn(Max sstable size of {}MB is configured; having a 
unit of compaction this large is probably a bad idea, 
configuredMaxSSTableSize);
+}
 }
 maxSSTableSizeInMB = configuredMaxSSTableSize;
 

http://git-wip-us.apache.org/repos/asf/cassandra/blob/03f01166/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java 
b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
index 864e70c..f62c796 100644
--- a/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
+++ b/src/java/org/apache/cassandra/db/compaction/LeveledManifest.java
@@ -70,8 +70,10 @@ public class LeveledManifest
 this.cfs = cfs;
 this.maxSSTableSizeInBytes = maxSSTableSizeInMB * 1024 * 1024;
 
-// allocate enough generations for a PB of data
-int n = (int) Math.log10(1000 * 1000 * 1000 / maxSSTableSizeInMB);
+// allocate enough generations for a PB of data, with a 1-MB sstable 
size.  (Note that if maxSSTableSize is
+// updated, we will still have sstables of the older, potentially 
smaller size.  So don't make this
+// dependent on maxSSTableSize.)
+int n = (int) Math.log10(1000 * 1000 * 1000);
 generations = new List[n];
 lastCompactedKeys = new RowPosition[n];
 for (int i = 0; i  generations.length; i++)

[jira] [Resolved] (CASSANDRA-5593) Auth.isExistingUser is periodically throwing org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE

2013-05-23 Thread Aleksey Yeschenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko resolved CASSANDRA-5593.
--

Resolution: Invalid

This is not a good fix. At all.

This behavior is expected. What you should've done instead was to increase RF 
on system_auth keyspace (with a regular ALTER KEYSPACE system_auth .. query).

 Auth.isExistingUser is periodically throwing 
 org.apache.cassandra.exceptions.UnavailableException: Cannot achieve 
 consistency level ONE
 ---

 Key: CASSANDRA-5593
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5593
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Three node cluster
Reporter: Justin Geiser

 When setting up authentication on a clustered setup we're periodically 
 getting an UnavailableException: Cannot achieve consistency level ONE 
 whenever one or two of the cluster nodes is down.
 {code}
 java.lang.RuntimeException: 
 org.apache.cassandra.exceptions.UnavailableException: Cannot achieve 
 consistency level ONE
 at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:75)
 at 
 com.resolve.cassandra.auth.SimpleAuthenticator.setup(SimpleAuthenticator.java:273)
 at org.apache.cassandra.auth.Auth.setup(Auth.java:139)
 at 
 org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:781)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
 Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot 
 achieve consistency level ONE
 at 
 org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:250)
 at 
 org.apache.cassandra.service.ReadCallback.assureSufficientLiveNodes(ReadCallback.java:152)
 at 
 org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:891)
 at 
 org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:829)
 at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:126)
 at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:1)
 at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:132)
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:143)
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:151)
 at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:71)
 ... 8 more
 {code}
 Digging into the issue it looks like the problem is the 
 SimpleStrategy.calculateNaturalEndpoints method is only returning 1 entry, 
 because the replication factor for the system_auth column family is 1, and if 
 this node happens to be one of the down nodes it gets removed by 
 getLiveNaturalEndpoints in StorageService.  So by the time it reaches 
 StorageProxy.fetchRows(StorageProxy.java:891) the endpoints list is empty, 
 even though we have valid nodes running.
 For a quick fix I removed the endpoints.size()  replicas check from the 
 while loop in SimpleStrategy.calculateNaturalEndpoints:
 {code}
 public ListInetAddress calculateNaturalEndpoints(Token token, 
 TokenMetadata metadata)
 {
 int replicas = getReplicationFactor();
 ArrayListToken tokens = metadata.sortedTokens();
 ListInetAddress endpoints = new ArrayListInetAddress(replicas);
 if (tokens.isEmpty())
 return endpoints;
 // Add the token at the index by default
 IteratorToken iter = TokenMetadata.ringIterator(tokens, token, 
 false);
 while (iter.hasNext())
 {
 InetAddress ep = metadata.getEndpoint(iter.next());
 if (!endpoints.contains(ep))
 {
 endpoints.add(ep);
 }
 }
 return endpoints;
 }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5593) Auth.isExistingUser is periodically throwing org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE

2013-05-23 Thread Justin Geiser (JIRA)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-5593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665971#comment-13665971
 ] 

Justin Geiser commented on CASSANDRA-5593:
--

Didn't think it was a good fix, just a quick one.  Didn't think it would be 
quite that bad though :).  The problem is the consitencyForUser in Auth is hard 
coded as ONE (except for the super user), and because the endpoint list is 
cached the isExistingUser will fail every time until the node it picked the 
first time comes back up.  Because we don't allow anonymous authentication, and 
aren't using the super user, the node we're trying to access is effectively 
hosed until the down node comes back up.

 Auth.isExistingUser is periodically throwing 
 org.apache.cassandra.exceptions.UnavailableException: Cannot achieve 
 consistency level ONE
 ---

 Key: CASSANDRA-5593
 URL: https://issues.apache.org/jira/browse/CASSANDRA-5593
 Project: Cassandra
  Issue Type: Bug
  Components: Core
Affects Versions: 1.2.4
 Environment: Three node cluster
Reporter: Justin Geiser

 When setting up authentication on a clustered setup we're periodically 
 getting an UnavailableException: Cannot achieve consistency level ONE 
 whenever one or two of the cluster nodes is down.
 {code}
 java.lang.RuntimeException: 
 org.apache.cassandra.exceptions.UnavailableException: Cannot achieve 
 consistency level ONE
 at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:75)
 at 
 com.resolve.cassandra.auth.SimpleAuthenticator.setup(SimpleAuthenticator.java:273)
 at org.apache.cassandra.auth.Auth.setup(Auth.java:139)
 at 
 org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:781)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:542)
 at 
 org.apache.cassandra.service.StorageService.initServer(StorageService.java:439)
 at 
 org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:323)
 at 
 org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:411)
 at 
 org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:454)
 Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot 
 achieve consistency level ONE
 at 
 org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:250)
 at 
 org.apache.cassandra.service.ReadCallback.assureSufficientLiveNodes(ReadCallback.java:152)
 at 
 org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:891)
 at 
 org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:829)
 at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:126)
 at 
 org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:1)
 at 
 org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:132)
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:143)
 at 
 org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:151)
 at org.apache.cassandra.auth.Auth.isExistingUser(Auth.java:71)
 ... 8 more
 {code}
 Digging into the issue it looks like the problem is the 
 SimpleStrategy.calculateNaturalEndpoints method is only returning 1 entry, 
 because the replication factor for the system_auth column family is 1, and if 
 this node happens to be one of the down nodes it gets removed by 
 getLiveNaturalEndpoints in StorageService.  So by the time it reaches 
 StorageProxy.fetchRows(StorageProxy.java:891) the endpoints list is empty, 
 even though we have valid nodes running.
 For a quick fix I removed the endpoints.size()  replicas check from the 
 while loop in SimpleStrategy.calculateNaturalEndpoints:
 {code}
 public ListInetAddress calculateNaturalEndpoints(Token token, 
 TokenMetadata metadata)
 {
 int replicas = getReplicationFactor();
 ArrayListToken tokens = metadata.sortedTokens();
 ListInetAddress endpoints = new ArrayListInetAddress(replicas);
 if (tokens.isEmpty())
 return endpoints;
 // Add the token at the index by default
 IteratorToken iter = TokenMetadata.ringIterator(tokens, token, 
 false);
 while (iter.hasNext())
 {
 InetAddress ep = metadata.getEndpoint(iter.next());
 if (!endpoints.contains(ep))
 {
 endpoints.add(ep);
 }
 }
 return endpoints;
 }
 {code}

--
This message is automatically generated by JIRA.
If you think it was sent

[jira] [Updated] (CASSANDRA-5555) Allow sstableloader to handle a larger number of files

2013-05-23 Thread Dave Brosius (JIRA)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dave Brosius updated CASSANDRA-:


Attachment: -01.txt

A premature patch added to see if approach is relatively ok, haven't formulated 
tests yet, so *definitely* not ready yet.

Also not that this implementation leaks into StorageService, which is probably 
not desired, and a real fix probably wants a separate interface into 
SSTableLoader.stream() as opposed to a complete replace. But not sure.

 Allow sstableloader to handle a larger number of files
 --

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Dave Brosius
 Fix For: 1.2.6

 Attachments: -01.txt


 With the default heap size, sstableloader will OOM when there are roughly 25k 
 files in the directory to load.  It's easy to reach this number of files in a 
 single LCS column family.
 By avoiding creating all SSTableReaders up front in SSTableLoader, we should 
 be able to increase the number of files that sstableloader can handle 
 considerably.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5555) Allow sstableloader to handle a larger number of files

2013-05-23 Thread Dave Brosius (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665977#comment-13665977
]

Dave Brosius edited comment on CASSANDRA- at 5/24/13 3:50 AM:
--

A premature patch added to see if approach is relatively ok, haven't formulated
tests yet, so *definitely* not ready yet.

Also note that this implementation leaks into StorageService, which is probably
not desired, and a real fix probably wants a separate interface into
SSTableLoader.stream() as opposed to a complete replace. But not sure.

was (Author: dbrosius):
A premature patch added to see if approach is relatively ok, haven't
formulated tests yet, so *definitely* not ready yet.

Also not that this implementation leaks into StorageService, which is probably
not desired, and a real fix probably wants a separate interface into
SSTableLoader.stream() as opposed to a complete replace. But not sure.

Allow sstableloader to handle a larger number of files
--

Key: CASSANDRA-
URL: https://issues.apache.org/jira/browse/CASSANDRA-
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Tyler Hobbs
Assignee: Dave Brosius
Fix For: 1.2.6

Attachments: -01.txt

With the default heap size, sstableloader will OOM when there are roughly 25k
files in the directory to load. It's easy to reach this number of files in a
single LCS column family.
By avoiding creating all SSTableReaders up front in SSTableLoader, we should
be able to increase the number of files that sstableloader can handle
considerably.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (CASSANDRA-5555) Allow sstableloader to handle a larger number of files

2013-05-23 Thread Dave Brosius (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665977#comment-13665977
]

Dave Brosius edited comment on CASSANDRA- at 5/24/13 3:51 AM:
--

A premature patch added to see if approach is relatively ok, haven't formulated
tests yet, so *definitely* not ready yet.

(patch against trunk)

was (Author: dbrosius):
A premature patch added to see if approach is relatively ok, haven't
formulated tests yet, so *definitely* not ready yet.

Allow sstableloader to handle a larger number of files
--

Key: CASSANDRA-
URL: https://issues.apache.org/jira/browse/CASSANDRA-
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Tyler Hobbs
Assignee: Dave Brosius
Fix For: 1.2.6

Attachments: -01.txt

[jira] [Commented] (CASSANDRA-5351) Avoid repairing already-repaired data by default

2013-05-23 Thread Charlie Groves (JIRA)

[
https://issues.apache.org/jira/browse/CASSANDRA-5351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13665995#comment-13665995
]

Charlie Groves commented on CASSANDRA-5351:
---

bq. So the two options that I see are (1) making ranges repaired, rather than
sstables, or (2) anti-compacting repaired parts into new sstables.

I don't believe either of these fix the problem of getting all repairs to nodes
other than the coordinator. If node A is coordinating a repair for ranges held
by nodes A, B, and C, B and C don't attempt to repair each other's ranges. If
those ranges were marked as repaired on all nodes then, B and C would never
repair each other.

Maybe the way to fix that is to make first node more of an initiator than a
coordinator. The first node initiates the repair of a given range, and every
node getting that request performs essentially the same repair the coordinator
is doing now. That way all the ranges go between all the involved nodes, and
they can safely mark the ranges repaired when all involved nodes finish. They'd
only need to build the merkle tree once per initiated request, so it shouldn't
be any extra work.

Avoid repairing already-repaired data by default

Key: CASSANDRA-5351
URL: https://issues.apache.org/jira/browse/CASSANDRA-5351
Project: Cassandra
Issue Type: Task
Components: Core
Reporter: Jonathan Ellis
Labels: repair
Fix For: 2.1

Repair has always built its merkle tree from all the data in a columnfamily,
which is guaranteed to work but is inefficient.
We can improve this by remembering which sstables have already been
successfully repaired, and only repairing sstables new since the last repair.
(This automatically makes CASSANDRA-3362 much less of a problem too.)
The tricky part is, compaction will (if not taught otherwise) mix repaired
data together with non-repaired. So we should segregate unrepaired sstables
from the repaired ones.

[jira] [Updated] (CASSANDRA-5555) Allow sstableloader to handle a larger number of files


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jonathan Ellis updated CASSANDRA-:
--

Reviewer: jbellis

 Allow sstableloader to handle a larger number of files
 --

 Key: CASSANDRA-
 URL: https://issues.apache.org/jira/browse/CASSANDRA-
 Project: Cassandra
  Issue Type: Improvement
  Components: Tools
Reporter: Tyler Hobbs
Assignee: Dave Brosius
 Fix For: 1.2.6

 Attachments: -01.txt


 With the default heap size, sstableloader will OOM when there are roughly 25k 
 files in the directory to load.  It's easy to reach this number of files in a 
 single LCS column family.
 By avoiding creating all SSTableReaders up front in SSTableLoader, we should 
 be able to increase the number of files that sstableloader can handle 
 considerably.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (CASSANDRA-5351) Avoid repairing already-repaired data by default