git commit: Remove homegrown and now unused Throttle class
Updated Branches:
refs/heads/trunk c432e6733 - cc83177af
Remove homegrown and now unused Throttle class
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/cc83177a
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/cc83177a
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/cc83177a
Branch: refs/heads/trunk
Commit: cc83177af8a81499d4dfa2ba182427f2c2443359
Parents: c432e67
Author: Sylvain Lebresne sylv...@datastax.com
Authored: Fri Jun 28 08:27:38 2013 +0200
Committer: Sylvain Lebresne sylv...@datastax.com
Committed: Fri Jun 28 08:27:38 2013 +0200
--
.../org/apache/cassandra/utils/Throttle.java| 111 ---
1 file changed, 111 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/cc83177a/src/java/org/apache/cassandra/utils/Throttle.java
--
diff --git a/src/java/org/apache/cassandra/utils/Throttle.java
b/src/java/org/apache/cassandra/utils/Throttle.java
deleted file mode 100644
index d33fab2..000
--- a/src/java/org/apache/cassandra/utils/Throttle.java
+++ /dev/null
@@ -1,111 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * License); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an AS IS BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.utils;
-
-import java.util.concurrent.TimeUnit;
-
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import com.google.common.util.concurrent.Uninterruptibles;
-
-/**
- * Encapsulates the timing/state required to throttle a caller to a target
throughput in
- * bytes per millisecond, when periodically passed an absolute count of bytes.
- */
-public class Throttle
-{
-private static final Logger logger =
LoggerFactory.getLogger(Throttle.class);
-
-private final String name;
-private final ThroughputFunction fun;
-
-// the bytes that had been handled the last time we delayed to throttle,
-// and the time in milliseconds when we last throttled
-private long bytesAtLastDelay;
-private long timeAtLastDelay;
-
-// current target bytes of throughput per millisecond
-private int targetBytesPerMS = -1;
-
-public Throttle(String name, ThroughputFunction fun)
-{
-this.name = name;
-this.fun = fun;
-}
-
-/** @param currentBytes Bytes of throughput since the beginning of the
task. */
-public void throttle(long currentBytes)
-{
-throttleDelta(currentBytes - bytesAtLastDelay);
-}
-
-/** @param bytesDelta Bytes of throughput since the last call to
throttle*() */
-public void throttleDelta(long bytesDelta)
-{
-int newTargetBytesPerMS = fun.targetThroughput();
-if (newTargetBytesPerMS 1)
-// throttling disabled
-return;
-
-if (newTargetBytesPerMS != targetBytesPerMS)
-{
-// restart throttling based on the new target to avoid getting
bogus answers based on comparing
-// the rate under the old throttle, with the desired rate under
the new. (If the new rate is higher
-// than the old, it doesn't much matter, but if the new rate is
lower, it would result in a long
-// sleep to bring the average down. See CASSANDRA-5087.)
-logger.debug({} target throughput now {} bytes/ms., this,
newTargetBytesPerMS);
-targetBytesPerMS = newTargetBytesPerMS;
-bytesAtLastDelay += bytesDelta;
-timeAtLastDelay = System.nanoTime();
-return;
-}
-
-// time passed since last delay
-long msSinceLast = TimeUnit.NANOSECONDS.toMillis(System.nanoTime() -
timeAtLastDelay);
-// the excess bytes in this period
-long excessBytes = bytesDelta - msSinceLast * targetBytesPerMS;
-
-// the time to delay to recap the deficit
-long timeToDelay = excessBytes / Math.max(1, targetBytesPerMS);
-if (timeToDelay 0)
-{
-if (logger.isTraceEnabled())
-logger.trace(String.format(%s actual throughput was %d bytes
in %d ms: throttling for %d ms,
-
[jira] [Resolved] (CASSANDRA-5522) Migrate FileStreamTask to Guava RateLimiter and remove homegrown Throttle
[ https://issues.apache.org/jira/browse/CASSANDRA-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne resolved CASSANDRA-5522. - Resolution: Duplicate This has actually been done as part of CASSANDRA-5286. The Throttle class was still around but it was unused so I just removed it. Migrate FileStreamTask to Guava RateLimiter and remove homegrown Throttle - Key: CASSANDRA-5522 URL: https://issues.apache.org/jira/browse/CASSANDRA-5522 Project: Cassandra Issue Type: Task Components: Core Reporter: Jonathan Ellis Assignee: Tyler Hobbs Priority: Minor Fix For: 2.1 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5544) Hadoop jobs assigns only one mapper in task
[
https://issues.apache.org/jira/browse/CASSANDRA-5544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695303#comment-13695303
]
Shamim Ahmed commented on CASSANDRA-5544:
-
At last , i could manage a few hours to try the fix. Definitely it's working
now, every mapper works on their own range, however i have test in single node
cluster with Hadoop 1.1.2 + Pig 0.11.1 and Cassandra 1.2.6. Thankx.
Hadoop jobs assigns only one mapper in task
Key: CASSANDRA-5544
URL: https://issues.apache.org/jira/browse/CASSANDRA-5544
Project: Cassandra
Issue Type: Bug
Components: Hadoop
Affects Versions: 1.2.1
Environment: Red hat linux 5.4, Hadoop 1.0.3, pig 0.11.1
Reporter: Shamim Ahmed
Assignee: Alex Liu
Fix For: 1.2.6
Attachments: 5544-1.txt, 5544-2.txt, 5544-3.txt, 5544.txt, Screen
Shot 2013-05-26 at 4.49.48 PM.png
We have got very strange beheviour of hadoop cluster after upgrading
Cassandra from 1.1.5 to Cassandra 1.2.1. We have 5 nodes cluster of
Cassandra, where three of them are hodoop slaves. Now when we are submitting
job through Pig script, only one map assigns in task running on one of the
hadoop slaves regardless of
volume of data (already tried with more than million rows).
Configure of pig as follows:
export PIG_HOME=/oracle/pig-0.10.0
export PIG_CONF_DIR=${HADOOP_HOME}/conf
export PIG_INITIAL_ADDRESS=192.168.157.103
export PIG_RPC_PORT=9160
export PIG_PARTITIONER=org.apache.cassandra.dht.Murmur3Partitioner
Also we have these following properties in hadoop:
property
namemapred.tasktracker.map.tasks.maximum/name
value10/value
/property
property
namemapred.map.tasks/name
value4/value
/property
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5706) OOM while loading key cache at startup
[ https://issues.apache.org/jira/browse/CASSANDRA-5706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695310#comment-13695310 ] Fabien Rousseau commented on CASSANDRA-5706: Sorry for the delay. Yep, it looks good to me. OOM while loading key cache at startup -- Key: CASSANDRA-5706 URL: https://issues.apache.org/jira/browse/CASSANDRA-5706 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Fabien Rousseau Assignee: Fabien Rousseau Fix For: 1.2.7 Attachments: 5706-OOM-while-loading-key-cache-at-startup.patch, 5706-v2-txt Steps to be able to reproduce it : - have a heap of 1Gb - have a saved key cache without the SSTables When looking at KeyCacheSerializer.serialize : it always writes a Boolean When looking at KeyCacheSerializer.deserialize : no Boolean is read if SSTable is missing... In case of a promoted index, RowIndexEntry.serializer.skip(...) should be called rather than RowIndexEntry.serializer.skipPromotedIndex(...) (again for symmetry between serialization/deserialization) Attached is a proposed patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5712) Reverse slice queries can skip range tombstones
Sylvain Lebresne created CASSANDRA-5712: --- Summary: Reverse slice queries can skip range tombstones Key: CASSANDRA-5712 URL: https://issues.apache.org/jira/browse/CASSANDRA-5712 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.2.7 On disk, we represent range tombstones by a marker at the beginning of the range covered. Since we repeat such markers when they overlap an index block and since an index block is always read in forward order (even in reverse queries), we are guaranteed to see a range tombstone before any column it covers. However, IndexedSliceReader returns the columns of an index block in reverse order and thus can return a range tombstone *after* columns it covers. It follows that some range tombstone can be skipped during a reversed range slice. We need to fix IndexedSliceReader to always return range tombstone first (or at least before the first column covered by each range tombstone). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5712) Reverse slice queries can skip range tombstones
[ https://issues.apache.org/jira/browse/CASSANDRA-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sylvain Lebresne updated CASSANDRA-5712: Attachment: 5712.txt Attaching patch with a unit test to demonstrate the bug as well as a fix. The fix simply make sure that for an index block we always return all the range tombstone first before returning any column. In theory this means we might sometimes return a few range tombstones the caller won't really need, but it's harmless and probably not worth bothering with. Reverse slice queries can skip range tombstones --- Key: CASSANDRA-5712 URL: https://issues.apache.org/jira/browse/CASSANDRA-5712 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.2.7 Attachments: 5712.txt On disk, we represent range tombstones by a marker at the beginning of the range covered. Since we repeat such markers when they overlap an index block and since an index block is always read in forward order (even in reverse queries), we are guaranteed to see a range tombstone before any column it covers. However, IndexedSliceReader returns the columns of an index block in reverse order and thus can return a range tombstone *after* columns it covers. It follows that some range tombstone can be skipped during a reversed range slice. We need to fix IndexedSliceReader to always return range tombstone first (or at least before the first column covered by each range tombstone). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[Cassandra Wiki] Update of GettingStarted by jeremyhanna
Dear Wiki user, You have subscribed to a wiki page or wiki category on Cassandra Wiki for change notification. The GettingStarted page has been changed by jeremyhanna: https://wiki.apache.org/cassandra/GettingStarted?action=diffrev1=90rev2=91 Comment: Trying to update recommendations for heap and heap new size. #MAX_HEAP_SIZE=4G #HEAP_NEWSIZE=800M }}} - As a rule of thumb, you should set `HEAP_NEWSIZE` to be 1/4 of `MAX_HEAP_SIZE`. If you face OutOfMemory exceptions or massive GCs with this configuration, increase both of these values. + For `MAX_HEAP_SIZE` use as little as you can get away with. It's recommended to stay within 8G because much beyond that, the CMS GC pauses interfere with normal operations. + For `HEAP_NEWSIZE` use the number of cores * 100 but don't exceed 800M. With too much allocated, ParNew GC pauses become detrimental. == Step 3: Start Cassandra ==
[1/3] git commit: Fix loading key cache when a saved entry is no longer valid patch by Fabien Rousseau; reviewed by jbellis for CASSANDRA-5706
Updated Branches:
refs/heads/cassandra-1.2 9837173d8 - 43a98c212
refs/heads/trunk cc83177af - 66fe21647
Fix loading key cache when a saved entry is no longer valid
patch by Fabien Rousseau; reviewed by jbellis for CASSANDRA-5706
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43a98c21
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43a98c21
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43a98c21
Branch: refs/heads/cassandra-1.2
Commit: 43a98c21204648549062f66be09b30dccbfd9a21
Parents: 9837173
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 28 08:28:09 2013 -0700
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 28 08:28:09 2013 -0700
--
CHANGES.txt | 14 ++
.../org/apache/cassandra/service/CacheService.java| 12 ++--
2 files changed, 16 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43a98c21/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ee21526..1b9634c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,11 +1,15 @@
1.2.7
+ * Fix loading key cache when a saved entry is no longer valid (CASSANDRA-5706)
* Fix serialization of the LEFT gossip value (CASSANDRA-5696)
* Pig: support for cql3 tables (CASSANDRA-5234)
* cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
- * Race condition in detecting version on a mixed 1.1/1.2 cluster
(CASSANDRA-5692)
+ * Race condition in detecting version on a mixed 1.1/1.2 cluster
+ (CASSANDRA-5692)
+
1.2.6
- * Fix tracing when operation completes before all responses arrive
(CASSANDRA-5668)
+ * Fix tracing when operation completes before all responses arrive
+ (CASSANDRA-5668)
* Fix cross-DC mutation forwarding (CASSANDRA-5632)
* Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
* (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622)
@@ -31,14 +35,16 @@
* Correct blob literal + ReversedType parsing (CASSANDRA-5629)
* Allow GPFS to prefer the internal IP like EC2MRS (CASSANDRA-5630)
* fix help text for -tspw cassandra-cli (CASSANDRA-5643)
- * don't throw away initial causes exceptions for internode encryption issues
(CASSANDRA-5644)
+ * don't throw away initial causes exceptions for internode encryption issues
+ (CASSANDRA-5644)
* Fix message spelling errors for cql select statements (CASSANDRA-5647)
* Suppress custom exceptions thru jmx (CASSANDRA-5652)
* Update CREATE CUSTOM INDEX syntax (CASSANDRA-5639)
* Fix PermissionDetails.equals() method (CASSANDRA-5655)
* Never allow partition key ranges in CQL3 without token() (CASSANDRA-5666)
* Gossiper incorrectly drops AppState for an upgrading node (CASSANDRA-5660)
- * Connection thrashing during multi-region ec2 during upgrade, due to
messaging version (CASSANDRA-5669)
+ * Connection thrashing during multi-region ec2 during upgrade, due to
+ messaging version (CASSANDRA-5669)
* Avoid over reconnecting in EC2MRS (CASSANDRA-5678)
* Fix ReadResponseSerializer.serializedSize() for digest reads
(CASSANDRA-5476)
* allow sstable2json on 2i CFs (CASSANDRA-5694)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43a98c21/src/java/org/apache/cassandra/service/CacheService.java
--
diff --git a/src/java/org/apache/cassandra/service/CacheService.java
b/src/java/org/apache/cassandra/service/CacheService.java
index bc218c4..f77110c 100644
--- a/src/java/org/apache/cassandra/service/CacheService.java
+++ b/src/java/org/apache/cassandra/service/CacheService.java
@@ -379,16 +379,16 @@ public class CacheService implements CacheServiceMBean
ByteBuffer key = ByteBufferUtil.readWithLength(input);
int generation = input.readInt();
SSTableReader reader = findDesc(generation, cfs.getSSTables());
+boolean promotedIndexes = input.readBoolean();
if (reader == null)
{
-RowIndexEntry.serializer.skipPromotedIndex(input);
+if (promotedIndexes)
+RowIndexEntry.serializer.skip(input,
Descriptor.Version.CURRENT);
return null;
}
-RowIndexEntry entry;
-if (input.readBoolean())
-entry = RowIndexEntry.serializer.deserialize(input,
reader.descriptor.version);
-else
-entry =
reader.getPosition(reader.partitioner.decorateKey(key), Operator.EQ);
+RowIndexEntry entry = promotedIndexes
+? RowIndexEntry.serializer.deserialize(input,
reader.descriptor.version)
+
[2/3] git commit: Fix loading key cache when a saved entry is no longer valid patch by Fabien Rousseau; reviewed by jbellis for CASSANDRA-5706
Fix loading key cache when a saved entry is no longer valid
patch by Fabien Rousseau; reviewed by jbellis for CASSANDRA-5706
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/43a98c21
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/43a98c21
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/43a98c21
Branch: refs/heads/trunk
Commit: 43a98c21204648549062f66be09b30dccbfd9a21
Parents: 9837173
Author: Jonathan Ellis jbel...@apache.org
Authored: Fri Jun 28 08:28:09 2013 -0700
Committer: Jonathan Ellis jbel...@apache.org
Committed: Fri Jun 28 08:28:09 2013 -0700
--
CHANGES.txt | 14 ++
.../org/apache/cassandra/service/CacheService.java| 12 ++--
2 files changed, 16 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43a98c21/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index ee21526..1b9634c 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,11 +1,15 @@
1.2.7
+ * Fix loading key cache when a saved entry is no longer valid (CASSANDRA-5706)
* Fix serialization of the LEFT gossip value (CASSANDRA-5696)
* Pig: support for cql3 tables (CASSANDRA-5234)
* cqlsh: Don't show 'null' in place of empty values (CASSANDRA-5675)
- * Race condition in detecting version on a mixed 1.1/1.2 cluster
(CASSANDRA-5692)
+ * Race condition in detecting version on a mixed 1.1/1.2 cluster
+ (CASSANDRA-5692)
+
1.2.6
- * Fix tracing when operation completes before all responses arrive
(CASSANDRA-5668)
+ * Fix tracing when operation completes before all responses arrive
+ (CASSANDRA-5668)
* Fix cross-DC mutation forwarding (CASSANDRA-5632)
* Scale hinted_handoff_throttle_in_kb to cluster size (CASSANDRA-5272)
* (Hadoop) Add CQL3 input/output formats (CASSANDRA-4421, 5622)
@@ -31,14 +35,16 @@
* Correct blob literal + ReversedType parsing (CASSANDRA-5629)
* Allow GPFS to prefer the internal IP like EC2MRS (CASSANDRA-5630)
* fix help text for -tspw cassandra-cli (CASSANDRA-5643)
- * don't throw away initial causes exceptions for internode encryption issues
(CASSANDRA-5644)
+ * don't throw away initial causes exceptions for internode encryption issues
+ (CASSANDRA-5644)
* Fix message spelling errors for cql select statements (CASSANDRA-5647)
* Suppress custom exceptions thru jmx (CASSANDRA-5652)
* Update CREATE CUSTOM INDEX syntax (CASSANDRA-5639)
* Fix PermissionDetails.equals() method (CASSANDRA-5655)
* Never allow partition key ranges in CQL3 without token() (CASSANDRA-5666)
* Gossiper incorrectly drops AppState for an upgrading node (CASSANDRA-5660)
- * Connection thrashing during multi-region ec2 during upgrade, due to
messaging version (CASSANDRA-5669)
+ * Connection thrashing during multi-region ec2 during upgrade, due to
+ messaging version (CASSANDRA-5669)
* Avoid over reconnecting in EC2MRS (CASSANDRA-5678)
* Fix ReadResponseSerializer.serializedSize() for digest reads
(CASSANDRA-5476)
* allow sstable2json on 2i CFs (CASSANDRA-5694)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/43a98c21/src/java/org/apache/cassandra/service/CacheService.java
--
diff --git a/src/java/org/apache/cassandra/service/CacheService.java
b/src/java/org/apache/cassandra/service/CacheService.java
index bc218c4..f77110c 100644
--- a/src/java/org/apache/cassandra/service/CacheService.java
+++ b/src/java/org/apache/cassandra/service/CacheService.java
@@ -379,16 +379,16 @@ public class CacheService implements CacheServiceMBean
ByteBuffer key = ByteBufferUtil.readWithLength(input);
int generation = input.readInt();
SSTableReader reader = findDesc(generation, cfs.getSSTables());
+boolean promotedIndexes = input.readBoolean();
if (reader == null)
{
-RowIndexEntry.serializer.skipPromotedIndex(input);
+if (promotedIndexes)
+RowIndexEntry.serializer.skip(input,
Descriptor.Version.CURRENT);
return null;
}
-RowIndexEntry entry;
-if (input.readBoolean())
-entry = RowIndexEntry.serializer.deserialize(input,
reader.descriptor.version);
-else
-entry =
reader.getPosition(reader.partitioner.decorateKey(key), Operator.EQ);
+RowIndexEntry entry = promotedIndexes
+? RowIndexEntry.serializer.deserialize(input,
reader.descriptor.version)
+:
reader.getPosition(reader.partitioner.decorateKey(key), Operator.EQ);
return
[3/3] git commit: merge from 1.2
merge from 1.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66fe2164 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66fe2164 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66fe2164 Branch: refs/heads/trunk Commit: 66fe21647e6e8c9e5485632f43f011e4c4faee00 Parents: cc83177 43a98c2 Author: Jonathan Ellis jbel...@apache.org Authored: Fri Jun 28 08:31:01 2013 -0700 Committer: Jonathan Ellis jbel...@apache.org Committed: Fri Jun 28 08:31:01 2013 -0700 -- CHANGES.txt | 14 ++ .../org/apache/cassandra/service/CacheService.java| 5 ++--- 2 files changed, 12 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/66fe2164/CHANGES.txt -- diff --cc CHANGES.txt index 7565017,1b9634c..87aa99d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,81 -1,15 +1,85 @@@ +2.0 + * Removed on-heap row cache (CASSANDRA-5348) + * use nanotime consistently for node-local timeouts (CASSANDRA-5581) + * Avoid unnecessary second pass on name-based queries (CASSANDRA-5577) + * Experimental triggers (CASSANDRA-1311) + * JEMalloc support for off-heap allocation (CASSANDRA-3997) + * Single-pass compaction (CASSANDRA-4180) + * Removed token range bisection (CASSANDRA-5518) + * Removed compatibility with pre-1.2.5 sstables and network messages + (CASSANDRA-5511) + * removed PBSPredictor (CASSANDRA-5455) + * CAS support (CASSANDRA-5062, 5441, 5442, 5443, 5619) + * Leveled compaction performs size-tiered compactions in L0 + (CASSANDRA-5371, 5439) + * Add yaml network topology snitch for mixed ec2/other envs (CASSANDRA-5339) + * Log when a node is down longer than the hint window (CASSANDRA-4554) + * Optimize tombstone creation for ExpiringColumns (CASSANDRA-4917) + * Improve LeveledScanner work estimation (CASSANDRA-5250, 5407) + * Replace compaction lock with runWithCompactionsDisabled (CASSANDRA-3430) + * Change Message IDs to ints (CASSANDRA-5307) + * Move sstable level information into the Stats component, removing the + need for a separate Manifest file (CASSANDRA-4872) + * avoid serializing to byte[] on commitlog append (CASSANDRA-5199) + * make index_interval configurable per columnfamily (CASSANDRA-3961, CASSANDRA-5650) + * add default_time_to_live (CASSANDRA-3974) + * add memtable_flush_period_in_ms (CASSANDRA-4237) + * replace supercolumns internally by composites (CASSANDRA-3237, 5123) + * upgrade thrift to 0.9.0 (CASSANDRA-3719) + * drop unnecessary keyspace parameter from user-defined compaction API + (CASSANDRA-5139) + * more robust solution to incomplete compactions + counters (CASSANDRA-5151) + * Change order of directory searching for c*.in.sh (CASSANDRA-3983) + * Add tool to reset SSTable compaction level for LCS (CASSANDRA-5271) + * Allow custom configuration loader (CASSANDRA-5045) + * Remove memory emergency pressure valve logic (CASSANDRA-3534) + * Reduce request latency with eager retry (CASSANDRA-4705) + * cqlsh: Remove ASSUME command (CASSANDRA-5331) + * Rebuild BF when loading sstables if bloom_filter_fp_chance + has changed since compaction (CASSANDRA-5015) + * remove row-level bloom filters (CASSANDRA-4885) + * Change Kernel Page Cache skipping into row preheating (disabled by default) + (CASSANDRA-4937) + * Improve repair by deciding on a gcBefore before sending + out TreeRequests (CASSANDRA-4932) + * Add an official way to disable compactions (CASSANDRA-5074) + * Reenable ALTER TABLE DROP with new semantics (CASSANDRA-3919) + * Add binary protocol versioning (CASSANDRA-5436) + * Swap THshaServer for TThreadedSelectorServer (CASSANDRA-5530) + * Add alias support to SELECT statement (CASSANDRA-5075) + * Don't create empty RowMutations in CommitLogReplayer (CASSANDRA-5541) + * Use range tombstones when dropping cfs/columns from schema (CASSANDRA-5579) + * cqlsh: drop CQL2/CQL3-beta support (CASSANDRA-5585) + * Track max/min column names in sstables to be able to optimize slice + queries (CASSANDRA-5514, CASSANDRA-5595, CASSANDRA-5600) + * Binary protocol: allow batching already prepared statements (CASSANDRA-4693) + * Allow preparing timestamp, ttl and limit in CQL3 queries (CASSANDRA-4450) + * Support native link w/o JNA in Java7 (CASSANDRA-3734) + * Use SASL authentication in binary protocol v2 (CASSANDRA-5545) + * Replace Thrift HsHa with LMAX Disruptor based implementation (CASSANDRA-5582) + * cqlsh: Add row count to SELECT output (CASSANDRA-5636) + * Include a timestamp with all read commands to determine column expiration + (CASSANDRA-5149) + * Streaming 2.0 (CASSANDRA-5286) + * Conditional create/drop ks/table/index statements in
[jira] [Commented] (CASSANDRA-5712) Reverse slice queries can skip range tombstones
[ https://issues.apache.org/jira/browse/CASSANDRA-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695523#comment-13695523 ] Jonathan Ellis commented on CASSANDRA-5712: --- Do we really need rangeTombstonesReversed? ISTM that we could just special case tombstones in addColumn (rename to addAtom) to always addFirst regardless of reversed-ness. Reverse slice queries can skip range tombstones --- Key: CASSANDRA-5712 URL: https://issues.apache.org/jira/browse/CASSANDRA-5712 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.2.7 Attachments: 5712.txt On disk, we represent range tombstones by a marker at the beginning of the range covered. Since we repeat such markers when they overlap an index block and since an index block is always read in forward order (even in reverse queries), we are guaranteed to see a range tombstone before any column it covers. However, IndexedSliceReader returns the columns of an index block in reverse order and thus can return a range tombstone *after* columns it covers. It follows that some range tombstone can be skipped during a reversed range slice. We need to fix IndexedSliceReader to always return range tombstone first (or at least before the first column covered by each range tombstone). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5712) Reverse slice queries can skip range tombstones
[ https://issues.apache.org/jira/browse/CASSANDRA-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695549#comment-13695549 ] Sylvain Lebresne commented on CASSANDRA-5712: - bq. ISTM that we could just special case tombstones in addColumn (rename to addAtom) to always addFirst regardless of reversed-ness. I don't think that works :). We get t(0), c(1), c(2), t(3), c(4) from the reader and we want to ultimately return them as t(3), t(0), c(4), c(2), c(1). If we do addFirst for columns and addLast for tombstone, we'll get c(4), c(2), c(1), t(0), t(3), which gives us all tombstone last (and if we pollLast instead of pollFirst, the columns won't be reversed). If we do addLast for columns and addFirst for tombstone, we have the exact equivalent of pollLast in the previous case. We can probably do some peekLast first and do pollLast as long as it's a tombstone and pollFirst if it's not, but is that really much of an improvement compared to the patch? Reverse slice queries can skip range tombstones --- Key: CASSANDRA-5712 URL: https://issues.apache.org/jira/browse/CASSANDRA-5712 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.2.7 Attachments: 5712.txt On disk, we represent range tombstones by a marker at the beginning of the range covered. Since we repeat such markers when they overlap an index block and since an index block is always read in forward order (even in reverse queries), we are guaranteed to see a range tombstone before any column it covers. However, IndexedSliceReader returns the columns of an index block in reverse order and thus can return a range tombstone *after* columns it covers. It follows that some range tombstone can be skipped during a reversed range slice. We need to fix IndexedSliceReader to always return range tombstone first (or at least before the first column covered by each range tombstone). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[3/3] git commit: Merge branch 'cassandra-1.2' into trunk
Merge branch 'cassandra-1.2' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03ff2687 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03ff2687 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03ff2687 Branch: refs/heads/trunk Commit: 03ff268748080cff3c871d882b0194c9c99bce81 Parents: 66fe216 d265fde Author: Brandon Williams brandonwilli...@apache.org Authored: Fri Jun 28 14:04:31 2013 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Fri Jun 28 14:04:31 2013 -0500 -- src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/03ff2687/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml --
[1/3] git commit: spelling
Updated Branches: refs/heads/cassandra-1.2 43a98c212 - d265fded8 refs/heads/trunk 66fe21647 - 03ff26874 spelling Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d265fded Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d265fded Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d265fded Branch: refs/heads/cassandra-1.2 Commit: d265fded8572d23c2709e137ef1bbe0d0b40acd8 Parents: 43a98c2 Author: Brandon Williams brandonwilli...@apache.org Authored: Fri Jun 28 14:04:22 2013 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Fri Jun 28 14:04:22 2013 -0500 -- src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d265fded/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml -- diff --git a/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml b/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml index 568aa20..6e366f4 100644 --- a/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml +++ b/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml @@ -108,7 +108,7 @@ commands: Print network information on provided host (connecting node by default) - name: move new token help: | - Move node on the token ring to a new token. (for -ve tokens, use \\ to escape, Example: move \\-123) + Move node on the token ring to a new token. (for negative tokens, use \\ to escape, Example: move \\-123) - name: removenode status|force|ID help: | Show status of current node removal, force completion of pending removal or remove provided ID
[2/3] git commit: spelling
spelling Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d265fded Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d265fded Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d265fded Branch: refs/heads/trunk Commit: d265fded8572d23c2709e137ef1bbe0d0b40acd8 Parents: 43a98c2 Author: Brandon Williams brandonwilli...@apache.org Authored: Fri Jun 28 14:04:22 2013 -0500 Committer: Brandon Williams brandonwilli...@apache.org Committed: Fri Jun 28 14:04:22 2013 -0500 -- src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d265fded/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml -- diff --git a/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml b/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml index 568aa20..6e366f4 100644 --- a/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml +++ b/src/resources/org/apache/cassandra/tools/NodeToolHelp.yaml @@ -108,7 +108,7 @@ commands: Print network information on provided host (connecting node by default) - name: move new token help: | - Move node on the token ring to a new token. (for -ve tokens, use \\ to escape, Example: move \\-123) + Move node on the token ring to a new token. (for negative tokens, use \\ to escape, Example: move \\-123) - name: removenode status|force|ID help: | Show status of current node removal, force completion of pending removal or remove provided ID
[jira] [Commented] (CASSANDRA-5681) Refactor IESCS in Snitches
[ https://issues.apache.org/jira/browse/CASSANDRA-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695778#comment-13695778 ] Jason Brown commented on CASSANDRA-5681: Hmm, maybe I need a copy of your intellij Code Style settings :) Refactor IESCS in Snitches -- Key: CASSANDRA-5681 URL: https://issues.apache.org/jira/browse/CASSANDRA-5681 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: snitch Fix For: 1.2.7 Attachments: 5681-v1.diff, 5681-v2_YFNTS.diff Reduce/refactor duplicated IESCS implementations in Ec2MRS, GPFS, and YPNTS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5151) Implement better way of eliminating compaction left overs.
[
https://issues.apache.org/jira/browse/CASSANDRA-5151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695794#comment-13695794
]
Michael Kjellman commented on CASSANDRA-5151:
-
[~slebresne] man this was a while back but if I remember it was fairly easy to
reproduce.
Implement better way of eliminating compaction left overs.
--
Key: CASSANDRA-5151
URL: https://issues.apache.org/jira/browse/CASSANDRA-5151
Project: Cassandra
Issue Type: Bug
Affects Versions: 1.1.3
Reporter: Yuki Morishita
Assignee: Yuki Morishita
Fix For: 2.0 beta 1
Attachments:
0001-move-scheduling-MeteredFlusher-to-CassandraDaemon.patch, 5151-1.2.txt,
5151-v2.txt
This is from discussion in CASSANDRA-5137. Currently we skip loading SSTables
that are left over from incomplete compaction to not over-count counter, but
the way we track compaction completion is not secure.
One possible solution is to create system CF like:
{code}
create table compaction_log (
id uuid primary key,
inputs setint,
outputs setint
);
{code}
to track incomplete compaction.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: Refactor IESCS in Snitches, (part 2 - rework YFNTS) patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
Updated Branches:
refs/heads/trunk 03ff26874 - ff339c58c
Refactor IESCS in Snitches, (part 2 - rework YFNTS)
patch by jasobrown; reviewed by jbellis for CASSANDRA-5681
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff339c58
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff339c58
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff339c58
Branch: refs/heads/trunk
Commit: ff339c58cc6726c676039e8a7ed4c2f6a0ea4f14
Parents: 03ff268
Author: Jason Brown jasedbr...@gmail.com
Authored: Fri Jun 28 14:04:27 2013 -0700
Committer: Jason Brown jasedbr...@gmail.com
Committed: Fri Jun 28 14:04:27 2013 -0700
--
.../locator/YamlFileNetworkTopologySnitch.java | 207 +++
.../YamlFileNetworkTopologySnitchTest.java | 17 --
2 files changed, 23 insertions(+), 201 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff339c58/src/java/org/apache/cassandra/locator/YamlFileNetworkTopologySnitch.java
--
diff --git
a/src/java/org/apache/cassandra/locator/YamlFileNetworkTopologySnitch.java
b/src/java/org/apache/cassandra/locator/YamlFileNetworkTopologySnitch.java
index eb3fd3e..f299b61 100644
--- a/src/java/org/apache/cassandra/locator/YamlFileNetworkTopologySnitch.java
+++ b/src/java/org/apache/cassandra/locator/YamlFileNetworkTopologySnitch.java
@@ -28,10 +28,8 @@ import
org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.gms.ApplicationState;
import org.apache.cassandra.gms.EndpointState;
import org.apache.cassandra.gms.Gossiper;
-import org.apache.cassandra.gms.IEndpointStateChangeSubscriber;
import org.apache.cassandra.gms.VersionedValue;
import org.apache.cassandra.io.util.FileUtils;
-import org.apache.cassandra.net.MessagingService;
import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
@@ -57,9 +55,7 @@ public class YamlFileNetworkTopologySnitch
extends AbstractNetworkTopologySnitch
{
-/** Logger. */
-private static final Logger logger = LoggerFactory
-.getLogger(YamlFileNetworkTopologySnitch.class);
+private static final Logger logger =
LoggerFactory.getLogger(YamlFileNetworkTopologySnitch.class);
/**
* How often to check the topology configuration file, in milliseconds;
defaults to one minute.
@@ -163,89 +159,6 @@ public class YamlFileNetworkTopologySnitch
}
/**
- * Returns the preferred non-broadcast address for the endpoint, or null
if none was specified.
- * p
- * Currently, the only preferred address that is considered is the
data-center-local address.
- * /p
- *
- * @param endpoint
- *the broadcast address for the endpoint
- * @return the preferred non-broadcast address for the endpoint, or null
if none was specified
- */
-private InetAddress getPreferredAddress(final InetAddress endpoint)
-{
-return getDcLocalAddress(endpoint);
-}
-
-/**
- * Returns the data-center-local address for the endpoint, or null if none
was specified.
- *
- * @param endpoint
- *the broadcast address for the endpoint
- * @return the data-center-local address for the endpoint, or null if none
was specified
- */
-private InetAddress getDcLocalAddress(final InetAddress endpoint)
-{
-final NodeData nodeData = nodeDataMap.get(endpoint);
-return nodeData != null ? nodeData.dcLocalAddress : null;
-}
-
-/**
- * Reconnects to the endpoint through the preferred non-broadcast address
if necessary.
- * p
- * A reconnect is performed if
- * ul
- * lithe endpoint is not in the local data center and
- * li
- * lithe endpoint has a configured preferred address as determined by
{@link #getPreferredAddress(InetAddress)}.
- * /ul
- * /p
- *
- * @param endpoint
- *the endpoint's broadcast address
- */
-private void reconnectViaPreferredAddress(final InetAddress endpoint)
-{
-if (!localNodeData.datacenter.equals(getDatacenter(endpoint)))
-{
-return;
-}
-
-reconnectViaPreferredAddress(endpoint, getPreferredAddress(endpoint));
-}
-
-/**
- * Reconnects to the endpoint through the preferred non-broadcast address
if necessary.
- * p
- * A reconnect is performed to {@code preferredAddress} if the {@code
preferredAddress} argument is not null.
- * /p
- * p
- * This method is only meant to be called by {@link
#reconnectViaPreferredAddress(InetAddress)}, but is declared to
- * have package-private scope in order to facilitate
[jira] [Commented] (CASSANDRA-5681) Refactor IESCS in Snitches
[ https://issues.apache.org/jira/browse/CASSANDRA-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695797#comment-13695797 ] Jason Brown commented on CASSANDRA-5681: Committed the YFNTS changes to trunk. Refactor IESCS in Snitches -- Key: CASSANDRA-5681 URL: https://issues.apache.org/jira/browse/CASSANDRA-5681 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: snitch Fix For: 1.2.7 Attachments: 5681-v1.diff, 5681-v2_YFNTS.diff Reduce/refactor duplicated IESCS implementations in Ec2MRS, GPFS, and YPNTS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-5681) Refactor IESCS in Snitches
[ https://issues.apache.org/jira/browse/CASSANDRA-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-5681: --- Fix Version/s: (was: 1.2.7) 1.2.6 Refactor IESCS in Snitches -- Key: CASSANDRA-5681 URL: https://issues.apache.org/jira/browse/CASSANDRA-5681 Project: Cassandra Issue Type: Improvement Components: Core Affects Versions: 1.2.5 Reporter: Jason Brown Assignee: Jason Brown Priority: Minor Labels: snitch Fix For: 1.2.6, 2.0 beta 1 Attachments: 5681-v1.diff, 5681-v2_YFNTS.diff Reduce/refactor duplicated IESCS implementations in Ec2MRS, GPFS, and YPNTS. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3772) Evaluate Murmur3-based partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695804#comment-13695804 ] Lex Lythius commented on CASSANDRA-3772: Sorry to bring a skeleton back with bad news from the day after it was buried. Apparently, finding collisions for Murmur3 hash would be easy enough to attempt choking Cassandra. http://emboss.github.io/blog/2012/12/14/breaking-murmur-hash-flooding-dos-reloaded/ Just wanted to make sure you are aware of this. Evaluate Murmur3-based partitioner -- Key: CASSANDRA-3772 URL: https://issues.apache.org/jira/browse/CASSANDRA-3772 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 1.2.0 beta 1 Attachments: 0001-CASSANDRA-3772.patch, 0001-CASSANDRA-3772-Test.patch, CASSANDRA-3772-v2.patch, CASSANDRA-3772-v3.patch, CASSANDRA-3772-v4.patch, hashed_partitioner_3.diff, hashed_partitioner.diff, MumPartitionerTest.docx, try_murmur3_2.diff, try_murmur3.diff MD5 is a relatively heavyweight hash to use when we don't need cryptographic qualities, just a good output distribution. Let's see how much overhead we can save by using Murmur3 instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5713) CQL3 token() = token() does not work as expected
Colin B. created CASSANDRA-5713:
---
Summary: CQL3 token() = token() does not work as expected
Key: CASSANDRA-5713
URL: https://issues.apache.org/jira/browse/CASSANDRA-5713
Project: Cassandra
Issue Type: Bug
Reporter: Colin B.
Priority: Minor
Using tokens to go backward through a table in CQL3 does not work as expected.
Say there is some data available:
{code}
SELECT key FROM data_list where token(key) = token('6') limit 10;
key
-
6
7
9
4
3
A
5
8
2
1
{code}
I expect:
{code}
SELECT key FROM data_list where token(key) = token('1') limit 5;
key
-
A
5
8
2
1
{code}
However the following occurs:
{code}
SELECT key FROM data_list where token(key) = token('1') limit 5;
key
-
6
7
9
4
3
{code}
The '' operator has similar unexpected behavior.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3772) Evaluate Murmur3-based partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695845#comment-13695845 ] Brandon Williams commented on CASSANDRA-3772: - It doesn't really matter if you can create M3 collisions, we only use it to find the node to store the data, the key sent and stored is still the actual key, not the hash. If you knew a token in the cluster the worst you could do is cause some (probably imperceptible) imbalance. Evaluate Murmur3-based partitioner -- Key: CASSANDRA-3772 URL: https://issues.apache.org/jira/browse/CASSANDRA-3772 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 1.2.0 beta 1 Attachments: 0001-CASSANDRA-3772.patch, 0001-CASSANDRA-3772-Test.patch, CASSANDRA-3772-v2.patch, CASSANDRA-3772-v3.patch, CASSANDRA-3772-v4.patch, hashed_partitioner_3.diff, hashed_partitioner.diff, MumPartitionerTest.docx, try_murmur3_2.diff, try_murmur3.diff MD5 is a relatively heavyweight hash to use when we don't need cryptographic qualities, just a good output distribution. Let's see how much overhead we can save by using Murmur3 instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3772) Evaluate Murmur3-based partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695905#comment-13695905 ] Lex Lythius commented on CASSANDRA-3772: @Brandon Yes, I understand that we're talking about map bucket (hence node) collisions, not row collisions here. Still I wanted to bring this to your attention, if nothing else to disregard it as a non-issue. Would feeding masses of data to a particular bucket actually be imperceptible? I figured that could saturate a node, cause row/key cache issues, etc. Evaluate Murmur3-based partitioner -- Key: CASSANDRA-3772 URL: https://issues.apache.org/jira/browse/CASSANDRA-3772 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 1.2.0 beta 1 Attachments: 0001-CASSANDRA-3772.patch, 0001-CASSANDRA-3772-Test.patch, CASSANDRA-3772-v2.patch, CASSANDRA-3772-v3.patch, CASSANDRA-3772-v4.patch, hashed_partitioner_3.diff, hashed_partitioner.diff, MumPartitionerTest.docx, try_murmur3_2.diff, try_murmur3.diff MD5 is a relatively heavyweight hash to use when we don't need cryptographic qualities, just a good output distribution. Let's see how much overhead we can save by using Murmur3 instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3772) Evaluate Murmur3-based partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695907#comment-13695907 ] Jonathan Ellis commented on CASSANDRA-3772: --- If you're exposing Cassandra directly to attackers there are easier ways to DOS it. Evaluate Murmur3-based partitioner -- Key: CASSANDRA-3772 URL: https://issues.apache.org/jira/browse/CASSANDRA-3772 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 1.2.0 beta 1 Attachments: 0001-CASSANDRA-3772.patch, 0001-CASSANDRA-3772-Test.patch, CASSANDRA-3772-v2.patch, CASSANDRA-3772-v3.patch, CASSANDRA-3772-v4.patch, hashed_partitioner_3.diff, hashed_partitioner.diff, MumPartitionerTest.docx, try_murmur3_2.diff, try_murmur3.diff MD5 is a relatively heavyweight hash to use when we don't need cryptographic qualities, just a good output distribution. Let's see how much overhead we can save by using Murmur3 instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3772) Evaluate Murmur3-based partitioner
[ https://issues.apache.org/jira/browse/CASSANDRA-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695919#comment-13695919 ] Lex Lythius commented on CASSANDRA-3772: Right. I was thinking some web scenarios using tables with non-UUID keys and not much screening of input. Anyway... Evaluate Murmur3-based partitioner -- Key: CASSANDRA-3772 URL: https://issues.apache.org/jira/browse/CASSANDRA-3772 Project: Cassandra Issue Type: New Feature Components: Core Reporter: Jonathan Ellis Assignee: Pavel Yaskevich Fix For: 1.2.0 beta 1 Attachments: 0001-CASSANDRA-3772.patch, 0001-CASSANDRA-3772-Test.patch, CASSANDRA-3772-v2.patch, CASSANDRA-3772-v3.patch, CASSANDRA-3772-v4.patch, hashed_partitioner_3.diff, hashed_partitioner.diff, MumPartitionerTest.docx, try_murmur3_2.diff, try_murmur3.diff MD5 is a relatively heavyweight hash to use when we don't need cryptographic qualities, just a good output distribution. Let's see how much overhead we can save by using Murmur3 instead. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
git commit: fix up logging contexts to match class
Updated Branches:
refs/heads/trunk ff339c58c - 66f301457
fix up logging contexts to match class
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/66f30145
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/66f30145
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/66f30145
Branch: refs/heads/trunk
Commit: 66f301457f90a49ffbd3cc4c0964f36c86871ff5
Parents: ff339c5
Author: Dave Brosius dbros...@apache.org
Authored: Fri Jun 28 19:49:16 2013 -0400
Committer: Dave Brosius dbros...@apache.org
Committed: Fri Jun 28 19:49:16 2013 -0400
--
src/java/org/apache/cassandra/repair/Differencer.java | 4 +---
src/java/org/apache/cassandra/repair/NodePair.java | 1 -
src/java/org/apache/cassandra/repair/RepairJob.java | 3 +--
src/java/org/apache/cassandra/repair/RepairSession.java | 2 +-
src/java/org/apache/cassandra/repair/StreamingRepairTask.java | 4
src/java/org/apache/cassandra/repair/Validator.java | 5 +
.../org/apache/cassandra/repair/messages/RepairMessage.java | 1 -
src/java/org/apache/cassandra/repair/messages/SyncRequest.java | 2 --
.../org/apache/cassandra/repair/messages/ValidationRequest.java | 1 -
9 files changed, 4 insertions(+), 19 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f30145/src/java/org/apache/cassandra/repair/Differencer.java
--
diff --git a/src/java/org/apache/cassandra/repair/Differencer.java
b/src/java/org/apache/cassandra/repair/Differencer.java
index 82b331b..214d2c9 100644
--- a/src/java/org/apache/cassandra/repair/Differencer.java
+++ b/src/java/org/apache/cassandra/repair/Differencer.java
@@ -30,8 +30,6 @@ import org.apache.cassandra.dht.Token;
import org.apache.cassandra.net.MessagingService;
import org.apache.cassandra.repair.messages.SyncComplete;
import org.apache.cassandra.repair.messages.SyncRequest;
-import org.apache.cassandra.service.ActiveRepairService;
-import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.MerkleTree;
@@ -40,7 +38,7 @@ import org.apache.cassandra.utils.MerkleTree;
*/
public class Differencer implements Runnable
{
-private static Logger logger =
LoggerFactory.getLogger(ActiveRepairService.class);
+private static Logger logger = LoggerFactory.getLogger(Differencer.class);
private final RepairJobDesc desc;
public final TreeResponse r1;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f30145/src/java/org/apache/cassandra/repair/NodePair.java
--
diff --git a/src/java/org/apache/cassandra/repair/NodePair.java
b/src/java/org/apache/cassandra/repair/NodePair.java
index f510dc4..fb7a72a 100644
--- a/src/java/org/apache/cassandra/repair/NodePair.java
+++ b/src/java/org/apache/cassandra/repair/NodePair.java
@@ -23,7 +23,6 @@ import java.io.IOException;
import java.net.InetAddress;
import com.google.common.base.Objects;
-import com.google.common.collect.Sets;
import org.apache.cassandra.io.IVersionedSerializer;
import org.apache.cassandra.net.CompactEndpointSerializationHelper;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f30145/src/java/org/apache/cassandra/repair/RepairJob.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairJob.java
b/src/java/org/apache/cassandra/repair/RepairJob.java
index 277075c..16daf4e 100644
--- a/src/java/org/apache/cassandra/repair/RepairJob.java
+++ b/src/java/org/apache/cassandra/repair/RepairJob.java
@@ -38,7 +38,6 @@ import org.apache.cassandra.net.IAsyncCallback;
import org.apache.cassandra.net.MessageIn;
import org.apache.cassandra.net.MessagingService;
import org.apache.cassandra.repair.messages.ValidationRequest;
-import org.apache.cassandra.service.ActiveRepairService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.MerkleTree;
import org.apache.cassandra.utils.SimpleCondition;
@@ -48,7 +47,7 @@ import org.apache.cassandra.utils.SimpleCondition;
*/
public class RepairJob
{
-private static Logger logger =
LoggerFactory.getLogger(ActiveRepairService.class);
+private static Logger logger = LoggerFactory.getLogger(RepairJob.class);
public final RepairJobDesc desc;
private final boolean isSequential;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/66f30145/src/java/org/apache/cassandra/repair/RepairSession.java
--
diff --git a/src/java/org/apache/cassandra/repair/RepairSession.java
[jira] [Updated] (CASSANDRA-5524) Allow upgradesstables to be run against a specified directory
[ https://issues.apache.org/jira/browse/CASSANDRA-5524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Zarutin updated CASSANDRA-5524: Attachment: CASSANDRA-5524.txt Issue has verified, on both failure to load sstables w/o upgrade using this tool, and upgrade and load upgraded sstables. Details and steps to repro can we foundin attached CASSANDRA-5524.txt that was reviewed by Nick Bailey. Allow upgradesstables to be run against a specified directory - Key: CASSANDRA-5524 URL: https://issues.apache.org/jira/browse/CASSANDRA-5524 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Nick Bailey Priority: Minor Fix For: 1.2.6 Attachments: 0001-Add-a-snapshot-upgrade-tool.patch, 0002-Rename-snapshotupgrade-to-sstableupgrade.patch, 0003-Update-NEWS.txt-and-debian-scripts.patch, CASSANDRA-5524.txt Currently, upgradesstables only modifies live SSTables. Because sstableloader cannot stream old SSTable formats, this makes it difficult to restore data from a snapshot taken in a previous major version of Cassandra. Allowing the user to specify a directory for upgradesstables would resolve this, but it may also be nice to upgrade SSTables in snapshot directories automatically or with a separate flag. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5524) Allow upgradesstables to be run against a specified directory
[ https://issues.apache.org/jira/browse/CASSANDRA-5524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13695962#comment-13695962 ] Alex Zarutin commented on CASSANDRA-5524: - Issue has been tested on CASSANDRA 1.2.6 and should be resolved and closed. Allow upgradesstables to be run against a specified directory - Key: CASSANDRA-5524 URL: https://issues.apache.org/jira/browse/CASSANDRA-5524 Project: Cassandra Issue Type: Improvement Components: Tools Reporter: Tyler Hobbs Assignee: Nick Bailey Priority: Minor Fix For: 1.2.6 Attachments: 0001-Add-a-snapshot-upgrade-tool.patch, 0002-Rename-snapshotupgrade-to-sstableupgrade.patch, 0003-Update-NEWS.txt-and-debian-scripts.patch, CASSANDRA-5524.txt Currently, upgradesstables only modifies live SSTables. Because sstableloader cannot stream old SSTable formats, this makes it difficult to restore data from a snapshot taken in a previous major version of Cassandra. Allowing the user to specify a directory for upgradesstables would resolve this, but it may also be nice to upgrade SSTables in snapshot directories automatically or with a separate flag. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (CASSANDRA-5714) Allow coordinator failover for cursors
Michaël Figuière created CASSANDRA-5714:
---
Summary: Allow coordinator failover for cursors
Key: CASSANDRA-5714
URL: https://issues.apache.org/jira/browse/CASSANDRA-5714
Project: Cassandra
Issue Type: Improvement
Affects Versions: 1.2.0 beta 1
Reporter: Michaël Figuière
Priority: Minor
With CASSANDRA-4415 if a coordinator fails or gets slow, causing the {{NEXT}}
request to timeout, the client application won't be able to complete its
browsing of the result. That implies that most of the time when the developer
will rely on cursors he will have to write some logic to handle a retry request
for results starting where the iteration failed. This will quickly become
painful.
Ideally the driver should handle this failover by itself by transparently
issuing this retry query when {{NEXT}} fail, but as the driver doesn't
understand CQL queries, the only thing it's aware of is the number of rows
already read. Therefore we should allow an optional parameter
{{initial_row_number}} in {{QUERY}} and {{EXECUTE}} messages that would allow
a kind of stateless failover of cursors.
With such an option, developers wouldn't have to write any failover/retry logic
on failure as they would know that everything has already been tried by the
driver.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5714) Allow coordinator failover for cursors
[
https://issues.apache.org/jira/browse/CASSANDRA-5714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13696018#comment-13696018
]
Jonathan Ellis commented on CASSANDRA-5714:
---
Row number is not indexable, use start-with-this-row instead.
Allow coordinator failover for cursors
--
Key: CASSANDRA-5714
URL: https://issues.apache.org/jira/browse/CASSANDRA-5714
Project: Cassandra
Issue Type: Improvement
Affects Versions: 1.2.0 beta 1
Reporter: Michaël Figuière
Priority: Minor
With CASSANDRA-4415 if a coordinator fails or gets slow, causing the {{NEXT}}
request to timeout, the client application won't be able to complete its
browsing of the result. That implies that most of the time when the developer
will rely on cursors he will have to write some logic to handle a retry
request for results starting where the iteration failed. This will quickly
become painful.
Ideally the driver should handle this failover by itself by transparently
issuing this retry query when {{NEXT}} fail, but as the driver doesn't
understand CQL queries, the only thing it's aware of is the number of rows
already read. Therefore we should allow an optional parameter
{{initial_row_number}} in {{QUERY}} and {{EXECUTE}} messages that would
allow a kind of stateless failover of cursors.
With such an option, developers wouldn't have to write any failover/retry
logic on failure as they would know that everything has already been tried by
the driver.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5712) Reverse slice queries can skip range tombstones
[ https://issues.apache.org/jira/browse/CASSANDRA-5712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13696055#comment-13696055 ] Jonathan Ellis commented on CASSANDRA-5712: --- I'm not a huge fan of having two Collections of atoms involved, but you're right, it's not as simple as I thought. +1 on the patch. Reverse slice queries can skip range tombstones --- Key: CASSANDRA-5712 URL: https://issues.apache.org/jira/browse/CASSANDRA-5712 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 1.2.7 Attachments: 5712.txt On disk, we represent range tombstones by a marker at the beginning of the range covered. Since we repeat such markers when they overlap an index block and since an index block is always read in forward order (even in reverse queries), we are guaranteed to see a range tombstone before any column it covers. However, IndexedSliceReader returns the columns of an index block in reverse order and thus can return a range tombstone *after* columns it covers. It follows that some range tombstone can be skipped during a reversed range slice. We need to fix IndexedSliceReader to always return range tombstone first (or at least before the first column covered by each range tombstone). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-5454) Changing column_index_size_in_kb on different nodes might corrupt files
[ https://issues.apache.org/jira/browse/CASSANDRA-5454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13696056#comment-13696056 ] Jonathan Ellis commented on CASSANDRA-5454: --- +1 Changing column_index_size_in_kb on different nodes might corrupt files --- Key: CASSANDRA-5454 URL: https://issues.apache.org/jira/browse/CASSANDRA-5454 Project: Cassandra Issue Type: Bug Affects Versions: 1.2.0 Reporter: Sylvain Lebresne Assignee: Sylvain Lebresne Fix For: 2.0 Attachments: 5454.txt RangeTombstones requires that we sometimes repeat a few markers in the data file at index boundaries. Meaning that the same row with different column_index_size_in_kb will not have the same data size. This is a problem for streaming, because if the column_index_size_in_kb is different in the source and the destination, the resulting row should have a different size on the destination, but streaming rely on the data size not changing in 1.2. Now, while having different column_index_size on different nodes is probably not extremely useful in the long run, you may still have temporal discrepancies because there is no real way to change the setting on all node atomically. Besides, it's not to hard to get different setting on different nodes due to human error. And currently, the result is that if a file is stream while the setting is not consistent, then we'll end up corrupting the received file (due to the fix from CASSANDRA-5418 to be precise). I don't see a good way to fix this in 1.2, so users will have to be careful not to have streaming happening while they change the column_index_size_in_kb setting. But in 2.0, once CASSANDRA-4180 is committed, we won't have the problem of having to respect the dataSize from the source on the destination anymore. So basically we should revert the fix from CASSANDRA-5418 (though we may still want to avoid repeating unneeded marker, but the tombstoneTracker can give us that easily). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
