[jira] [Commented] (CASSANDRA-12008) Allow retrying failed streams (or stop them from failing)
[
https://issues.apache.org/jira/browse/CASSANDRA-12008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15333150#comment-15333150
]
Tom van der Woerdt commented on CASSANDRA-12008:
That seems to have helped, the decommission I tried finished nicely.
As for "nodetool decommission resume", can we have that? :)
> Allow retrying failed streams (or stop them from failing)
> -
>
> Key: CASSANDRA-12008
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12008
> Project: Cassandra
> Issue Type: Bug
> Components: Lifecycle
>Reporter: Tom van der Woerdt
>
> We're dealing with large data sets (multiple terabytes per node) and
> sometimes we need to add or remove nodes. These operations are very dependent
> on the entire cluster being up, so while we're joining a new node (which
> sometimes takes 6 hours or longer) a lot can go wrong and in a lot of cases
> something does.
> It would be great if the ability to retry streams was implemented.
> Example to illustrate the problem :
> {code}
> 03:18 PM ~ $ nodetool decommission
> error: Stream failed
> -- StackTrace --
> org.apache.cassandra.streaming.StreamException: Stream failed
> at
> org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85)
> at com.google.common.util.concurrent.Futures$6.run(Futures.java:1310)
> at
> com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
> at
> com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
> at
> com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
> at
> com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202)
> at
> org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:210)
> at
> org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:186)
> at
> org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:430)
> at
> org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:622)
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:486)
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:274)
> at java.lang.Thread.run(Thread.java:745)
> 08:04 PM ~ $ nodetool decommission
> nodetool: Unsupported operation: Node in LEAVING state; wait for status to
> become normal or restart
> See 'nodetool help' or 'nodetool help '.
> {code}
> Streaming failed, probably due to load :
> {code}
> ERROR [STREAM-IN-/] 2016-06-14 18:05:47,275 StreamSession.java:520 -
> [Stream #] Streaming error occurred
> java.net.SocketTimeoutException: null
> at
> sun.nio.ch.SocketAdaptor$SocketInputStream.read(SocketAdaptor.java:211)
> ~[na:1.8.0_77]
> at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103)
> ~[na:1.8.0_77]
> at
> java.nio.channels.Channels$ReadableByteChannelImpl.read(Channels.java:385)
> ~[na:1.8.0_77]
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:54)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at
> org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:268)
> ~[apache-cassandra-3.0.6.jar:3.0.6]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> {code}
> If implementing retries is not possible, can we have a 'nodetool decommission
> resume'?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
cassandra git commit: validate the version string, before trying to build a Version object from it
Repository: cassandra
Updated Branches:
refs/heads/trunk 9c6c072d9 -> ed538f90e
validate the version string, before trying to build a Version object from it
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ed538f90
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ed538f90
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ed538f90
Branch: refs/heads/trunk
Commit: ed538f90efde68cda8571798d2a402e852fcba51
Parents: 9c6c072
Author: Dave Brosius
Authored: Wed Jun 15 23:12:32 2016 -0400
Committer: Dave Brosius
Committed: Wed Jun 15 23:12:32 2016 -0400
--
src/java/org/apache/cassandra/io/sstable/Descriptor.java | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/ed538f90/src/java/org/apache/cassandra/io/sstable/Descriptor.java
--
diff --git a/src/java/org/apache/cassandra/io/sstable/Descriptor.java
b/src/java/org/apache/cassandra/io/sstable/Descriptor.java
index ff4abfc..7840985 100644
--- a/src/java/org/apache/cassandra/io/sstable/Descriptor.java
+++ b/src/java/org/apache/cassandra/io/sstable/Descriptor.java
@@ -278,11 +278,12 @@ public class Descriptor
// version
nexttok = tokenStack.pop();
-Version version = fmt.info.getVersion(nexttok);
-if (!version.validate(nexttok))
+if (!Version.validate(nexttok))
throw new UnsupportedOperationException("SSTable " + name + " is
too old to open. Upgrade to 2.0 first, and run upgradesstables");
+Version version = fmt.info.getVersion(nexttok);
+
// ks/cf names
String ksname, cfname;
if (version.hasNewFileName())
cassandra git commit: make sure spillbuffer is read thoroughly
Repository: cassandra
Updated Branches:
refs/heads/trunk 91b12fc4b -> 9c6c072d9
make sure spillbuffer is read thoroughly
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9c6c072d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9c6c072d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9c6c072d
Branch: refs/heads/trunk
Commit: 9c6c072d93811a300cff754f00b5178035bf7112
Parents: 91b12fc
Author: Dave Brosius
Authored: Wed Jun 15 22:59:17 2016 -0400
Committer: Dave Brosius
Committed: Wed Jun 15 22:59:17 2016 -0400
--
.../apache/cassandra/io/util/RewindableDataInputStreamPlus.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9c6c072d/src/java/org/apache/cassandra/io/util/RewindableDataInputStreamPlus.java
--
diff --git
a/src/java/org/apache/cassandra/io/util/RewindableDataInputStreamPlus.java
b/src/java/org/apache/cassandra/io/util/RewindableDataInputStreamPlus.java
index af4d4a0..ad2e8bd 100644
--- a/src/java/org/apache/cassandra/io/util/RewindableDataInputStreamPlus.java
+++ b/src/java/org/apache/cassandra/io/util/RewindableDataInputStreamPlus.java
@@ -277,7 +277,7 @@ public class RewindableDataInputStreamPlus extends
FilterInputStream implements
if (len > 0 && diskTailAvailable > 0)
{
int readFromTail = diskTailAvailable < len? diskTailAvailable
: len;
-getIfNotClosed(spillBuffer).read(b, off, readFromTail);
+readFromTail = getIfNotClosed(spillBuffer).read(b, off,
readFromTail);
readBytes += readFromTail;
diskTailAvailable -= readFromTail;
off += readFromTail;
@@ -288,7 +288,7 @@ public class RewindableDataInputStreamPlus extends
FilterInputStream implements
if (len > 0 && diskHeadAvailable > 0)
{
int readFromHead = diskHeadAvailable < len? diskHeadAvailable
: len;
-getIfNotClosed(spillBuffer).read(b, off, readFromHead);
+readFromHead = getIfNotClosed(spillBuffer).read(b, off,
readFromHead);
readBytes += readFromHead;
diskHeadAvailable -= readFromHead;
off += readFromHead;
cassandra git commit: fix logger format string
Repository: cassandra
Updated Branches:
refs/heads/trunk b83b084df -> 91b12fc4b
fix logger format string
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/91b12fc4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/91b12fc4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/91b12fc4
Branch: refs/heads/trunk
Commit: 91b12fc4b31fb77e69705cc6e4899156b5639f14
Parents: b83b084
Author: Dave Brosius
Authored: Wed Jun 15 22:47:36 2016 -0400
Committer: Dave Brosius
Committed: Wed Jun 15 22:47:36 2016 -0400
--
.../cassandra/db/compaction/TimeWindowCompactionStrategy.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/91b12fc4/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
index d1630c5..da3ef70 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategy.java
@@ -239,7 +239,7 @@ public class TimeWindowCompactionStrategy extends
AbstractCompactionStrategy
maxTimestamp = bounds.left;
}
-logger.trace("buckets {}, max timestamp", buckets, maxTimestamp);
+logger.trace("buckets {}, max timestamp {}", buckets, maxTimestamp);
return Pair.create(buckets, maxTimestamp);
}
cassandra git commit: make inner class JMXCallbackHandler, static
Repository: cassandra
Updated Branches:
refs/heads/trunk 589528096 -> b83b084df
make inner class JMXCallbackHandler, static
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/b83b084d
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/b83b084d
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/b83b084d
Branch: refs/heads/trunk
Commit: b83b084df203d6d68452d1a09655f6f14ebfd9f0
Parents: 5895280
Author: Dave Brosius
Authored: Wed Jun 15 22:45:51 2016 -0400
Committer: Dave Brosius
Committed: Wed Jun 15 22:45:51 2016 -0400
--
src/java/org/apache/cassandra/auth/jmx/AuthenticationProxy.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/b83b084d/src/java/org/apache/cassandra/auth/jmx/AuthenticationProxy.java
--
diff --git a/src/java/org/apache/cassandra/auth/jmx/AuthenticationProxy.java
b/src/java/org/apache/cassandra/auth/jmx/AuthenticationProxy.java
index 1fe6b63..0c13e3b 100644
--- a/src/java/org/apache/cassandra/auth/jmx/AuthenticationProxy.java
+++ b/src/java/org/apache/cassandra/auth/jmx/AuthenticationProxy.java
@@ -121,7 +121,7 @@ public final class AuthenticationProxy implements
JMXAuthenticator
* prompting is necessary because the credentials are already available to
* this class (via its enclosing class).
*/
-private final class JMXCallbackHandler implements CallbackHandler
+private static final class JMXCallbackHandler implements CallbackHandler
{
private char[] username;
private char[] password;
[jira] [Commented] (CASSANDRA-11940) Look into better default file_cache_size for 2.2
[ https://issues.apache.org/jira/browse/CASSANDRA-11940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332921#comment-15332921 ] T Jake Luciani commented on CASSANDRA-11940: So looks like this is caused by the DeflateCompressor.Since there is no DirectByteBuffer call for this Compressor it is copying the buffer from mmap to an on heap then writing the decompressed output to an offheap byte buffer. Let me prove that's the only issue but in the case of Deflate compressor we should automatically use non-mmap disk access. Alternatively we can look for a decompressor library that works with off-heap byte buffers. > Look into better default file_cache_size for 2.2 > > > Key: CASSANDRA-11940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11940 > Project: Cassandra > Issue Type: Improvement >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 2.2.x > > Attachments: abnormal disk read throughput.png > > > CASSANDRA-8464 added support for mmapped decompression where in version <= > 2.1 the reads were all decompressed in standard heap buffers. > Since the usage of the file_cache is based solely on the buffer capacity we > should/can make this much larger in 2.2 when the disk access mode is mmap. > The downside of this cache being too small is made worse by 8464 since the > buffers are mmapped/unmapped causing explicit page faults. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
cassandra git commit: fix String.format to match markers and parameters
Repository: cassandra
Updated Branches:
refs/heads/trunk f2251aa41 -> 589528096
fix String.format to match markers and parameters
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/58952809
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/58952809
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/58952809
Branch: refs/heads/trunk
Commit: 5895280968d4476330806e571fca06080dce858b
Parents: f2251aa
Author: Dave Brosius
Authored: Wed Jun 15 20:49:18 2016 -0400
Committer: Dave Brosius
Committed: Wed Jun 15 20:49:18 2016 -0400
--
.../db/compaction/TimeWindowCompactionStrategyOptions.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/58952809/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategyOptions.java
--
diff --git
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategyOptions.java
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategyOptions.java
index bcbdab6..07df606 100644
---
a/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategyOptions.java
+++
b/src/java/org/apache/cassandra/db/compaction/TimeWindowCompactionStrategyOptions.java
@@ -114,12 +114,12 @@ public final class TimeWindowCompactionStrategyOptions
int sstableWindowSize = optionValue == null ?
DEFAULT_COMPACTION_WINDOW_SIZE : Integer.parseInt(optionValue);
if (sstableWindowSize < 1)
{
-throw new ConfigurationException(String.format("%s must be
greater than 1", DEFAULT_COMPACTION_WINDOW_SIZE, sstableWindowSize));
+throw new ConfigurationException(String.format("%d must be
greater than 1 for %s", sstableWindowSize, COMPACTION_WINDOW_SIZE_KEY));
}
}
catch (NumberFormatException e)
{
-throw new ConfigurationException(String.format("%s is not a
parsable int (base10) for %s", optionValue, DEFAULT_COMPACTION_WINDOW_SIZE), e);
+throw new ConfigurationException(String.format("%s is not a
parsable int (base10) for %s", optionValue, COMPACTION_WINDOW_SIZE_KEY), e);
}
optionValue = options.get(EXPIRED_SSTABLE_CHECK_FREQUENCY_SECONDS_KEY);
cassandra git commit: remove dead code
Repository: cassandra
Updated Branches:
refs/heads/trunk fe5ef557c -> f2251aa41
remove dead code
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f2251aa4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f2251aa4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f2251aa4
Branch: refs/heads/trunk
Commit: f2251aa41a766bf65363540608909238685a6b1a
Parents: fe5ef55
Author: Dave Brosius
Authored: Wed Jun 15 20:29:34 2016 -0400
Committer: Dave Brosius
Committed: Wed Jun 15 20:29:34 2016 -0400
--
src/java/org/apache/cassandra/db/view/View.java | 8
1 file changed, 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/f2251aa4/src/java/org/apache/cassandra/db/view/View.java
--
diff --git a/src/java/org/apache/cassandra/db/view/View.java
b/src/java/org/apache/cassandra/db/view/View.java
index 771c242..8babc99 100644
--- a/src/java/org/apache/cassandra/db/view/View.java
+++ b/src/java/org/apache/cassandra/db/view/View.java
@@ -17,9 +17,7 @@
*/
package org.apache.cassandra.db.view;
-import java.nio.ByteBuffer;
import java.util.*;
-import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import javax.annotation.Nullable;
@@ -31,16 +29,11 @@ import org.apache.cassandra.cql3.statements.ParsedStatement;
import org.apache.cassandra.cql3.statements.SelectStatement;
import org.apache.cassandra.db.*;
import org.apache.cassandra.config.*;
-import org.apache.cassandra.cql3.ColumnIdentifier;
import org.apache.cassandra.db.compaction.CompactionManager;
-import org.apache.cassandra.db.partitions.*;
import org.apache.cassandra.db.rows.*;
import org.apache.cassandra.schema.KeyspaceMetadata;
import org.apache.cassandra.service.ClientState;
-import org.apache.cassandra.service.pager.QueryPager;
-import org.apache.cassandra.transport.Server;
import org.apache.cassandra.utils.FBUtilities;
-import org.apache.cassandra.utils.btree.BTreeSet;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -96,7 +89,6 @@ public class View
{
this.definition = definition;
-CFMetaData viewCfm = definition.metadata;
List nonPKDefPartOfViewPK = new ArrayList<>();
for (ColumnDefinition baseColumn : baseCfs.metadata.allColumns())
{
cassandra git commit: if there's no truststore, no need to fetch other ssl attributes
Repository: cassandra
Updated Branches:
refs/heads/trunk 693e52d66 -> fe5ef557c
if there's no truststore, no need to fetch other ssl attributes
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/fe5ef557
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/fe5ef557
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/fe5ef557
Branch: refs/heads/trunk
Commit: fe5ef557c076c9730e6d71efafc711c43194905c
Parents: 693e52d
Author: Dave Brosius
Authored: Wed Jun 15 20:19:47 2016 -0400
Committer: Dave Brosius
Committed: Wed Jun 15 20:19:47 2016 -0400
--
.../org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/fe5ef557/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
--
diff --git a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
index 757be65..f76f7d9 100644
--- a/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
+++ b/src/java/org/apache/cassandra/hadoop/cql3/CqlConfigHelper.java
@@ -529,13 +529,14 @@ public class CqlConfigHelper
public static Optional getSSLOptions(Configuration conf)
{
Optional truststorePath =
getInputNativeSSLTruststorePath(conf);
-Optional keystorePath = getInputNativeSSLKeystorePath(conf);
-Optional truststorePassword =
getInputNativeSSLTruststorePassword(conf);
-Optional keystorePassword =
getInputNativeSSLKeystorePassword(conf);
-Optional cipherSuites = getInputNativeSSLCipherSuites(conf);
if (truststorePath.isPresent())
{
+Optional keystorePath =
getInputNativeSSLKeystorePath(conf);
+Optional truststorePassword =
getInputNativeSSLTruststorePassword(conf);
+Optional keystorePassword =
getInputNativeSSLKeystorePassword(conf);
+Optional cipherSuites =
getInputNativeSSLCipherSuites(conf);
+
SSLContext context;
try
{
[jira] [Updated] (CASSANDRA-12013) Optimize the isEOF() checking in RAR
[ https://issues.apache.org/jira/browse/CASSANDRA-12013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu updated CASSANDRA-12013: -- Status: Patch Available (was: Open) > Optimize the isEOF() checking in RAR > > > Key: CASSANDRA-12013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12013 > Project: Cassandra > Issue Type: Improvement >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Minor > Labels: compaction > Fix For: 2.2.x > > Attachments: 0001-Optimize-the-isEOF-checking-in-RAR.patch, > Screenshot 2016-06-15 11.20.51.png > > > When I profile compaction in 2.1, I find the isEOF() is an expensive > operation, and we do the check for each read in RAR, which I think is > unnecessary. > I attach a patch to only check isEOF() when we need to rebuffer, and then I > do not see it from hotspot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332742#comment-15332742 ] Tyler Hobbs commented on CASSANDRA-8700: Pull request for replication and tunable consistency levels: https://github.com/pcmanus/cassandra/pull/55 > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne >Priority: Blocker > Fix For: 3.8 > > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11873) Add duration type
[
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332694#comment-15332694
]
Patrick McFadin commented on CASSANDRA-11873:
-
I am +1 on this. Making up completely new syntax is just harsh to the end user.
No need to re-invent the wheel.
> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration
> type.
> In my opinion, it should be represented internally as a number of
> microseconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11873) Add duration type
[
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332685#comment-15332685
]
Brian Hess commented on CASSANDRA-11873:
-
So, one question about syntax. SQL has a time interval type with its own
syntax. Instead of inventing additional syntax, what about taking on the
Postgres or Oracle/SQL-Standard syntax.
For example, Postgres uses "1 day ago" or "-1 day" (or hour, minute, etc):
https://www.postgresql.org/docs/current/static/datatype-datetime.html
Oracle uses "INTERVAL '1' HOUR":
https://docs.oracle.com/cd/B28359_01/server.111/b28286/sql_elements003.htm#i38598
I suggest we choose something similar to one that exists, rather than create
our own new syntax.
> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration
> type.
> In my opinion, it should be represented internally as a number of
> microseconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11519) Add support for IBM POWER
[ https://issues.apache.org/jira/browse/CASSANDRA-11519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332667#comment-15332667 ] Joshua McKenzie commented on CASSANDRA-11519: - bq. The latest POWER8 processor, released two years ago, supports hardware-based unaligned access for every combination of 32-bit/64-bit integer/floating-point numbers. bq. Actually, the linked document is quite old Yep, that's on me - I should have spent a little more time and caught that. It's pretty clear that one of the big selling points of POWER8 is the unaligned access performance. Is there a way to differentiate POWER8 vs. older architectures via os.arch and at least warn about the performance implications? I don't know how widespread running C* on POWER is so I'd hate to put in a change that would negatively impact some users out there. > Add support for IBM POWER > - > > Key: CASSANDRA-11519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11519 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: POWER architecture >Reporter: Rei Odaira >Assignee: Rei Odaira >Priority: Minor > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x > > Attachments: 11519-2.1.txt, 11519-3.0.txt > > > Add support for the IBM POWER architecture (ppc, ppc64, and ppc64le) in > org.apache.cassandra.utils.FastByteOperations, > org.apache.cassandra.utils.memory.MemoryUtil, and > org.apache.cassandra.io.util.Memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9507) range metrics are not updated for timeout and unavailable in StorageProxy
[ https://issues.apache.org/jira/browse/CASSANDRA-9507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Brown updated CASSANDRA-9507: --- Assignee: Nachiket Patil > range metrics are not updated for timeout and unavailable in StorageProxy > - > > Key: CASSANDRA-9507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9507 > Project: Cassandra > Issue Type: Bug > Components: Observability >Reporter: sankalp kohli >Assignee: Nachiket Patil >Priority: Minor > Attachments: Cassandra-9507.diff > > > Looking at the code, it looks like range metrics are not updated for timeouts > and unavailable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11949) GC log directory should be created in startup scripts
[
https://issues.apache.org/jira/browse/CASSANDRA-11949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332627#comment-15332627
]
Mahdi Mohammadi commented on CASSANDRA-11949:
-
Branch: [2.2|https://github.com/mm-binary/cassandra/tree/11949-2.2]
> GC log directory should be created in startup scripts
> -
>
> Key: CASSANDRA-11949
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11949
> Project: Cassandra
> Issue Type: Bug
>Reporter: Joel Knighton
>Assignee: Mahdi Mohammadi
>Priority: Minor
>
> In [CASSANDRA-10140], we enabled GC logging by default, since the overhead
> was low and asking people providing diagnostics to restart can often make it
> more difficult to diagnose problems.
> The default GC log path is set to {{$CASSANDRA_HOME/logs/gc.log}} in
> {{cassandra-env.sh}}, a directory that is not present in a fresh
> clone/install. Even if logback creates this directory later in startup, it is
> not present when the JVM initiates GC logging, so GC logging will silently
> fail for this first Cassandra run
> I haven't tested this in Windows but suspect the same problem may occur.
> Since lots of tooling around Cassandra won't create this directory, we should
> instead consider attempting to create it in our startup scripts.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11519) Add support for IBM POWER
[ https://issues.apache.org/jira/browse/CASSANDRA-11519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332607#comment-15332607 ] Rei Odaira commented on CASSANDRA-11519: Hi, The linked document says PowerPC does not support unaligned 64-bit floating-point access in the hardware, but it also says PowerPC supports hardware-based unaligned integer access. As far as I understand, all of the unaligned accesses in Memory.java, MemoryUtil.java, and FastByteOperations.java are for integers. Actually, the linked document is quite old (written in 2005). The latest POWER8 processor, released two years ago, supports hardware-based unaligned access for every combination of 32-bit/64-bit integer/floating-point numbers. In my measurement, there is no measurable performance penalty in unaligned access, compared with aligned access. The only exception is access that crosses a page boundary, which is 10x slower than normal access, but it is also the case in x86. If you need to know the situation in POWER7 and older processors, I'll investigate them. There are some other implementations of the POWER architecture (e.g. for game consoles), but I believe POWER8, which is targeting the server market, is the only POWER implementation that practically runs Cassandra. > Add support for IBM POWER > - > > Key: CASSANDRA-11519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11519 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: POWER architecture >Reporter: Rei Odaira >Assignee: Rei Odaira >Priority: Minor > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x > > Attachments: 11519-2.1.txt, 11519-3.0.txt > > > Add support for the IBM POWER architecture (ppc, ppc64, and ppc64le) in > org.apache.cassandra.utils.FastByteOperations, > org.apache.cassandra.utils.memory.MemoryUtil, and > org.apache.cassandra.io.util.Memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-12012) CQLSSTableWriter and composite clustering keys trigger NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-12012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahdi Mohammadi reassigned CASSANDRA-12012:
---
Assignee: Mahdi Mohammadi
> CQLSSTableWriter and composite clustering keys trigger NPE
> --
>
> Key: CASSANDRA-12012
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12012
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Pierre N.
>Assignee: Mahdi Mohammadi
>
> It triggers when using multiple clustering keys in the primary keys
> {code}
> package tests;
> import java.io.File;
> import org.apache.cassandra.io.sstable.CQLSSTableWriter;
> import org.apache.cassandra.config.Config;
> public class DefaultWriter {
>
> public static void main(String[] args) throws Exception {
> Config.setClientMode(true);
>
> String createTableQuery = "CREATE TABLE ks_test.table_test ("
> + "pk1 int,"
> + "ck1 int,"
> + "ck2 int,"
> + "PRIMARY KEY ((pk1), ck1, ck2)"
> + ");";
> String insertQuery = "INSERT INTO ks_test.table_test(pk1, ck1, ck2)
> VALUES(?,?,?)";
>
> CQLSSTableWriter writer = CQLSSTableWriter.builder()
> .inDirectory(File.createTempFile("sstdir", "-tmp"))
> .forTable(createTableQuery)
> .using(insertQuery)
> .build();
> writer.close();
> }
> }
> {code}
> Exception :
> {code}
> Exception in thread "main" java.lang.ExceptionInInitializerError
> at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
> at org.apache.cassandra.db.Keyspace.(Keyspace.java:305)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
> at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
> at org.apache.cassandra.db.Keyspace.openAndGetStore(Keyspace.java:159)
> at
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.hasSupportingIndex(PrimaryKeyRestrictionSet.java:156)
> at
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.(PrimaryKeyRestrictionSet.java:118)
> at
> org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.mergeWith(PrimaryKeyRestrictionSet.java:213)
> at
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.addSingleColumnRestriction(StatementRestrictions.java:266)
> at
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.addRestriction(StatementRestrictions.java:250)
> at
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:159)
> at
> org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:183)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:782)
> at
> org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:768)
> at
> org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:505)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.getStatement(CQLSSTableWriter.java:508)
> at
> org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.using(CQLSSTableWriter.java:439)
> at tests.DefaultWriter.main(DefaultWriter.java:29)
> Caused by: java.lang.NullPointerException
> at
> org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1188)
> at
> org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:127)
> ... 18 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11576) Add support for JNA mlockall(2) on POWER
[
https://issues.apache.org/jira/browse/CASSANDRA-11576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332582#comment-15332582
]
Joshua McKenzie commented on CASSANDRA-11576:
-
Oh, I missed the fact that you were going the octal route. How about we keep
them as hex so we have consistency w/the original declaration they're referring
to?
> Add support for JNA mlockall(2) on POWER
>
>
> Key: CASSANDRA-11576
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11576
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: POWER architecture
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: 11576-2.1.txt
>
>
> org.apache.cassandra.utils.CLibrary contains hard-coded C-macro values to be
> passed to system calls through JNA. These values are system-dependent, and as
> far as I investigated, Linux and AIX on the IBM POWER architecture define
> {{MCL_CURRENT}} and {{MCL_FUTURE}} (for mlockall(2)) as different values than
> the current hard-coded values. As a result, mlockall(2) fails on these
> platforms.
> {code}
> WARN 18:51:51 Unknown mlockall error 22
> {code}
> I am going to provide a patch to support JNA mlockall(2) on POWER.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12014) IndexSummary > 2G causes an assertion error
[
https://issues.apache.org/jira/browse/CASSANDRA-12014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-12014:
-
Description:
{noformat}
ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444
CassandraDaemon.java:229 - Exception in thread
Thread[CompactionExecutor:1546280,1,main]
java.lang.AssertionError: null
at
org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{noformat}
I believe this can be fixed by raising the min_index_interval, but we should
have a better method of coping with this than throwing the AE.
was:
{{noformat}}
ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444
CassandraDaemon.java:229 - Exception in thread
Thread[CompactionExecutor:1546280,1,main]
java.lang.AssertionError: null
at
org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{{noformat}
I believe this can be fixed by raising the min_index_interval, but we should
have a better method of coping with this than throwing the AE.
> IndexSummary > 2G causes an assertion error
> ---
>
> Key: CASSANDRA-12014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Brando
[jira] [Created] (CASSANDRA-12014) IndexSummary > 2G causes an assertion error
Brandon Williams created CASSANDRA-12014:
Summary: IndexSummary > 2G causes an assertion error
Key: CASSANDRA-12014
URL: https://issues.apache.org/jira/browse/CASSANDRA-12014
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Brandon Williams
Fix For: 2.1.x
{{noformat}}
ERROR [CompactionExecutor:1546280] 2016-06-01 13:21:00,444
CassandraDaemon.java:229 - Exception in thread
Thread[CompactionExecutor:1546280,1,main]
java.lang.AssertionError: null
at
org.apache.cassandra.io.sstable.IndexSummaryBuilder.maybeAddEntry(IndexSummaryBuilder.java:171)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter$IndexWriter.append(SSTableWriter.java:634)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter.afterAppend(SSTableWriter.java:179)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:205)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.io.sstable.SSTableRewriter.append(SSTableRewriter.java:126)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:197)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:73)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
~[cassandra-all-2.1.12.1046.jar:2.1.12.1046]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
~[na:1.7.0_51]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_51]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_51]
at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
{{noformat}
I believe this can be fixed by raising the min_index_interval, but we should
have a better method of coping with this than throwing the AE.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10786) Include hash of result set metadata in prepared statement id
[
https://issues.apache.org/jira/browse/CASSANDRA-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-10786:
Status: Patch Available (was: Open)
What I've done was simply re-preparing and having try/catch around. But I think
you're right, it might be too error-prone to do it that way. It might be better
to let users re-prepare the very first statement and let the rest be handled
through the new {{Rows}}.
The patch is located here. Test failure for
{{UserTypesTest#testAlteringUserTypeNestedWithinNonFrozenMap}} even though
looks related, was failing on trunk, the patch for it is submitted
[here|https://issues.apache.org/jira/browse/CASSANDRA-12010]. The other tests
seem unrelated and are failing on trunk, too.
Please note that the patched binary for the driver was updated in tree, for
test purposes.
Test runs:
|[trunk|https://github.com/ifesdjeen/cassandra/tree/10786-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-10756-trunk-testall/]|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-10786-trunk-dtest/]|
And patch for the driver is located
[here|https://github.com/ifesdjeen/java-driver/tree/10786-v5].
> Include hash of result set metadata in prepared statement id
>
>
> Key: CASSANDRA-10786
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10786
> Project: Cassandra
> Issue Type: Bug
> Components: CQL
>Reporter: Olivier Michallat
>Assignee: Alex Petrov
>Priority: Minor
> Labels: client-impacting, doc-impacting, protocolv5
> Fix For: 3.x
>
>
> This is a follow-up to CASSANDRA-7910, which was about invalidating a
> prepared statement when the table is altered, to force clients to update
> their local copy of the metadata.
> There's still an issue if multiple clients are connected to the same host.
> The first client to execute the query after the cache was invalidated will
> receive an UNPREPARED response, re-prepare, and update its local metadata.
> But other clients might miss it entirely (the MD5 hasn't changed), and they
> will keep using their old metadata. For example:
> # {{SELECT * ...}} statement is prepared in Cassandra with md5 abc123,
> clientA and clientB both have a cache of the metadata (columns b and c)
> locally
> # column a gets added to the table, C* invalidates its cache entry
> # clientA sends an EXECUTE request for md5 abc123, gets UNPREPARED response,
> re-prepares on the fly and updates its local metadata to (a, b, c)
> # prepared statement is now in C*’s cache again, with the same md5 abc123
> # clientB sends an EXECUTE request for id abc123. Because the cache has been
> populated again, the query succeeds. But clientB still has not updated its
> metadata, it’s still (b,c)
> One solution that was suggested is to include a hash of the result set
> metadata in the md5. This way the md5 would change at step 3, and any client
> using the old md5 would get an UNPREPARED, regardless of whether another
> client already reprepared.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332407#comment-15332407 ] Tyler Hobbs commented on CASSANDRA-8700: Pull requests for cqlsh docs: https://github.com/pcmanus/cassandra/pull/54 > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne >Priority: Blocker > Fix For: 3.8 > > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12010) UserTypesTest# is failing on trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Knighton updated CASSANDRA-12010: -- Reviewer: Joel Knighton > UserTypesTest# is failing on trunk > -- > > Key: CASSANDRA-12010 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12010 > Project: Cassandra > Issue Type: Test >Reporter: Alex Petrov >Assignee: Alex Petrov > > Test failure: > http://cassci.datastax.com/job/trunk_utest/1445/testReport/org.apache.cassandra.cql3.validation.entities/UserTypesTest/testAlteringUserTypeNestedWithinNonFrozenMap/ > This was caused by the merge after > [11604|https://issues.apache.org/jira/browse/CASSANDRA-11604] which probably > coincided with some other change, as this failure did not happen during the > [test run on the > branch|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11604-trunk-testall/lastCompletedBuild/testReport/]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332356#comment-15332356 ] Sylvain Lebresne commented on CASSANDRA-8700: - bq. I suggest not putting the generated HTML in git, but rather have it be a build artifact like a jar would be. That's exactly what it does currently, and I agree with the general sentiment, but in practice our web site is managed through SVN (through some kind of pubsub managed by the ASF) so the html will have to get there at some point (but we push to the website rarely and without concurrency so I don't think "it'll turn into a mess" in practice). And my mention that I'd rather have this done with git rather than SVN is completely orthogonal to the documentation problem, I was just rambling, let's forget about it here. > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne >Priority: Blocker > Fix For: 3.8 > > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11576) Add support for JNA mlockall(2) on POWER
[
https://issues.apache.org/jira/browse/CASSANDRA-11576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332355#comment-15332355
]
Rei Odaira commented on CASSANDRA-11576:
Sorry for the confusion, but 0x2000 == 02 and 0x4000 == 04. If they
should be in hexadecimal numbers, I can fix the patch.
> Add support for JNA mlockall(2) on POWER
>
>
> Key: CASSANDRA-11576
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11576
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Environment: POWER architecture
>Reporter: Rei Odaira
>Assignee: Rei Odaira
>Priority: Minor
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
> Attachments: 11576-2.1.txt
>
>
> org.apache.cassandra.utils.CLibrary contains hard-coded C-macro values to be
> passed to system calls through JNA. These values are system-dependent, and as
> far as I investigated, Linux and AIX on the IBM POWER architecture define
> {{MCL_CURRENT}} and {{MCL_FUTURE}} (for mlockall(2)) as different values than
> the current hard-coded values. As a result, mlockall(2) fails on these
> platforms.
> {code}
> WARN 18:51:51 Unknown mlockall error 22
> {code}
> I am going to provide a patch to support JNA mlockall(2) on POWER.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12013) Optimize the isEOF() checking in RAR
[ https://issues.apache.org/jira/browse/CASSANDRA-12013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dikang Gu updated CASSANDRA-12013: -- Attachment: 0001-Optimize-the-isEOF-checking-in-RAR.patch the patch based on 2.2 > Optimize the isEOF() checking in RAR > > > Key: CASSANDRA-12013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12013 > Project: Cassandra > Issue Type: Improvement >Reporter: Dikang Gu >Assignee: Dikang Gu >Priority: Minor > Labels: compaction > Fix For: 2.2.x > > Attachments: 0001-Optimize-the-isEOF-checking-in-RAR.patch, > Screenshot 2016-06-15 11.20.51.png > > > When I profile compaction in 2.1, I find the isEOF() is an expensive > operation, and we do the check for each read in RAR, which I think is > unnecessary. > I attach a patch to only check isEOF() when we need to rebuffer, and then I > do not see it from hotspot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[Cassandra Wiki] Update of "Committers" by TylerHobbs
Dear Wiki user, You have subscribed to a wiki page or wiki category on "Cassandra Wiki" for change notification. The "Committers" page has been changed by TylerHobbs: https://wiki.apache.org/cassandra/Committers?action=diff&rev1=58&rev2=59 Comment: Mark thobbs as a PMC member ||Jason Brown ||Feb 2013 ||Apple || || ||Marcus Eriksson ||Apr 2013 ||Datastax || || ||Mikhail Stepura ||Jan 2014 ||Apple || || - ||Tyler Hobbs ||Mar 2014 ||Datastax || || + ||Tyler Hobbs ||Mar 2014 ||Datastax ||PMC member || ||Benedict Elliott Smith ||May 2014 ||Vast || || ||Josh Mckenzie ||Jul 2014 ||Datastax || || ||Robert Stupp ||Jan 2015 ||Datastax || ||
[jira] [Created] (CASSANDRA-12013) Optimize the isEOF() checking in RAR
Dikang Gu created CASSANDRA-12013: - Summary: Optimize the isEOF() checking in RAR Key: CASSANDRA-12013 URL: https://issues.apache.org/jira/browse/CASSANDRA-12013 Project: Cassandra Issue Type: Improvement Reporter: Dikang Gu Assignee: Dikang Gu Priority: Minor Fix For: 2.2.x Attachments: Screenshot 2016-06-15 11.20.51.png When I profile compaction in 2.1, I find the isEOF() is an expensive operation, and we do the check for each read in RAR, which I think is unnecessary. I attach a patch to only check isEOF() when we need to rebuffer, and then I do not see it from hotspot. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12012) CQLSSTableWriter and composite clustering keys trigger NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-12012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre N. updated CASSANDRA-12012:
--
Description:
It triggers when using multiple clustering keys in the primary keys
{code}
package tests;
import java.io.File;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;
import org.apache.cassandra.config.Config;
public class DefaultWriter {
public static void main(String[] args) throws Exception {
Config.setClientMode(true);
String createTableQuery = "CREATE TABLE ks_test.table_test ("
+ "pk1 int,"
+ "ck1 int,"
+ "ck2 int,"
+ "PRIMARY KEY ((pk1), ck1, ck2)"
+ ");";
String insertQuery = "INSERT INTO ks_test.table_test(pk1, ck1, ck2)
VALUES(?,?,?)";
CQLSSTableWriter writer = CQLSSTableWriter.builder()
.inDirectory(File.createTempFile("sstdir", "-tmp"))
.forTable(createTableQuery)
.using(insertQuery)
.build();
writer.close();
}
}
{code}
Exception :
{code}
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
at org.apache.cassandra.db.Keyspace.(Keyspace.java:305)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
at org.apache.cassandra.db.Keyspace.openAndGetStore(Keyspace.java:159)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.hasSupportingIndex(PrimaryKeyRestrictionSet.java:156)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.(PrimaryKeyRestrictionSet.java:118)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.mergeWith(PrimaryKeyRestrictionSet.java:213)
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.addSingleColumnRestriction(StatementRestrictions.java:266)
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.addRestriction(StatementRestrictions.java:250)
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:159)
at
org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:183)
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:782)
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:768)
at
org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:505)
at
org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.getStatement(CQLSSTableWriter.java:508)
at
org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.using(CQLSSTableWriter.java:439)
at tests.DefaultWriter.main(DefaultWriter.java:29)
Caused by: java.lang.NullPointerException
at
org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1188)
at
org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:127)
... 18 more
{code}
was:
It triggers when using multiple clustering keys in the primary keys
{code}
package tests;
import java.io.File;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;
import org.apache.cassandra.config.Config;
public class DefaultWriter {
public static void main(String[] args) throws Exception {
Config.setClientMode(true);
String createTableQuery = "CREATE TABLE ks_test.table_test ("
+ "pk1 int,"
+ "ck1 int,"
+ "ck2 int,"
+ "PRIMARY KEY ((pk1), ck1, ck2)"
+ ");";
String insertQuery = "INSERT INTO ks_test.table_test(pk1, ck1, ck2)
VALUES(?,?,?)";
CQLSSTableWriter writer = CQLSSTableWriter.builder()
.inDirectory(File.createTempFile("sstdir", "-tmp"))
.forTable(createTableQuery)
.using(insertQuery)
.build();
writer.close();
}
}
{/code}
Exception :
{code}
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
at org.apache.cassandra.db.Keyspace.(Keyspace.java:305)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
at org.apache.cassandra.db.Keyspace.openAndGetStore(Keyspace.java:159)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.hasSupportingIndex(PrimaryKeyRestrictionSet.java:156)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.(PrimaryKeyRestrictionSet.java:118)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.mergeWith(PrimaryKeyRestrictionSet.java:
[jira] [Created] (CASSANDRA-12012) CQLSSTableWriter and composite clustering keys trigger NPE
Pierre N. created CASSANDRA-12012:
-
Summary: CQLSSTableWriter and composite clustering keys trigger NPE
Key: CASSANDRA-12012
URL: https://issues.apache.org/jira/browse/CASSANDRA-12012
Project: Cassandra
Issue Type: Bug
Components: Streaming and Messaging
Reporter: Pierre N.
It triggers when using multiple clustering keys in the primary keys
{code}
package tests;
import java.io.File;
import org.apache.cassandra.io.sstable.CQLSSTableWriter;
import org.apache.cassandra.config.Config;
public class DefaultWriter {
public static void main(String[] args) throws Exception {
Config.setClientMode(true);
String createTableQuery = "CREATE TABLE ks_test.table_test ("
+ "pk1 int,"
+ "ck1 int,"
+ "ck2 int,"
+ "PRIMARY KEY ((pk1), ck1, ck2)"
+ ");";
String insertQuery = "INSERT INTO ks_test.table_test(pk1, ck1, ck2)
VALUES(?,?,?)";
CQLSSTableWriter writer = CQLSSTableWriter.builder()
.inDirectory(File.createTempFile("sstdir", "-tmp"))
.forTable(createTableQuery)
.using(insertQuery)
.build();
writer.close();
}
}
{/code}
Exception :
{code}
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.apache.cassandra.db.Keyspace.initCf(Keyspace.java:368)
at org.apache.cassandra.db.Keyspace.(Keyspace.java:305)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:129)
at org.apache.cassandra.db.Keyspace.open(Keyspace.java:106)
at org.apache.cassandra.db.Keyspace.openAndGetStore(Keyspace.java:159)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.hasSupportingIndex(PrimaryKeyRestrictionSet.java:156)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.(PrimaryKeyRestrictionSet.java:118)
at
org.apache.cassandra.cql3.restrictions.PrimaryKeyRestrictionSet.mergeWith(PrimaryKeyRestrictionSet.java:213)
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.addSingleColumnRestriction(StatementRestrictions.java:266)
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.addRestriction(StatementRestrictions.java:250)
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.(StatementRestrictions.java:159)
at
org.apache.cassandra.cql3.statements.UpdateStatement$ParsedInsert.prepareInternal(UpdateStatement.java:183)
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:782)
at
org.apache.cassandra.cql3.statements.ModificationStatement$Parsed.prepare(ModificationStatement.java:768)
at
org.apache.cassandra.cql3.QueryProcessor.getStatement(QueryProcessor.java:505)
at
org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.getStatement(CQLSSTableWriter.java:508)
at
org.apache.cassandra.io.sstable.CQLSSTableWriter$Builder.using(CQLSSTableWriter.java:439)
at tests.DefaultWriter.main(DefaultWriter.java:29)
Caused by: java.lang.NullPointerException
at
org.apache.cassandra.config.DatabaseDescriptor.getFlushWriters(DatabaseDescriptor.java:1188)
at
org.apache.cassandra.db.ColumnFamilyStore.(ColumnFamilyStore.java:127)
... 18 more
{/code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8700: -- Fix Version/s: (was: 3.x) 3.8 > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne > Fix For: 3.8 > > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8700: -- Priority: Blocker (was: Major) > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne >Priority: Blocker > Fix For: 3.8 > > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8700: -- Priority: Major (was: Minor) > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne > Fix For: 3.x > > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani updated CASSANDRA-8700: -- Fix Version/s: 3.x > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne >Priority: Minor > Fix For: 3.x > > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-12011) Logback shutdown hook races with StorageServiceShutdownHook
Joel Knighton created CASSANDRA-12011:
-
Summary: Logback shutdown hook races with
StorageServiceShutdownHook
Key: CASSANDRA-12011
URL: https://issues.apache.org/jira/browse/CASSANDRA-12011
Project: Cassandra
Issue Type: Bug
Reporter: Joel Knighton
Priority: Minor
Fix For: 2.2.x, 3.0.x, 3.x
In {{StorageService}}, we add a JVM shutdown hook called
{{StorageServiceShutdownHook}} that shuts down gossip and flushes writes for
non durable keyspaces, among other things.
In {{logback.xml}}, we add a JVM shutdown hook that tears down logback
resources.
Since JVM shutdown hooks are started concurrently, the logback shutdown hook
almost always completes during the {{StorageServiceShutdownHook}}, which means
that any log statements in this hook will not work since the logback resources
have been torn down.
This makes debugging the {{StorageServiceShutdownHook}} challenging. We should
do our own logback teardown that does not run until after the completion of any
Cassandra-oriented JVM shutdown hooks such as the
{{StorageServiceShutdownHook}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8119) More Expressive Consistency Levels
[
https://issues.apache.org/jira/browse/CASSANDRA-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332147#comment-15332147
]
Aleksey Yeschenko commented on CASSANDRA-8119:
--
I'll elaborate eventually, once I'm off vacation. That said, I'd be somewhat
fine with it as proposed initially (optionally CL composed of a map of DC ->
level).
> More Expressive Consistency Levels
> --
>
> Key: CASSANDRA-8119
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8119
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Tyler Hobbs
> Fix For: 3.x
>
>
> For some multi-datacenter environments, the current set of consistency levels
> are too restrictive. For example, the following consistency requirements
> cannot be expressed:
> * LOCAL_QUORUM in two specific DCs
> * LOCAL_QUORUM in the local DC plus LOCAL_QUORUM in at least one other DC
> * LOCAL_QUORUM in the local DC plus N remote replicas in any DC
> I propose that we add a new consistency level: CUSTOM. In the v4 (or v5)
> protocol, this would be accompanied by an additional map argument. A map of
> {DC: CL} or a map of {DC: int} is sufficient to cover the first example. If
> we accept a special keys to represent "any datacenter", the second case can
> be handled. A similar technique could be used for "any other nodes".
> I'm not in love with the special keys, so if anybody has ideas for something
> more elegant, feel free to propose them. The main idea is that we want to be
> flexible enough to cover any reasonable consistency or durability
> requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8700) replace the wiki with docs in the git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-8700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332106#comment-15332106 ] Jon Haddad commented on CASSANDRA-8700: --- I suggest not putting the generated HTML in git, but rather have it be a build artifact like a jar would be. If you do any sort of image generation through plugins like nwdiag (for network diagrams) it'll turn into a mess. Doesn't the ASF provide some space for this sort of thing? > replace the wiki with docs in the git repo > -- > > Key: CASSANDRA-8700 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8700 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Sylvain Lebresne >Priority: Minor > Attachments: TombstonesAndGcGrace.md, bloom_filters.md, > compression.md, contributing.zip, getting_started.zip, hardware.md > > > The wiki as it stands is pretty terrible. It takes several minutes to apply > a single update, and as a result, it's almost never updated. The information > there has very little context as to what version it applies to. Most people > I've talked to that try to use the information they find there find it is > more confusing than helpful. > I'd like to propose that instead of using the wiki, the doc directory in the > cassandra repo be used for docs (already used for CQL3 spec) in a format that > can be built to a variety of output formats like HTML / epub / etc. I won't > start the bikeshedding on which markup format is preferable - but there are > several options that can work perfectly fine. I've personally use sphinx w/ > restructured text, and markdown. Both can build easily and as an added bonus > be pushed to readthedocs (or something similar) automatically. For an > example, see cqlengine's documentation, which I think is already > significantly better than the wiki: > http://cqlengine.readthedocs.org/en/latest/ > In addition to being overall easier to maintain, putting the documentation in > the git repo adds context, since it evolves with the versions of Cassandra. > If the wiki were kept even remotely up to date, I wouldn't bother with this, > but not having at least some basic documentation in the repo, or anywhere > associated with the project, is frustrating. > For reference, the last 3 updates were: > 1/15/15 - updating committers list > 1/08/15 - updating contributers and how to contribute > 12/16/14 - added a link to CQL docs from wiki frontpage (by me) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)
[ https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332097#comment-15332097 ] Carl Yeksigian commented on CASSANDRA-8844: --- Nothing more from me - code looks good and I just ran CASSANDRA-11575 against it again to make sure it was all still working. +1 > Change Data Capture (CDC) > - > > Key: CASSANDRA-8844 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8844 > Project: Cassandra > Issue Type: New Feature > Components: Coordination, Local Write-Read Paths >Reporter: Tupshin Harper >Assignee: Joshua McKenzie >Priority: Critical > Fix For: 3.x > > > "In databases, change data capture (CDC) is a set of software design patterns > used to determine (and track) the data that has changed so that action can be > taken using the changed data. Also, Change data capture (CDC) is an approach > to data integration that is based on the identification, capture and delivery > of the changes made to enterprise data sources." > -Wikipedia > As Cassandra is increasingly being used as the Source of Record (SoR) for > mission critical data in large enterprises, it is increasingly being called > upon to act as the central hub of traffic and data flow to other systems. In > order to try to address the general need, we (cc [~brianmhess]), propose > implementing a simple data logging mechanism to enable per-table CDC patterns. > h2. The goals: > # Use CQL as the primary ingestion mechanism, in order to leverage its > Consistency Level semantics, and in order to treat it as the single > reliable/durable SoR for the data. > # To provide a mechanism for implementing good and reliable > (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) > continuous semi-realtime feeds of mutations going into a Cassandra cluster. > # To eliminate the developmental and operational burden of users so that they > don't have to do dual writes to other systems. > # For users that are currently doing batch export from a Cassandra system, > give them the opportunity to make that realtime with a minimum of coding. > h2. The mechanism: > We propose a durable logging mechanism that functions similar to a commitlog, > with the following nuances: > - Takes place on every node, not just the coordinator, so RF number of copies > are logged. > - Separate log per table. > - Per-table configuration. Only tables that are specified as CDC_LOG would do > any logging. > - Per DC. We are trying to keep the complexity to a minimum to make this an > easy enhancement, but most likely use cases would prefer to only implement > CDC logging in one (or a subset) of the DCs that are being replicated to > - In the critical path of ConsistencyLevel acknowledgment. Just as with the > commitlog, failure to write to the CDC log should fail that node's write. If > that means the requested consistency level was not met, then clients *should* > experience UnavailableExceptions. > - Be written in a Row-centric manner such that it is easy for consumers to > reconstitute rows atomically. > - Written in a simple format designed to be consumed *directly* by daemons > written in non JVM languages > h2. Nice-to-haves > I strongly suspect that the following features will be asked for, but I also > believe that they can be deferred for a subsequent release, and to guage > actual interest. > - Multiple logs per table. This would make it easy to have multiple > "subscribers" to a single table's changes. A workaround would be to create a > forking daemon listener, but that's not a great answer. > - Log filtering. Being able to apply filters, including UDF-based filters > would make Casandra a much more versatile feeder into other systems, and > again, reduce complexity that would otherwise need to be built into the > daemons. > h2. Format and Consumption > - Cassandra would only write to the CDC log, and never delete from it. > - Cleaning up consumed logfiles would be the client daemon's responibility > - Logfile size should probably be configurable. > - Logfiles should be named with a predictable naming schema, making it > triivial to process them in order. > - Daemons should be able to checkpoint their work, and resume from where they > left off. This means they would have to leave some file artifact in the CDC > log's directory. > - A sophisticated daemon should be able to be written that could > -- Catch up, in written-order, even when it is multiple logfiles behind in > processing > -- Be able to continuously "tail" the most recent logfile and get > low-latency(ms?) access to the data as it is written. > h2. Alternate approach > In order to make consuming a change log easy and efficient to do with low > latency, the following could supplement the appr
[jira] [Updated] (CASSANDRA-11991) On clock skew, paxos may "corrupt" the node clock
[ https://issues.apache.org/jira/browse/CASSANDRA-11991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sankalp kohli updated CASSANDRA-11991: -- Reviewer: sankalp kohli > On clock skew, paxos may "corrupt" the node clock > - > > Key: CASSANDRA-11991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11991 > Project: Cassandra > Issue Type: Bug >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 2.1.x, 2.2.x, 3.0.x > > > W made a mistake in CASSANDRA-9649 so that a temporal clock skew on one node > can "corrupt" other node clocks through Paxos. That wasn't intended and we > should fix that. I'll attach a patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11991) On clock skew, paxos may "corrupt" the node clock
[ https://issues.apache.org/jira/browse/CASSANDRA-11991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332092#comment-15332092 ] sankalp kohli commented on CASSANDRA-11991: --- I can review the patch. We should try to get it in for 2.1.15 if possible > On clock skew, paxos may "corrupt" the node clock > - > > Key: CASSANDRA-11991 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11991 > Project: Cassandra > Issue Type: Bug >Reporter: Sylvain Lebresne >Assignee: Sylvain Lebresne > Fix For: 2.1.x, 2.2.x, 3.0.x > > > W made a mistake in CASSANDRA-9649 so that a temporal clock skew on one node > can "corrupt" other node clocks through Paxos. That wasn't intended and we > should fix that. I'll attach a patch later. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11878) dtest failure in upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_3_x_To_indev_3_x.select_key_in_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332068#comment-15332068
]
Alex Petrov commented on CASSANDRA-11878:
-
Should we maybe add try/catch clauses to make the possible IOException in
{{disableAutoCompaction}} more specific?
Although we don't do it in several other places, so it's on commiter's
discretion.
+1
Tests are triggered here:
|[dtest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-marcuse-11878-3.7-dtest/]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-marcuse-11878-3.7-testall/]|
> dtest failure in
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_3_x_To_indev_3_x.select_key_in_test
> -
>
> Key: CASSANDRA-11878
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11878
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Marcus Eriksson
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/upgrade_tests-all/47/testReport/upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_3_x_To_indev_3_x/select_key_in_test
> Failed on CassCI build upgrade_tests-all #47
> Attached logs for test failure.
> {code}
> ERROR [CompactionExecutor:2] 2016-05-21 23:10:35,678 CassandraDaemon.java:195
> - Exception in thread Thread[CompactionExecutor:2,1,main]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
> down
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
> ~[na:1.8.0_51]
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:165)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
> ~[na:1.8.0_51]
> at
> org.apache.cassandra.db.compaction.CompactionManager.submitBackground(CompactionManager.java:184)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:270)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_51]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11878) dtest failure in upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_3_x_To_indev_3_x.select_key_in_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-11878:
Status: Ready to Commit (was: Patch Available)
> dtest failure in
> upgrade_tests.cql_tests.TestCQLNodes2RF1_Upgrade_current_3_x_To_indev_3_x.select_key_in_test
> -
>
> Key: CASSANDRA-11878
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11878
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean McCarthy
>Assignee: Marcus Eriksson
> Labels: dtest
> Fix For: 3.x
>
> Attachments: node1.log, node1_debug.log, node2.log, node2_debug.log
>
>
> example failure:
> http://cassci.datastax.com/job/upgrade_tests-all/47/testReport/upgrade_tests.cql_tests/TestCQLNodes2RF1_Upgrade_current_3_x_To_indev_3_x/select_key_in_test
> Failed on CassCI build upgrade_tests-all #47
> Attached logs for test failure.
> {code}
> ERROR [CompactionExecutor:2] 2016-05-21 23:10:35,678 CassandraDaemon.java:195
> - Exception in thread Thread[CompactionExecutor:2,1,main]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
> down
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:823)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364)
> ~[na:1.8.0_51]
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:165)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
> ~[na:1.8.0_51]
> at
> org.apache.cassandra.db.compaction.CompactionManager.submitBackground(CompactionManager.java:184)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:270)
> ~[apache-cassandra-3.5.jar:3.5]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_51]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_51]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_51]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11961) Nonfatal NPE in CompactionMetrics
[
https://issues.apache.org/jira/browse/CASSANDRA-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332040#comment-15332040
]
Achal Shah commented on CASSANDRA-11961:
Sounds good, thanks!
> Nonfatal NPE in CompactionMetrics
> -
>
> Key: CASSANDRA-11961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11961
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Achal Shah
>Priority: Minor
> Labels: lhf
>
> Just saw the following NPE on trunk. Means, that {{metaData}} from
> {{CFMetaData metaData = compaction.getCompactionInfo().getCFMetaData();}} is
> {{null}}. A simple {{if (metaData == null) continue;}} should fix this.
> {code}
> Caused by: java.lang.NullPointerException: null
> at
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:78)
> ~[main/:na]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-12010) UserTypesTest# is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-12010:
Status: Patch Available (was: Open)
{{3.0}} is unaffected. Since only unit test was changed, only unit tests CI was
triggered.
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12010-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12010-trunk-testall/]
> UserTypesTest# is failing on trunk
> --
>
> Key: CASSANDRA-12010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12010
> Project: Cassandra
> Issue Type: Test
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> Test failure:
> http://cassci.datastax.com/job/trunk_utest/1445/testReport/org.apache.cassandra.cql3.validation.entities/UserTypesTest/testAlteringUserTypeNestedWithinNonFrozenMap/
> This was caused by the merge after
> [11604|https://issues.apache.org/jira/browse/CASSANDRA-11604] which probably
> coincided with some other change, as this failure did not happen during the
> [test run on the
> branch|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11604-trunk-testall/lastCompletedBuild/testReport/].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-12010) UserTypesTest# is failing on trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332030#comment-15332030
]
Alex Petrov edited comment on CASSANDRA-12010 at 6/15/16 4:30 PM:
--
{{3.0}} is
[unaffected|http://cassci.datastax.com/job/cassandra-3.0_utest/lastCompletedBuild/testReport/]
(also ran tests locally). Since only unit test was changed, only unit tests CI
was triggered:
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12010-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12010-trunk-testall/]
was (Author: ifesdjeen):
{{3.0}} is unaffected. Since only unit test was changed, only unit tests CI was
triggered.
|[trunk|https://github.com/ifesdjeen/cassandra/tree/12010-trunk]|[utest|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-12010-trunk-testall/]
> UserTypesTest# is failing on trunk
> --
>
> Key: CASSANDRA-12010
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12010
> Project: Cassandra
> Issue Type: Test
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>
> Test failure:
> http://cassci.datastax.com/job/trunk_utest/1445/testReport/org.apache.cassandra.cql3.validation.entities/UserTypesTest/testAlteringUserTypeNestedWithinNonFrozenMap/
> This was caused by the merge after
> [11604|https://issues.apache.org/jira/browse/CASSANDRA-11604] which probably
> coincided with some other change, as this failure did not happen during the
> [test run on the
> branch|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11604-trunk-testall/lastCompletedBuild/testReport/].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-12010) UserTypesTest# is failing on trunk
[ https://issues.apache.org/jira/browse/CASSANDRA-12010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov reassigned CASSANDRA-12010: --- Assignee: Alex Petrov > UserTypesTest# is failing on trunk > -- > > Key: CASSANDRA-12010 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12010 > Project: Cassandra > Issue Type: Test >Reporter: Alex Petrov >Assignee: Alex Petrov > > Test failure: > http://cassci.datastax.com/job/trunk_utest/1445/testReport/org.apache.cassandra.cql3.validation.entities/UserTypesTest/testAlteringUserTypeNestedWithinNonFrozenMap/ > This was caused by the merge after > [11604|https://issues.apache.org/jira/browse/CASSANDRA-11604] which probably > coincided with some other change, as this failure did not happen during the > [test run on the > branch|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11604-trunk-testall/lastCompletedBuild/testReport/]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-12010) UserTypesTest# is failing on trunk
Alex Petrov created CASSANDRA-12010: --- Summary: UserTypesTest# is failing on trunk Key: CASSANDRA-12010 URL: https://issues.apache.org/jira/browse/CASSANDRA-12010 Project: Cassandra Issue Type: Test Reporter: Alex Petrov Test failure: http://cassci.datastax.com/job/trunk_utest/1445/testReport/org.apache.cassandra.cql3.validation.entities/UserTypesTest/testAlteringUserTypeNestedWithinNonFrozenMap/ This was caused by the merge after [11604|https://issues.apache.org/jira/browse/CASSANDRA-11604] which probably coincided with some other change, as this failure did not happen during the [test run on the branch|https://cassci.datastax.com/view/Dev/view/ifesdjeen/job/ifesdjeen-11604-trunk-testall/lastCompletedBuild/testReport/]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11940) Look into better default file_cache_size for 2.2
[
https://issues.apache.org/jira/browse/CASSANDRA-11940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331973#comment-15331973
]
Eric Evans commented on CASSANDRA-11940:
In the plot below, nodes cerium, praseodymium (worst. hostname. evar.), and
xenon are under a workload consisting primarily of random reads. At ~11:00,
{{disk_access_mode}} is changed from {{auto}} to {{mmap_index_only}} on cerium
and praseodymium only. When this happens, disk read throughput drops
dramatically, as does the number of major page faults generated by Cassandra
(from ~280/s to ~15/s).
Over the course of this run, I increased {{file_cache_size_in_mb}} from 512, to
768, 1024, and 2048, with no observable difference to disk read throughput, or
the rate of major page faults.
These machines are 12-way (w/ hyperthreading), 16G RAM, and rotational disks.
They have 4G heaps (though raised to 6G for the {{file_cache_size_in_mb: 2048}}
test)
!abnormal disk read throughput.png|width=960!
{noformat}
CREATE TABLE "local_group_wikipedia_T_parsoid_html".data (
"_domain" text,
key text,
rev int,
tid timeuuid,
"_del" timeuuid,
"content-location" text,
"content-sha256" blob,
"content-type" text,
"latestTid" timeuuid,
tags set,
value blob,
PRIMARY KEY (("_domain", key), rev, tid)
) WITH CLUSTERING ORDER BY (rev DESC, tid DESC)
AND bloom_filter_fp_chance = 0.1
AND caching = '{"keys":"ALL", "rows_per_partition":"NONE"}'
AND comment = ''
AND compaction = {'tombstone_threshold': '0.02',
'unchecked_tombstone_compaction': 'true', 'base_time_seconds': '45', 'class':
'org.apache.cassandra.db.compaction.DateTieredCompactionStrategy'}
AND compression = {'chunk_length_kb': '512', 'sstable_compression':
'org.apache.cassandra.io.compress.DeflateCompressor'}
AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 86400
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
AND read_repair_chance = 0.0
AND speculative_retry = '99.0PERCENTILE';
{noformat}
> Look into better default file_cache_size for 2.2
>
>
> Key: CASSANDRA-11940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11940
> Project: Cassandra
> Issue Type: Improvement
>Reporter: T Jake Luciani
>Assignee: T Jake Luciani
> Fix For: 2.2.x
>
> Attachments: abnormal disk read throughput.png
>
>
> CASSANDRA-8464 added support for mmapped decompression where in version <=
> 2.1 the reads were all decompressed in standard heap buffers.
> Since the usage of the file_cache is based solely on the buffer capacity we
> should/can make this much larger in 2.2 when the disk access mode is mmap.
> The downside of this cache being too small is made worse by 8464 since the
> buffers are mmapped/unmapped causing explicit page faults.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8119) More Expressive Consistency Levels
[
https://issues.apache.org/jira/browse/CASSANDRA-8119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331969#comment-15331969
]
Tyler Hobbs commented on CASSANDRA-8119:
bq. this is a niche requirement
I disagree. Good multi-datacenter support is one of Cassandra's most popular
features. Having the ability to control consistency levels per-DC can be very
useful.
bq. Additionally, you can't really encapsulate it in a UDF, as there are more
factors that are CL-dependent. The behaviour of read repair, and speculative
retry, can/should wary a lot depending on a CL
I don't think the CL should be _only_ a UDF. Other options could be specified
through {{WITH}} clauses. For example, something like {{WITH READ REPAIR =
'dclocal'}}. I'm just making that up, but we do have a lot of flexibility here.
bq. Also, while you could stick it to schema, I don't see how it's a fit,
conceptually, and not a (conceptually) dirty hack.
I'm not sure what you mean. Care to elaborate?
> More Expressive Consistency Levels
> --
>
> Key: CASSANDRA-8119
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8119
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Tyler Hobbs
> Fix For: 3.x
>
>
> For some multi-datacenter environments, the current set of consistency levels
> are too restrictive. For example, the following consistency requirements
> cannot be expressed:
> * LOCAL_QUORUM in two specific DCs
> * LOCAL_QUORUM in the local DC plus LOCAL_QUORUM in at least one other DC
> * LOCAL_QUORUM in the local DC plus N remote replicas in any DC
> I propose that we add a new consistency level: CUSTOM. In the v4 (or v5)
> protocol, this would be accompanied by an additional map argument. A map of
> {DC: CL} or a map of {DC: int} is sufficient to cover the first example. If
> we accept a special keys to represent "any datacenter", the second case can
> be handled. A similar technique could be used for "any other nodes".
> I'm not in love with the special keys, so if anybody has ideas for something
> more elegant, feel free to propose them. The main idea is that we want to be
> flexible enough to cover any reasonable consistency or durability
> requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11940) Look into better default file_cache_size for 2.2
[ https://issues.apache.org/jira/browse/CASSANDRA-11940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Evans updated CASSANDRA-11940: --- Attachment: abnormal disk read throughput.png > Look into better default file_cache_size for 2.2 > > > Key: CASSANDRA-11940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11940 > Project: Cassandra > Issue Type: Improvement >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 2.2.x > > Attachments: abnormal disk read throughput.png > > > CASSANDRA-8464 added support for mmapped decompression where in version <= > 2.1 the reads were all decompressed in standard heap buffers. > Since the usage of the file_cache is based solely on the buffer capacity we > should/can make this much larger in 2.2 when the disk access mode is mmap. > The downside of this cache being too small is made worse by 8464 since the > buffers are mmapped/unmapped causing explicit page faults. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)
[ https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331930#comment-15331930 ] Joshua McKenzie commented on CASSANDRA-8844: Rebased to current trunk and pushed. [~blambov] / [~carlyeks]: Either of you have any outstanding unaddressed concerns? > Change Data Capture (CDC) > - > > Key: CASSANDRA-8844 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8844 > Project: Cassandra > Issue Type: New Feature > Components: Coordination, Local Write-Read Paths >Reporter: Tupshin Harper >Assignee: Joshua McKenzie >Priority: Critical > Fix For: 3.x > > > "In databases, change data capture (CDC) is a set of software design patterns > used to determine (and track) the data that has changed so that action can be > taken using the changed data. Also, Change data capture (CDC) is an approach > to data integration that is based on the identification, capture and delivery > of the changes made to enterprise data sources." > -Wikipedia > As Cassandra is increasingly being used as the Source of Record (SoR) for > mission critical data in large enterprises, it is increasingly being called > upon to act as the central hub of traffic and data flow to other systems. In > order to try to address the general need, we (cc [~brianmhess]), propose > implementing a simple data logging mechanism to enable per-table CDC patterns. > h2. The goals: > # Use CQL as the primary ingestion mechanism, in order to leverage its > Consistency Level semantics, and in order to treat it as the single > reliable/durable SoR for the data. > # To provide a mechanism for implementing good and reliable > (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) > continuous semi-realtime feeds of mutations going into a Cassandra cluster. > # To eliminate the developmental and operational burden of users so that they > don't have to do dual writes to other systems. > # For users that are currently doing batch export from a Cassandra system, > give them the opportunity to make that realtime with a minimum of coding. > h2. The mechanism: > We propose a durable logging mechanism that functions similar to a commitlog, > with the following nuances: > - Takes place on every node, not just the coordinator, so RF number of copies > are logged. > - Separate log per table. > - Per-table configuration. Only tables that are specified as CDC_LOG would do > any logging. > - Per DC. We are trying to keep the complexity to a minimum to make this an > easy enhancement, but most likely use cases would prefer to only implement > CDC logging in one (or a subset) of the DCs that are being replicated to > - In the critical path of ConsistencyLevel acknowledgment. Just as with the > commitlog, failure to write to the CDC log should fail that node's write. If > that means the requested consistency level was not met, then clients *should* > experience UnavailableExceptions. > - Be written in a Row-centric manner such that it is easy for consumers to > reconstitute rows atomically. > - Written in a simple format designed to be consumed *directly* by daemons > written in non JVM languages > h2. Nice-to-haves > I strongly suspect that the following features will be asked for, but I also > believe that they can be deferred for a subsequent release, and to guage > actual interest. > - Multiple logs per table. This would make it easy to have multiple > "subscribers" to a single table's changes. A workaround would be to create a > forking daemon listener, but that's not a great answer. > - Log filtering. Being able to apply filters, including UDF-based filters > would make Casandra a much more versatile feeder into other systems, and > again, reduce complexity that would otherwise need to be built into the > daemons. > h2. Format and Consumption > - Cassandra would only write to the CDC log, and never delete from it. > - Cleaning up consumed logfiles would be the client daemon's responibility > - Logfile size should probably be configurable. > - Logfiles should be named with a predictable naming schema, making it > triivial to process them in order. > - Daemons should be able to checkpoint their work, and resume from where they > left off. This means they would have to leave some file artifact in the CDC > log's directory. > - A sophisticated daemon should be able to be written that could > -- Catch up, in written-order, even when it is multiple logfiles behind in > processing > -- Be able to continuously "tail" the most recent logfile and get > low-latency(ms?) access to the data as it is written. > h2. Alternate approach > In order to make consuming a change log easy and efficient to do with low > latency, the following could supplement the approach ou
[06/20] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbefa854
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbefa854
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbefa854
Branch: refs/heads/cassandra-3.0
Commit: dbefa854b403824334922dd35dc5e18ff1be51a9
Parents: c2566d1 615bf37
Author: Carl Yeksigian
Authored: Wed Jun 15 10:39:09 2016 -0400
Committer: Carl Yeksigian
Committed: Wed Jun 15 10:39:09 2016 -0400
--
.../cql3/selection/SelectionColumnMapping.java | 20 ++
.../cql3/selection/SelectionColumns.java| 20 ++
.../db/lifecycle/SSTableIntervalTree.java | 22 +++-
.../cassandra/hadoop/pig/StorageHelper.java | 20 ++
.../cassandra/locator/PendingRangeMaps.java | 20 ++
.../cassandra/repair/RepairParallelism.java | 20 ++
.../apache/cassandra/utils/CRC32Factory.java| 20 ++
.../apache/cassandra/utils/OverlapIterator.java | 22 +++-
.../utils/RMIServerSocketFactoryImpl.java | 20 ++
.../org/apache/cassandra/utils/SyncUtil.java| 20 ++
.../apache/cassandra/utils/concurrent/Ref.java | 20 ++
.../apache/cassandra/utils/concurrent/Refs.java | 20 ++
.../io/compress/CompressorPerformance.java | 20 ++
.../test/microbench/PendingRangesBench.java | 20 ++
.../selection/SelectionColumnMappingTest.java | 20 ++
.../validation/operations/SelectLimitTest.java | 20 ++
.../SelectOrderedPartitionerTest.java | 20 ++
.../gms/ArrayBackedBoundedStatsTest.java| 20 ++
.../cassandra/io/RandomAccessReaderTest.java| 20 ++
.../io/util/BufferedDataOutputStreamTest.java | 20 ++
.../io/util/NIODataInputStreamTest.java | 20 ++
.../cassandra/locator/PendingRangeMapsTest.java | 20 ++
.../cassandra/net/MessagingServiceTest.java | 20 ++
.../apache/cassandra/utils/TopKSamplerTest.java | 20 ++
24 files changed, 482 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbefa854/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
--
diff --cc
src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
index 33ef0af,000..8636f19
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
+++ b/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
@@@ -1,132 -1,0 +1,152 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.cql3.selection;
+
+import java.util.*;
+
+import com.google.common.base.Function;
+import com.google.common.base.Joiner;
+import com.google.common.base.Objects;
+import com.google.common.collect.*;
+
+import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.cql3.ColumnSpecification;
+
+/**
+ * Separately maintains the ColumnSpecifications and their mappings to
underlying
+ * columns as we may receive null mappings. This occurs where a query result
+ * includes a column specification which does not map to any particular real
+ * column, e.g. COUNT queries or where no-arg functions like now() are used
+ */
+public class SelectionColumnMapping implements SelectionColumns
+{
+private final ArrayList columnSpecifications;
+private final HashMultimap
columnMappings;
+
+private SelectionColumnMapping()
+{
+this.columnSpecifications = new ArrayList<>();
+this.columnMappings = HashMultimap.create();
+}
+
+protected static SelectionColumnMapping newMapping()
[20/20] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/693e52d6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/693e52d6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/693e52d6 Branch: refs/heads/trunk Commit: 693e52d66a3e4c12e2b9c8fcbc770d5d1a0cad93 Parents: 4210a25 05bacc7 Author: Carl Yeksigian Authored: Wed Jun 15 10:57:30 2016 -0400 Committer: Carl Yeksigian Committed: Wed Jun 15 10:57:30 2016 -0400 -- .../org/apache/cassandra/cache/CacheSize.java | 22 +++- .../org/apache/cassandra/cache/ChunkCache.java | 20 ++ .../cql3/selection/SelectionColumnMapping.java | 20 ++ .../cql3/selection/SelectionColumns.java| 20 ++ .../apache/cassandra/db/ClusteringBound.java| 22 +++- .../cassandra/db/ClusteringBoundOrBoundary.java | 20 ++ .../apache/cassandra/db/ClusteringBoundary.java | 22 +++- .../EncryptedFileSegmentInputStream.java| 20 ++ .../db/lifecycle/LogAwareFileLister.java| 20 ++ .../apache/cassandra/db/lifecycle/LogFile.java | 20 ++ .../cassandra/db/lifecycle/LogRecord.java | 20 ++ .../db/lifecycle/SSTableIntervalTree.java | 20 ++ .../cassandra/db/lifecycle/SSTableSet.java | 22 +++- .../UnfilteredRowIteratorWithLowerBound.java| 20 ++ .../cassandra/db/transform/BaseIterator.java| 20 ++ .../cassandra/db/transform/BasePartitions.java | 20 ++ .../apache/cassandra/db/transform/BaseRows.java | 20 ++ .../apache/cassandra/db/transform/Filter.java | 20 ++ .../db/transform/FilteredPartitions.java| 20 ++ .../cassandra/db/transform/FilteredRows.java| 20 ++ .../cassandra/db/transform/MoreContents.java| 20 ++ .../cassandra/db/transform/MorePartitions.java | 20 ++ .../apache/cassandra/db/transform/MoreRows.java | 20 ++ .../apache/cassandra/db/transform/Stack.java| 20 ++ .../db/transform/StoppingTransformation.java| 20 ++ .../cassandra/db/transform/Transformation.java | 20 ++ .../db/transform/UnfilteredPartitions.java | 20 ++ .../cassandra/db/transform/UnfilteredRows.java | 20 ++ src/java/org/apache/cassandra/index/Index.java | 20 ++ .../apache/cassandra/index/IndexRegistry.java | 20 ++ .../index/internal/CassandraIndex.java | 20 ++ .../index/internal/CassandraIndexSearcher.java | 20 ++ .../cassandra/index/internal/IndexEntry.java| 20 ++ .../index/internal/keys/KeysIndex.java | 20 ++ .../cassandra/index/sasi/SASIIndexBuilder.java | 20 ++ .../io/util/BufferManagingRebufferer.java | 20 ++ .../cassandra/io/util/LimitingRebufferer.java | 20 ++ .../cassandra/io/util/MmapRebufferer.java | 22 +++- .../cassandra/locator/PendingRangeMaps.java | 20 ++ .../cassandra/repair/RepairParallelism.java | 20 ++ .../cassandra/tools/BulkLoadException.java | 20 ++ .../apache/cassandra/tools/JsonTransformer.java | 22 +++- .../apache/cassandra/tools/LoaderOptions.java | 22 +++- .../apache/cassandra/utils/OverlapIterator.java | 22 +++- .../utils/RMIServerSocketFactoryImpl.java | 20 ++ .../org/apache/cassandra/utils/SyncUtil.java| 20 ++ .../apache/cassandra/utils/concurrent/Ref.java | 20 ++ .../apache/cassandra/utils/concurrent/Refs.java | 20 ++ .../cassandra/utils/memory/EnsureOnHeap.java| 20 ++ .../io/compress/CompressorPerformance.java | 20 ++ .../test/microbench/PendingRangesBench.java | 20 ++ .../cassandra/cql3/IndexQueryPagingTest.java| 20 ++ .../selection/SelectionColumnMappingTest.java | 20 ++ .../statements/PropertyDefinitionsTest.java | 20 ++ .../miscellaneous/SSTablesIteratedTest.java | 22 +++- .../validation/operations/SelectLimitTest.java | 20 ++ .../SelectOrderedPartitionerTest.java | 20 ++ .../db/SinglePartitionSliceCommandTest.java | 20 ++ .../commitlog/CommitLogSegmentManagerTest.java | 22 +++- .../db/marshal/AbstractCompositeTypeTest.java |
[17/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
--
diff --cc src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
index 98640ae,000..f000fcf
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
+++ b/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
@@@ -1,40 -1,0 +1,60 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.db.transform;
+
+import org.apache.cassandra.db.DeletionTime;
+import org.apache.cassandra.db.rows.EncodingStats;
+import org.apache.cassandra.db.rows.Unfiltered;
+import org.apache.cassandra.db.rows.UnfilteredRowIterator;
+
+final class UnfilteredRows extends BaseRows implements UnfilteredRowIterator
+{
+private DeletionTime partitionLevelDeletion;
+
+public UnfilteredRows(UnfilteredRowIterator input)
+{
+super(input);
+partitionLevelDeletion = input.partitionLevelDeletion();
+}
+
+@Override
+void add(Transformation add)
+{
+super.add(add);
+partitionLevelDeletion = add.applyToDeletion(partitionLevelDeletion);
+}
+
+public DeletionTime partitionLevelDeletion()
+{
+return partitionLevelDeletion;
+}
+
+public EncodingStats stats()
+{
+return input.stats();
+}
+
+@Override
+public boolean isEmpty()
+{
+return staticRow().isEmpty() && partitionLevelDeletion().isLive() &&
!hasNext();
+}
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/index/Index.java
--
diff --cc src/java/org/apache/cassandra/index/Index.java
index ab6665d,000..469ef07
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/index/Index.java
+++ b/src/java/org/apache/cassandra/index/Index.java
@@@ -1,452 -1,0 +1,472 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.index;
+
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.function.BiFunction;
+
+import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.cql3.Operator;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.filter.RowFilter;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.partitions.PartitionIterator;
+import org.apache.cassandra.db.partitions.PartitionUpdate;
+import org.apache.cassandra.db.partitions.UnfilteredPartitionIterator;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.index.transactions.IndexTransaction;
+import org.apache.cassandra.schema.IndexMetadata;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+
+/**
+ * Consisting of a top level Index interface and two sub-interfaces which
handle read and write operations,
+ * Searcher and Indexer respectively, this defines a secondary index
implementation.
+ * Instantiation is done via reflection and implementations must provide a
constructor which takes the base
+ * table's ColumnFamilyStore and the IndexMetadata which defines the Index
[13/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
--
diff --cc src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
index 98640ae,000..f000fcf
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
+++ b/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
@@@ -1,40 -1,0 +1,60 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.db.transform;
+
+import org.apache.cassandra.db.DeletionTime;
+import org.apache.cassandra.db.rows.EncodingStats;
+import org.apache.cassandra.db.rows.Unfiltered;
+import org.apache.cassandra.db.rows.UnfilteredRowIterator;
+
+final class UnfilteredRows extends BaseRows implements UnfilteredRowIterator
+{
+private DeletionTime partitionLevelDeletion;
+
+public UnfilteredRows(UnfilteredRowIterator input)
+{
+super(input);
+partitionLevelDeletion = input.partitionLevelDeletion();
+}
+
+@Override
+void add(Transformation add)
+{
+super.add(add);
+partitionLevelDeletion = add.applyToDeletion(partitionLevelDeletion);
+}
+
+public DeletionTime partitionLevelDeletion()
+{
+return partitionLevelDeletion;
+}
+
+public EncodingStats stats()
+{
+return input.stats();
+}
+
+@Override
+public boolean isEmpty()
+{
+return staticRow().isEmpty() && partitionLevelDeletion().isLive() &&
!hasNext();
+}
+}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/index/Index.java
--
diff --cc src/java/org/apache/cassandra/index/Index.java
index ab6665d,000..469ef07
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/index/Index.java
+++ b/src/java/org/apache/cassandra/index/Index.java
@@@ -1,452 -1,0 +1,472 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.index;
+
+import java.util.Optional;
+import java.util.concurrent.Callable;
+import java.util.function.BiFunction;
+
+import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.cql3.Operator;
+import org.apache.cassandra.db.*;
+import org.apache.cassandra.db.filter.RowFilter;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.partitions.PartitionIterator;
+import org.apache.cassandra.db.partitions.PartitionUpdate;
+import org.apache.cassandra.db.partitions.UnfilteredPartitionIterator;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.index.transactions.IndexTransaction;
+import org.apache.cassandra.schema.IndexMetadata;
+import org.apache.cassandra.utils.concurrent.OpOrder;
+
+/**
+ * Consisting of a top level Index interface and two sub-interfaces which
handle read and write operations,
+ * Searcher and Indexer respectively, this defines a secondary index
implementation.
+ * Instantiation is done via reflection and implementations must provide a
constructor which takes the base
+ * table's ColumnFamilyStore and the IndexMetadata which defines the Index
[jira] [Assigned] (CASSANDRA-11940) Look into better default file_cache_size for 2.2
[ https://issues.apache.org/jira/browse/CASSANDRA-11940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] T Jake Luciani reassigned CASSANDRA-11940: -- Assignee: T Jake Luciani > Look into better default file_cache_size for 2.2 > > > Key: CASSANDRA-11940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11940 > Project: Cassandra > Issue Type: Improvement >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 2.2.x > > > CASSANDRA-8464 added support for mmapped decompression where in version <= > 2.1 the reads were all decompressed in standard heap buffers. > Since the usage of the file_cache is based solely on the buffer capacity we > should/can make this much larger in 2.2 when the disk access mode is mmap. > The downside of this cache being too small is made worse by 8464 since the > buffers are mmapped/unmapped causing explicit page faults. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[05/20] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbefa854/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
--
diff --cc test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
index 71fab61,000..8c6cc90
mode 100644,00..100644
--- a/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
@@@ -1,231 -1,0 +1,251 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.io;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.UUID;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+
+import static org.junit.Assert.*;
+import org.apache.cassandra.io.util.ChannelProxy;
+import org.apache.cassandra.io.util.FileMark;
+import org.apache.cassandra.io.util.RandomAccessReader;
+import org.apache.cassandra.io.util.SequentialWriter;
+
+public class RandomAccessReaderTest
+{
+@Test
+public void testReadFully() throws IOException
+{
+final File f = File.createTempFile("testReadFully", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+
+SequentialWriter writer = SequentialWriter.open(f);
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(f.getAbsolutePath(), reader.getPath());
+assertEquals(expected.length(), reader.length());
+
+byte[] b = new byte[expected.length()];
+reader.readFully(b);
+assertEquals(expected, new String(b));
+
+assertTrue(reader.isEOF());
+assertEquals(0, reader.bytesRemaining());
+
+reader.close();
+channel.close();
+}
+
+@Test
+public void testReadBytes() throws IOException
+{
+File f = File.createTempFile("testReadBytes", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+
+SequentialWriter writer = SequentialWriter.open(f);
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(f.getAbsolutePath(), reader.getPath());
+assertEquals(expected.length(), reader.length());
+
+ByteBuffer b = reader.readBytes(expected.length());
+assertEquals(expected, new String(b.array(),
Charset.forName("UTF-8")));
+
+assertTrue(reader.isEOF());
+assertEquals(0, reader.bytesRemaining());
+
+reader.close();
+channel.close();
+}
+
+@Test
+public void testReset() throws IOException
+{
+File f = File.createTempFile("testMark", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+final int numIterations = 10;
+
+SequentialWriter writer = SequentialWriter.open(f);
+for (int i = 0; i < numIterations; i++)
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(expected.length() * numIterations, reader.length());
+
+ByteBuffer b = reader.readBytes(expected.length());
+assertEquals(expected, new String(b.array(),
Charset.forName("UTF-8")));
+
+assertFalse(reader.isEOF());
+assertEquals((numIterations - 1) * expected.length(),
reader.bytesRemaining());
+
+FileMark mark = reader.mark();
+assertEquals(0, reader.bytesPastMark());
+assertEquals(0, reader.bytesPastMark(mark));
+
+for (int i = 0; i <
[12/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/tools/JsonTransformer.java
--
diff --cc src/java/org/apache/cassandra/tools/JsonTransformer.java
index 364070e,000..3deed96
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/tools/JsonTransformer.java
+++ b/src/java/org/apache/cassandra/tools/JsonTransformer.java
@@@ -1,536 -1,0 +1,556 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.tools;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.io.OutputStreamWriter;
+import java.nio.ByteBuffer;
+import java.time.Instant;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Stream;
+
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.db.ClusteringPrefix;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.DeletionTime;
+import org.apache.cassandra.db.LivenessInfo;
+import org.apache.cassandra.db.RangeTombstone;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CollectionType;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.rows.Cell;
+import org.apache.cassandra.db.rows.ColumnData;
+import org.apache.cassandra.db.rows.ComplexColumnData;
+import org.apache.cassandra.db.rows.RangeTombstoneBoundMarker;
+import org.apache.cassandra.db.rows.RangeTombstoneBoundaryMarker;
+import org.apache.cassandra.db.rows.RangeTombstoneMarker;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.db.rows.Unfiltered;
+import org.apache.cassandra.db.rows.UnfilteredRowIterator;
+import org.apache.cassandra.io.sstable.ISSTableScanner;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.codehaus.jackson.JsonFactory;
+import org.codehaus.jackson.JsonGenerator;
+import org.codehaus.jackson.impl.Indenter;
+import org.codehaus.jackson.util.DefaultPrettyPrinter;
+import org.codehaus.jackson.util.DefaultPrettyPrinter.NopIndenter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public final class JsonTransformer
+{
+
+private static final Logger logger =
LoggerFactory.getLogger(JsonTransformer.class);
+
+private static final JsonFactory jsonFactory = new JsonFactory();
+
+private final JsonGenerator json;
+
+private final CompactIndenter objectIndenter = new CompactIndenter();
+
+private final CompactIndenter arrayIndenter = new CompactIndenter();
+
+private final CFMetaData metadata;
+
+private final ISSTableScanner currentScanner;
+
+private boolean rawTime = false;
+
+private long currentPosition = 0;
+
+private JsonTransformer(JsonGenerator json, ISSTableScanner
currentScanner, boolean rawTime, CFMetaData metadata)
+{
+this.json = json;
+this.metadata = metadata;
+this.currentScanner = currentScanner;
+this.rawTime = rawTime;
+
+DefaultPrettyPrinter prettyPrinter = new DefaultPrettyPrinter();
+prettyPrinter.indentObjectsWith(objectIndenter);
+prettyPrinter.indentArraysWith(arrayIndenter);
+json.setPrettyPrinter(prettyPrinter);
+}
+
+public static void toJson(ISSTableScanner currentScanner,
Stream partitions, boolean rawTime, CFMetaData metadata,
OutputStream out)
+throws IOException
+{
+try (JsonGenerator json = jsonFactory.createJsonGenerator(new
OutputStreamWriter(out, "UTF-8")))
+{
+JsonTransformer transformer = new JsonTransformer(json,
currentScanner, rawTime, metadata);
+json.writeStartArray();
+partitions.forEach(transformer::serializePartition);
+json.writeEndArray();
+}
+}
+
+public static void keysToJson(ISSTableScanner currentScanner,
Stream keys, boolean rawTime, CFMetaData metadata, OutputStream
out) throws IOException
+{
+try (JsonGenerator json = jsonFactory.createJsonGenerator(
[15/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/test/unit/org/apache/cassandra/index/CustomIndexTest.java
--
diff --cc test/unit/org/apache/cassandra/index/CustomIndexTest.java
index fa6700b,000..f02823c
mode 100644,00..100644
--- a/test/unit/org/apache/cassandra/index/CustomIndexTest.java
+++ b/test/unit/org/apache/cassandra/index/CustomIndexTest.java
@@@ -1,806 -1,0 +1,826 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.index;
+
+import java.util.*;
+import java.util.concurrent.Callable;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Collectors;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import org.junit.Test;
+
+import com.datastax.driver.core.exceptions.QueryValidationException;
+import org.apache.cassandra.Util;
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CQLTester;
+import org.apache.cassandra.cql3.ColumnIdentifier;
+import org.apache.cassandra.cql3.restrictions.IndexRestrictions;
+import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
+import org.apache.cassandra.cql3.statements.IndexTarget;
+import org.apache.cassandra.cql3.statements.ModificationStatement;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.ReadCommand;
+import org.apache.cassandra.db.ReadOrderGroup;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.Int32Type;
+import org.apache.cassandra.db.marshal.UTF8Type;
+import org.apache.cassandra.db.partitions.UnfilteredPartitionIterator;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.schema.IndexMetadata;
+import org.apache.cassandra.schema.Indexes;
+import org.apache.cassandra.transport.Server;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.Util.throwAssert;
+import static
org.apache.cassandra.cql3.statements.IndexTarget.CUSTOM_INDEX_OPTION_NAME;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+public class CustomIndexTest extends CQLTester
+{
+@Test
+public void testInsertsOnCfsBackedIndex() throws Throwable
+{
+// test to ensure that we don't deadlock when flushing CFS backed
custom indexers
+// see CASSANDRA-10181
+createTable("CREATE TABLE %s (a int, b int, c int, d int, PRIMARY KEY
(a, b))");
+createIndex("CREATE CUSTOM INDEX myindex ON %s(c) USING
'org.apache.cassandra.index.internal.CustomCassandraIndex'");
+
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 0,
2);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 0,
1);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 2, 0,
0);
+}
+
+@Test
+public void testTruncateWithNonCfsCustomIndex() throws Throwable
+{
+// deadlocks and times out the test in the face of the synchronisation
+// issues described in the comments on CASSANDRA-9669
+createTable("CREATE TABLE %s (a int, b int, c int, PRIMARY KEY (a))");
+createIndex("CREATE CUSTOM INDEX b_index ON %s(b) USING
'org.apache.cassandra.index.StubIndex'");
+execute("INSERT INTO %s (a, b, c) VALUES (?, ?, ?)", 0, 1, 2);
+getCurrentColumnFamilyStore().truncateBlocking();
+}
+
+@Test
+public void indexControlsIfIncludedInBuildOnNewSSTables() throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, PRIMARY KEY (a))");
+String toInclude = "include";
+String toExclude = "exclude";
+createIndex(String.format("CREATE CUSTOM INDEX %s ON %%s(b) USING
'%s'",
+ toInclude,
IndexIncludedInBuild.class.getName()));
+createIndex(String.format("CREATE CUSTOM INDEX %s ON %%s(b) US
[18/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05bacc75 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05bacc75 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05bacc75 Branch: refs/heads/trunk Commit: 05bacc756e983e2850af7fbd265951983e66f4a0 Parents: 70ee4ed dbefa85 Author: Carl Yeksigian Authored: Wed Jun 15 10:52:55 2016 -0400 Committer: Carl Yeksigian Committed: Wed Jun 15 10:52:55 2016 -0400 -- .../cql3/selection/SelectionColumnMapping.java | 20 ++ .../cql3/selection/SelectionColumns.java| 20 ++ .../db/lifecycle/LogAwareFileLister.java| 20 ++ .../apache/cassandra/db/lifecycle/LogFile.java | 20 ++ .../cassandra/db/lifecycle/LogRecord.java | 20 ++ .../db/lifecycle/SSTableIntervalTree.java | 20 ++ .../cassandra/db/lifecycle/SSTableSet.java | 22 +++- .../cassandra/db/transform/BaseIterator.java| 20 ++ .../cassandra/db/transform/BasePartitions.java | 20 ++ .../apache/cassandra/db/transform/BaseRows.java | 20 ++ .../apache/cassandra/db/transform/Filter.java | 20 ++ .../db/transform/FilteredPartitions.java| 20 ++ .../cassandra/db/transform/FilteredRows.java| 20 ++ .../cassandra/db/transform/MoreContents.java| 20 ++ .../cassandra/db/transform/MorePartitions.java | 20 ++ .../apache/cassandra/db/transform/MoreRows.java | 20 ++ .../apache/cassandra/db/transform/Stack.java| 20 ++ .../db/transform/StoppingTransformation.java| 20 ++ .../cassandra/db/transform/Transformation.java | 20 ++ .../db/transform/UnfilteredPartitions.java | 20 ++ .../cassandra/db/transform/UnfilteredRows.java | 20 ++ src/java/org/apache/cassandra/index/Index.java | 20 ++ .../apache/cassandra/index/IndexRegistry.java | 20 ++ .../index/internal/CassandraIndex.java | 20 ++ .../index/internal/CassandraIndexSearcher.java | 20 ++ .../cassandra/index/internal/IndexEntry.java| 20 ++ .../index/internal/keys/KeysIndex.java | 20 ++ .../cassandra/locator/PendingRangeMaps.java | 20 ++ .../cassandra/repair/RepairParallelism.java | 20 ++ .../apache/cassandra/tools/JsonTransformer.java | 22 +++- .../apache/cassandra/utils/OverlapIterator.java | 22 +++- .../utils/RMIServerSocketFactoryImpl.java | 20 ++ .../org/apache/cassandra/utils/SyncUtil.java| 20 ++ .../apache/cassandra/utils/concurrent/Ref.java | 20 ++ .../apache/cassandra/utils/concurrent/Refs.java | 20 ++ .../io/compress/CompressorPerformance.java | 20 ++ .../test/microbench/PendingRangesBench.java | 20 ++ .../cassandra/cql3/IndexQueryPagingTest.java| 20 ++ .../selection/SelectionColumnMappingTest.java | 20 ++ .../validation/operations/SelectLimitTest.java | 20 ++ .../SelectOrderedPartitionerTest.java | 20 ++ .../db/SinglePartitionSliceCommandTest.java | 20 ++ .../commitlog/CommitLogSegmentManagerTest.java | 22 +++- .../rows/RowAndDeletionMergeIteratorTest.java | 20 ++ .../gms/ArrayBackedBoundedStatsTest.java| 20 ++ .../apache/cassandra/index/CustomIndexTest.java | 20 ++ .../index/internal/CustomCassandraIndex.java| 20 ++ .../io/util/BufferedDataOutputStreamTest.java | 20 ++ .../io/util/NIODataInputStreamTest.java | 20 ++ .../io/util/RandomAccessReaderTest.java | 20 ++ .../cassandra/locator/PendingRangeMapsTest.java | 20 ++ .../cassandra/net/MessagingServiceTest.java | 20 ++ .../service/RMIServerSocketFactoryImplTest.java | 20 ++ .../apache/cassandra/utils/TopKSamplerTest.java | 20 ++ 54 files changed, 1084 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/c
[19/20] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
http://git-wip-us.apache.org/repos/asf/cassandra/blob/693e52d6/src/java/org/apache/cassandra/tools/LoaderOptions.java
--
diff --cc src/java/org/apache/cassandra/tools/LoaderOptions.java
index 91b3e8b,000..28d7bce
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/tools/LoaderOptions.java
+++ b/src/java/org/apache/cassandra/tools/LoaderOptions.java
@@@ -1,543 -1,0 +1,563 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.tools;
+
+import java.io.File;
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.net.*;
+import java.util.HashSet;
+import java.util.Set;
+
+import org.apache.cassandra.config.*;
+import org.apache.cassandra.exceptions.ConfigurationException;
+import org.apache.cassandra.tools.BulkLoader.CmdLineOptions;
+
+import com.datastax.driver.core.AuthProvider;
+import com.datastax.driver.core.PlainTextAuthProvider;
+import org.apache.commons.cli.*;
+
+public class LoaderOptions
+{
+
+public static final String HELP_OPTION = "help";
+public static final String VERBOSE_OPTION = "verbose";
+public static final String NOPROGRESS_OPTION = "no-progress";
+public static final String NATIVE_PORT_OPTION = "port";
+public static final String USER_OPTION = "username";
+public static final String PASSWD_OPTION = "password";
+public static final String AUTH_PROVIDER_OPTION = "auth-provider";
+public static final String INITIAL_HOST_ADDRESS_OPTION = "nodes";
+public static final String IGNORE_NODES_OPTION = "ignore";
+public static final String CONNECTIONS_PER_HOST = "connections-per-host";
+public static final String CONFIG_PATH = "conf-path";
+public static final String THROTTLE_MBITS = "throttle";
+public static final String INTER_DC_THROTTLE_MBITS = "inter-dc-throttle";
+public static final String TOOL_NAME = "sstableloader";
+
+/* client encryption options */
+public static final String SSL_TRUSTSTORE = "truststore";
+public static final String SSL_TRUSTSTORE_PW = "truststore-password";
+public static final String SSL_KEYSTORE = "keystore";
+public static final String SSL_KEYSTORE_PW = "keystore-password";
+public static final String SSL_PROTOCOL = "ssl-protocol";
+public static final String SSL_ALGORITHM = "ssl-alg";
+public static final String SSL_STORE_TYPE = "store-type";
+public static final String SSL_CIPHER_SUITES = "ssl-ciphers";
+
+public final File directory;
+public final boolean debug;
+public final boolean verbose;
+public final boolean noProgress;
+public final int nativePort;
+public final String user;
+public final String passwd;
+public final AuthProvider authProvider;
+public final int throttle;
+public final int interDcThrottle;
+public final int storagePort;
+public final int sslStoragePort;
+public final EncryptionOptions.ClientEncryptionOptions clientEncOptions;
+public final int connectionsPerHost;
+public final EncryptionOptions.ServerEncryptionOptions serverEncOptions;
+public final Set hosts;
+public final Set ignores = new HashSet<>();
+
+LoaderOptions(Builder builder)
+{
+directory = builder.directory;
+debug = builder.debug;
+verbose = builder.verbose;
+noProgress = builder.noProgress;
+nativePort = builder.nativePort;
+user = builder.user;
+passwd = builder.passwd;
+authProvider = builder.authProvider;
+throttle = builder.throttle;
+interDcThrottle = builder.interDcThrottle;
+storagePort = builder.storagePort;
+sslStoragePort = builder.sslStoragePort;
+clientEncOptions = builder.clientEncOptions;
+connectionsPerHost = builder.connectionsPerHost;
+serverEncOptions = builder.serverEncOptions;
+hosts = builder.hosts;
+}
+
+static class Builder
+{
+File directory;
+boolean debug;
+boolean verbose;
+boolea
[16/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/tools/JsonTransformer.java
--
diff --cc src/java/org/apache/cassandra/tools/JsonTransformer.java
index 364070e,000..3deed96
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/tools/JsonTransformer.java
+++ b/src/java/org/apache/cassandra/tools/JsonTransformer.java
@@@ -1,536 -1,0 +1,556 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.tools;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.io.OutputStreamWriter;
+import java.nio.ByteBuffer;
+import java.time.Instant;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Stream;
+
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.db.ClusteringPrefix;
+import org.apache.cassandra.db.DecoratedKey;
+import org.apache.cassandra.db.DeletionTime;
+import org.apache.cassandra.db.LivenessInfo;
+import org.apache.cassandra.db.RangeTombstone;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.CollectionType;
+import org.apache.cassandra.db.marshal.CompositeType;
+import org.apache.cassandra.db.rows.Cell;
+import org.apache.cassandra.db.rows.ColumnData;
+import org.apache.cassandra.db.rows.ComplexColumnData;
+import org.apache.cassandra.db.rows.RangeTombstoneBoundMarker;
+import org.apache.cassandra.db.rows.RangeTombstoneBoundaryMarker;
+import org.apache.cassandra.db.rows.RangeTombstoneMarker;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.db.rows.Unfiltered;
+import org.apache.cassandra.db.rows.UnfilteredRowIterator;
+import org.apache.cassandra.io.sstable.ISSTableScanner;
+import org.apache.cassandra.utils.ByteBufferUtil;
+import org.codehaus.jackson.JsonFactory;
+import org.codehaus.jackson.JsonGenerator;
+import org.codehaus.jackson.impl.Indenter;
+import org.codehaus.jackson.util.DefaultPrettyPrinter;
+import org.codehaus.jackson.util.DefaultPrettyPrinter.NopIndenter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+public final class JsonTransformer
+{
+
+private static final Logger logger =
LoggerFactory.getLogger(JsonTransformer.class);
+
+private static final JsonFactory jsonFactory = new JsonFactory();
+
+private final JsonGenerator json;
+
+private final CompactIndenter objectIndenter = new CompactIndenter();
+
+private final CompactIndenter arrayIndenter = new CompactIndenter();
+
+private final CFMetaData metadata;
+
+private final ISSTableScanner currentScanner;
+
+private boolean rawTime = false;
+
+private long currentPosition = 0;
+
+private JsonTransformer(JsonGenerator json, ISSTableScanner
currentScanner, boolean rawTime, CFMetaData metadata)
+{
+this.json = json;
+this.metadata = metadata;
+this.currentScanner = currentScanner;
+this.rawTime = rawTime;
+
+DefaultPrettyPrinter prettyPrinter = new DefaultPrettyPrinter();
+prettyPrinter.indentObjectsWith(objectIndenter);
+prettyPrinter.indentArraysWith(arrayIndenter);
+json.setPrettyPrinter(prettyPrinter);
+}
+
+public static void toJson(ISSTableScanner currentScanner,
Stream partitions, boolean rawTime, CFMetaData metadata,
OutputStream out)
+throws IOException
+{
+try (JsonGenerator json = jsonFactory.createJsonGenerator(new
OutputStreamWriter(out, "UTF-8")))
+{
+JsonTransformer transformer = new JsonTransformer(json,
currentScanner, rawTime, metadata);
+json.writeStartArray();
+partitions.forEach(transformer::serializePartition);
+json.writeEndArray();
+}
+}
+
+public static void keysToJson(ISSTableScanner currentScanner,
Stream keys, boolean rawTime, CFMetaData metadata, OutputStream
out) throws IOException
+{
+try (JsonGenerator json = jsonFactory.createJsonGenerator(
[02/20] cassandra git commit: ninja fix license headers
ninja fix license headers Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/615bf372 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/615bf372 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/615bf372 Branch: refs/heads/cassandra-2.2 Commit: 615bf3721e26db5dc1350d2a74d28cf907cffa53 Parents: 2f74831 Author: Carl Yeksigian Authored: Wed Jun 15 10:29:27 2016 -0400 Committer: Carl Yeksigian Committed: Wed Jun 15 10:29:27 2016 -0400 -- .../cql3/statements/SelectionColumnMapping.java | 20 .../cql3/statements/SelectionColumns.java | 20 .../cassandra/repair/RepairParallelism.java | 20 .../utils/RMIServerSocketFactoryImpl.java | 20 .../apache/cassandra/utils/concurrent/Ref.java | 20 .../apache/cassandra/utils/concurrent/Refs.java | 20 .../statements/SelectionColumnMappingTest.java | 20 .../validation/operations/SelectLimitTest.java | 20 .../SelectOrderedPartitionerTest.java | 20 .../gms/ArrayBackedBoundedStatsTest.java| 20 .../cassandra/net/MessagingServiceTest.java | 20 .../apache/cassandra/utils/TopKSamplerTest.java | 20 12 files changed, 240 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java index aba2631..39bd12c 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.*; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java index 5b18eff..fa6e214 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.List; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/repair/RepairParallelism.java -- diff --git a/src/java/org/apache/cassandra/repair/RepairParallelism.java b/src/java/org/apache/cassandra/repair/RepairParallelism.java index 12c22ca..0afde28 100644 --- a/src/java/org/apache/cassandra/repair/RepairParallelism.java +++ b/src/ja
[08/20] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbefa854
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbefa854
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbefa854
Branch: refs/heads/cassandra-2.2
Commit: dbefa854b403824334922dd35dc5e18ff1be51a9
Parents: c2566d1 615bf37
Author: Carl Yeksigian
Authored: Wed Jun 15 10:39:09 2016 -0400
Committer: Carl Yeksigian
Committed: Wed Jun 15 10:39:09 2016 -0400
--
.../cql3/selection/SelectionColumnMapping.java | 20 ++
.../cql3/selection/SelectionColumns.java| 20 ++
.../db/lifecycle/SSTableIntervalTree.java | 22 +++-
.../cassandra/hadoop/pig/StorageHelper.java | 20 ++
.../cassandra/locator/PendingRangeMaps.java | 20 ++
.../cassandra/repair/RepairParallelism.java | 20 ++
.../apache/cassandra/utils/CRC32Factory.java| 20 ++
.../apache/cassandra/utils/OverlapIterator.java | 22 +++-
.../utils/RMIServerSocketFactoryImpl.java | 20 ++
.../org/apache/cassandra/utils/SyncUtil.java| 20 ++
.../apache/cassandra/utils/concurrent/Ref.java | 20 ++
.../apache/cassandra/utils/concurrent/Refs.java | 20 ++
.../io/compress/CompressorPerformance.java | 20 ++
.../test/microbench/PendingRangesBench.java | 20 ++
.../selection/SelectionColumnMappingTest.java | 20 ++
.../validation/operations/SelectLimitTest.java | 20 ++
.../SelectOrderedPartitionerTest.java | 20 ++
.../gms/ArrayBackedBoundedStatsTest.java| 20 ++
.../cassandra/io/RandomAccessReaderTest.java| 20 ++
.../io/util/BufferedDataOutputStreamTest.java | 20 ++
.../io/util/NIODataInputStreamTest.java | 20 ++
.../cassandra/locator/PendingRangeMapsTest.java | 20 ++
.../cassandra/net/MessagingServiceTest.java | 20 ++
.../apache/cassandra/utils/TopKSamplerTest.java | 20 ++
24 files changed, 482 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbefa854/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
--
diff --cc
src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
index 33ef0af,000..8636f19
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
+++ b/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
@@@ -1,132 -1,0 +1,152 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.cql3.selection;
+
+import java.util.*;
+
+import com.google.common.base.Function;
+import com.google.common.base.Joiner;
+import com.google.common.base.Objects;
+import com.google.common.collect.*;
+
+import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.cql3.ColumnSpecification;
+
+/**
+ * Separately maintains the ColumnSpecifications and their mappings to
underlying
+ * columns as we may receive null mappings. This occurs where a query result
+ * includes a column specification which does not map to any particular real
+ * column, e.g. COUNT queries or where no-arg functions like now() are used
+ */
+public class SelectionColumnMapping implements SelectionColumns
+{
+private final ArrayList columnSpecifications;
+private final HashMultimap
columnMappings;
+
+private SelectionColumnMapping()
+{
+this.columnSpecifications = new ArrayList<>();
+this.columnMappings = HashMultimap.create();
+}
+
+protected static SelectionColumnMapping newMapping()
[09/20] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbefa854/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
--
diff --cc test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
index 71fab61,000..8c6cc90
mode 100644,00..100644
--- a/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
@@@ -1,231 -1,0 +1,251 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.io;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.UUID;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+
+import static org.junit.Assert.*;
+import org.apache.cassandra.io.util.ChannelProxy;
+import org.apache.cassandra.io.util.FileMark;
+import org.apache.cassandra.io.util.RandomAccessReader;
+import org.apache.cassandra.io.util.SequentialWriter;
+
+public class RandomAccessReaderTest
+{
+@Test
+public void testReadFully() throws IOException
+{
+final File f = File.createTempFile("testReadFully", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+
+SequentialWriter writer = SequentialWriter.open(f);
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(f.getAbsolutePath(), reader.getPath());
+assertEquals(expected.length(), reader.length());
+
+byte[] b = new byte[expected.length()];
+reader.readFully(b);
+assertEquals(expected, new String(b));
+
+assertTrue(reader.isEOF());
+assertEquals(0, reader.bytesRemaining());
+
+reader.close();
+channel.close();
+}
+
+@Test
+public void testReadBytes() throws IOException
+{
+File f = File.createTempFile("testReadBytes", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+
+SequentialWriter writer = SequentialWriter.open(f);
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(f.getAbsolutePath(), reader.getPath());
+assertEquals(expected.length(), reader.length());
+
+ByteBuffer b = reader.readBytes(expected.length());
+assertEquals(expected, new String(b.array(),
Charset.forName("UTF-8")));
+
+assertTrue(reader.isEOF());
+assertEquals(0, reader.bytesRemaining());
+
+reader.close();
+channel.close();
+}
+
+@Test
+public void testReset() throws IOException
+{
+File f = File.createTempFile("testMark", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+final int numIterations = 10;
+
+SequentialWriter writer = SequentialWriter.open(f);
+for (int i = 0; i < numIterations; i++)
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(expected.length() * numIterations, reader.length());
+
+ByteBuffer b = reader.readBytes(expected.length());
+assertEquals(expected, new String(b.array(),
Charset.forName("UTF-8")));
+
+assertFalse(reader.isEOF());
+assertEquals((numIterations - 1) * expected.length(),
reader.bytesRemaining());
+
+FileMark mark = reader.mark();
+assertEquals(0, reader.bytesPastMark());
+assertEquals(0, reader.bytesPastMark(mark));
+
+for (int i = 0; i <
[11/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/test/unit/org/apache/cassandra/index/CustomIndexTest.java
--
diff --cc test/unit/org/apache/cassandra/index/CustomIndexTest.java
index fa6700b,000..f02823c
mode 100644,00..100644
--- a/test/unit/org/apache/cassandra/index/CustomIndexTest.java
+++ b/test/unit/org/apache/cassandra/index/CustomIndexTest.java
@@@ -1,806 -1,0 +1,826 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.index;
+
+import java.util.*;
+import java.util.concurrent.Callable;
+import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.stream.Collectors;
+
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.ImmutableMap;
+import org.junit.Test;
+
+import com.datastax.driver.core.exceptions.QueryValidationException;
+import org.apache.cassandra.Util;
+import org.apache.cassandra.config.CFMetaData;
+import org.apache.cassandra.cql3.CQLTester;
+import org.apache.cassandra.cql3.ColumnIdentifier;
+import org.apache.cassandra.cql3.restrictions.IndexRestrictions;
+import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
+import org.apache.cassandra.cql3.statements.IndexTarget;
+import org.apache.cassandra.cql3.statements.ModificationStatement;
+import org.apache.cassandra.db.ColumnFamilyStore;
+import org.apache.cassandra.db.ReadCommand;
+import org.apache.cassandra.db.ReadOrderGroup;
+import org.apache.cassandra.db.marshal.AbstractType;
+import org.apache.cassandra.db.marshal.Int32Type;
+import org.apache.cassandra.db.marshal.UTF8Type;
+import org.apache.cassandra.db.partitions.UnfilteredPartitionIterator;
+import org.apache.cassandra.exceptions.InvalidRequestException;
+import org.apache.cassandra.schema.IndexMetadata;
+import org.apache.cassandra.schema.Indexes;
+import org.apache.cassandra.transport.Server;
+import org.apache.cassandra.utils.FBUtilities;
+
+import static org.apache.cassandra.Util.throwAssert;
+import static
org.apache.cassandra.cql3.statements.IndexTarget.CUSTOM_INDEX_OPTION_NAME;
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+public class CustomIndexTest extends CQLTester
+{
+@Test
+public void testInsertsOnCfsBackedIndex() throws Throwable
+{
+// test to ensure that we don't deadlock when flushing CFS backed
custom indexers
+// see CASSANDRA-10181
+createTable("CREATE TABLE %s (a int, b int, c int, d int, PRIMARY KEY
(a, b))");
+createIndex("CREATE CUSTOM INDEX myindex ON %s(c) USING
'org.apache.cassandra.index.internal.CustomCassandraIndex'");
+
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 0, 0,
2);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 1, 0,
1);
+execute("INSERT INTO %s (a, b, c, d) VALUES (?, ?, ?, ?)", 0, 2, 0,
0);
+}
+
+@Test
+public void testTruncateWithNonCfsCustomIndex() throws Throwable
+{
+// deadlocks and times out the test in the face of the synchronisation
+// issues described in the comments on CASSANDRA-9669
+createTable("CREATE TABLE %s (a int, b int, c int, PRIMARY KEY (a))");
+createIndex("CREATE CUSTOM INDEX b_index ON %s(b) USING
'org.apache.cassandra.index.StubIndex'");
+execute("INSERT INTO %s (a, b, c) VALUES (?, ?, ?)", 0, 1, 2);
+getCurrentColumnFamilyStore().truncateBlocking();
+}
+
+@Test
+public void indexControlsIfIncludedInBuildOnNewSSTables() throws Throwable
+{
+createTable("CREATE TABLE %s (a int, b int, PRIMARY KEY (a))");
+String toInclude = "include";
+String toExclude = "exclude";
+createIndex(String.format("CREATE CUSTOM INDEX %s ON %%s(b) USING
'%s'",
+ toInclude,
IndexIncludedInBuild.class.getName()));
+createIndex(String.format("CREATE CUSTOM INDEX %s ON %%s(b) US
[10/20] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/dbefa854
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/dbefa854
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/dbefa854
Branch: refs/heads/trunk
Commit: dbefa854b403824334922dd35dc5e18ff1be51a9
Parents: c2566d1 615bf37
Author: Carl Yeksigian
Authored: Wed Jun 15 10:39:09 2016 -0400
Committer: Carl Yeksigian
Committed: Wed Jun 15 10:39:09 2016 -0400
--
.../cql3/selection/SelectionColumnMapping.java | 20 ++
.../cql3/selection/SelectionColumns.java| 20 ++
.../db/lifecycle/SSTableIntervalTree.java | 22 +++-
.../cassandra/hadoop/pig/StorageHelper.java | 20 ++
.../cassandra/locator/PendingRangeMaps.java | 20 ++
.../cassandra/repair/RepairParallelism.java | 20 ++
.../apache/cassandra/utils/CRC32Factory.java| 20 ++
.../apache/cassandra/utils/OverlapIterator.java | 22 +++-
.../utils/RMIServerSocketFactoryImpl.java | 20 ++
.../org/apache/cassandra/utils/SyncUtil.java| 20 ++
.../apache/cassandra/utils/concurrent/Ref.java | 20 ++
.../apache/cassandra/utils/concurrent/Refs.java | 20 ++
.../io/compress/CompressorPerformance.java | 20 ++
.../test/microbench/PendingRangesBench.java | 20 ++
.../selection/SelectionColumnMappingTest.java | 20 ++
.../validation/operations/SelectLimitTest.java | 20 ++
.../SelectOrderedPartitionerTest.java | 20 ++
.../gms/ArrayBackedBoundedStatsTest.java| 20 ++
.../cassandra/io/RandomAccessReaderTest.java| 20 ++
.../io/util/BufferedDataOutputStreamTest.java | 20 ++
.../io/util/NIODataInputStreamTest.java | 20 ++
.../cassandra/locator/PendingRangeMapsTest.java | 20 ++
.../cassandra/net/MessagingServiceTest.java | 20 ++
.../apache/cassandra/utils/TopKSamplerTest.java | 20 ++
24 files changed, 482 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbefa854/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
--
diff --cc
src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
index 33ef0af,000..8636f19
mode 100644,00..100644
--- a/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
+++ b/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java
@@@ -1,132 -1,0 +1,152 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.cql3.selection;
+
+import java.util.*;
+
+import com.google.common.base.Function;
+import com.google.common.base.Joiner;
+import com.google.common.base.Objects;
+import com.google.common.collect.*;
+
+import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.cql3.ColumnSpecification;
+
+/**
+ * Separately maintains the ColumnSpecifications and their mappings to
underlying
+ * columns as we may receive null mappings. This occurs where a query result
+ * includes a column specification which does not map to any particular real
+ * column, e.g. COUNT queries or where no-arg functions like now() are used
+ */
+public class SelectionColumnMapping implements SelectionColumns
+{
+private final ArrayList columnSpecifications;
+private final HashMultimap
columnMappings;
+
+private SelectionColumnMapping()
+{
+this.columnSpecifications = new ArrayList<>();
+this.columnMappings = HashMultimap.create();
+}
+
+protected static SelectionColumnMapping newMapping()
+{
[07/20] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
http://git-wip-us.apache.org/repos/asf/cassandra/blob/dbefa854/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
--
diff --cc test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
index 71fab61,000..8c6cc90
mode 100644,00..100644
--- a/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
+++ b/test/unit/org/apache/cassandra/io/RandomAccessReaderTest.java
@@@ -1,231 -1,0 +1,251 @@@
++/*
++ *
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing,
++ * software distributed under the License is distributed on an
++ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
++ * KIND, either express or implied. See the License for the
++ * specific language governing permissions and limitations
++ * under the License.
++ *
++ */
+package org.apache.cassandra.io;
+
+import java.io.File;
+import java.io.IOException;
+import java.nio.ByteBuffer;
+import java.nio.charset.Charset;
+import java.util.UUID;
+import java.util.concurrent.ExecutorService;
+import java.util.concurrent.Executors;
+import java.util.concurrent.TimeUnit;
+
+import org.junit.Test;
+
+import static org.junit.Assert.*;
+import org.apache.cassandra.io.util.ChannelProxy;
+import org.apache.cassandra.io.util.FileMark;
+import org.apache.cassandra.io.util.RandomAccessReader;
+import org.apache.cassandra.io.util.SequentialWriter;
+
+public class RandomAccessReaderTest
+{
+@Test
+public void testReadFully() throws IOException
+{
+final File f = File.createTempFile("testReadFully", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+
+SequentialWriter writer = SequentialWriter.open(f);
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(f.getAbsolutePath(), reader.getPath());
+assertEquals(expected.length(), reader.length());
+
+byte[] b = new byte[expected.length()];
+reader.readFully(b);
+assertEquals(expected, new String(b));
+
+assertTrue(reader.isEOF());
+assertEquals(0, reader.bytesRemaining());
+
+reader.close();
+channel.close();
+}
+
+@Test
+public void testReadBytes() throws IOException
+{
+File f = File.createTempFile("testReadBytes", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+
+SequentialWriter writer = SequentialWriter.open(f);
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(f.getAbsolutePath(), reader.getPath());
+assertEquals(expected.length(), reader.length());
+
+ByteBuffer b = reader.readBytes(expected.length());
+assertEquals(expected, new String(b.array(),
Charset.forName("UTF-8")));
+
+assertTrue(reader.isEOF());
+assertEquals(0, reader.bytesRemaining());
+
+reader.close();
+channel.close();
+}
+
+@Test
+public void testReset() throws IOException
+{
+File f = File.createTempFile("testMark", "1");
+final String expected = "The quick brown fox jumps over the lazy dog";
+final int numIterations = 10;
+
+SequentialWriter writer = SequentialWriter.open(f);
+for (int i = 0; i < numIterations; i++)
+writer.write(expected.getBytes());
+writer.finish();
+
+assert f.exists();
+
+ChannelProxy channel = new ChannelProxy(f);
+RandomAccessReader reader = RandomAccessReader.open(channel);
+assertEquals(expected.length() * numIterations, reader.length());
+
+ByteBuffer b = reader.readBytes(expected.length());
+assertEquals(expected, new String(b.array(),
Charset.forName("UTF-8")));
+
+assertFalse(reader.isEOF());
+assertEquals((numIterations - 1) * expected.length(),
reader.bytesRemaining());
+
+FileMark mark = reader.mark();
+assertEquals(0, reader.bytesPastMark());
+assertEquals(0, reader.bytesPastMark(mark));
+
+for (int i = 0; i <
[14/20] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/05bacc75 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/05bacc75 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/05bacc75 Branch: refs/heads/cassandra-3.0 Commit: 05bacc756e983e2850af7fbd265951983e66f4a0 Parents: 70ee4ed dbefa85 Author: Carl Yeksigian Authored: Wed Jun 15 10:52:55 2016 -0400 Committer: Carl Yeksigian Committed: Wed Jun 15 10:52:55 2016 -0400 -- .../cql3/selection/SelectionColumnMapping.java | 20 ++ .../cql3/selection/SelectionColumns.java| 20 ++ .../db/lifecycle/LogAwareFileLister.java| 20 ++ .../apache/cassandra/db/lifecycle/LogFile.java | 20 ++ .../cassandra/db/lifecycle/LogRecord.java | 20 ++ .../db/lifecycle/SSTableIntervalTree.java | 20 ++ .../cassandra/db/lifecycle/SSTableSet.java | 22 +++- .../cassandra/db/transform/BaseIterator.java| 20 ++ .../cassandra/db/transform/BasePartitions.java | 20 ++ .../apache/cassandra/db/transform/BaseRows.java | 20 ++ .../apache/cassandra/db/transform/Filter.java | 20 ++ .../db/transform/FilteredPartitions.java| 20 ++ .../cassandra/db/transform/FilteredRows.java| 20 ++ .../cassandra/db/transform/MoreContents.java| 20 ++ .../cassandra/db/transform/MorePartitions.java | 20 ++ .../apache/cassandra/db/transform/MoreRows.java | 20 ++ .../apache/cassandra/db/transform/Stack.java| 20 ++ .../db/transform/StoppingTransformation.java| 20 ++ .../cassandra/db/transform/Transformation.java | 20 ++ .../db/transform/UnfilteredPartitions.java | 20 ++ .../cassandra/db/transform/UnfilteredRows.java | 20 ++ src/java/org/apache/cassandra/index/Index.java | 20 ++ .../apache/cassandra/index/IndexRegistry.java | 20 ++ .../index/internal/CassandraIndex.java | 20 ++ .../index/internal/CassandraIndexSearcher.java | 20 ++ .../cassandra/index/internal/IndexEntry.java| 20 ++ .../index/internal/keys/KeysIndex.java | 20 ++ .../cassandra/locator/PendingRangeMaps.java | 20 ++ .../cassandra/repair/RepairParallelism.java | 20 ++ .../apache/cassandra/tools/JsonTransformer.java | 22 +++- .../apache/cassandra/utils/OverlapIterator.java | 22 +++- .../utils/RMIServerSocketFactoryImpl.java | 20 ++ .../org/apache/cassandra/utils/SyncUtil.java| 20 ++ .../apache/cassandra/utils/concurrent/Ref.java | 20 ++ .../apache/cassandra/utils/concurrent/Refs.java | 20 ++ .../io/compress/CompressorPerformance.java | 20 ++ .../test/microbench/PendingRangesBench.java | 20 ++ .../cassandra/cql3/IndexQueryPagingTest.java| 20 ++ .../selection/SelectionColumnMappingTest.java | 20 ++ .../validation/operations/SelectLimitTest.java | 20 ++ .../SelectOrderedPartitionerTest.java | 20 ++ .../db/SinglePartitionSliceCommandTest.java | 20 ++ .../commitlog/CommitLogSegmentManagerTest.java | 22 +++- .../rows/RowAndDeletionMergeIteratorTest.java | 20 ++ .../gms/ArrayBackedBoundedStatsTest.java| 20 ++ .../apache/cassandra/index/CustomIndexTest.java | 20 ++ .../index/internal/CustomCassandraIndex.java| 20 ++ .../io/util/BufferedDataOutputStreamTest.java | 20 ++ .../io/util/NIODataInputStreamTest.java | 20 ++ .../io/util/RandomAccessReaderTest.java | 20 ++ .../cassandra/locator/PendingRangeMapsTest.java | 20 ++ .../cassandra/net/MessagingServiceTest.java | 20 ++ .../service/RMIServerSocketFactoryImplTest.java | 20 ++ .../apache/cassandra/utils/TopKSamplerTest.java | 20 ++ 54 files changed, 1084 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/apache/cassandra/cql3/selection/SelectionColumnMapping.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/05bacc75/src/java/org/
[03/20] cassandra git commit: ninja fix license headers
ninja fix license headers Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/615bf372 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/615bf372 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/615bf372 Branch: refs/heads/cassandra-3.0 Commit: 615bf3721e26db5dc1350d2a74d28cf907cffa53 Parents: 2f74831 Author: Carl Yeksigian Authored: Wed Jun 15 10:29:27 2016 -0400 Committer: Carl Yeksigian Committed: Wed Jun 15 10:29:27 2016 -0400 -- .../cql3/statements/SelectionColumnMapping.java | 20 .../cql3/statements/SelectionColumns.java | 20 .../cassandra/repair/RepairParallelism.java | 20 .../utils/RMIServerSocketFactoryImpl.java | 20 .../apache/cassandra/utils/concurrent/Ref.java | 20 .../apache/cassandra/utils/concurrent/Refs.java | 20 .../statements/SelectionColumnMappingTest.java | 20 .../validation/operations/SelectLimitTest.java | 20 .../SelectOrderedPartitionerTest.java | 20 .../gms/ArrayBackedBoundedStatsTest.java| 20 .../cassandra/net/MessagingServiceTest.java | 20 .../apache/cassandra/utils/TopKSamplerTest.java | 20 12 files changed, 240 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java index aba2631..39bd12c 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.*; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java index 5b18eff..fa6e214 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.List; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/repair/RepairParallelism.java -- diff --git a/src/java/org/apache/cassandra/repair/RepairParallelism.java b/src/java/org/apache/cassandra/repair/RepairParallelism.java index 12c22ca..0afde28 100644 --- a/src/java/org/apache/cassandra/repair/RepairParallelism.java +++ b/src/ja
[04/20] cassandra git commit: ninja fix license headers
ninja fix license headers Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/615bf372 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/615bf372 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/615bf372 Branch: refs/heads/trunk Commit: 615bf3721e26db5dc1350d2a74d28cf907cffa53 Parents: 2f74831 Author: Carl Yeksigian Authored: Wed Jun 15 10:29:27 2016 -0400 Committer: Carl Yeksigian Committed: Wed Jun 15 10:29:27 2016 -0400 -- .../cql3/statements/SelectionColumnMapping.java | 20 .../cql3/statements/SelectionColumns.java | 20 .../cassandra/repair/RepairParallelism.java | 20 .../utils/RMIServerSocketFactoryImpl.java | 20 .../apache/cassandra/utils/concurrent/Ref.java | 20 .../apache/cassandra/utils/concurrent/Refs.java | 20 .../statements/SelectionColumnMappingTest.java | 20 .../validation/operations/SelectLimitTest.java | 20 .../SelectOrderedPartitionerTest.java | 20 .../gms/ArrayBackedBoundedStatsTest.java| 20 .../cassandra/net/MessagingServiceTest.java | 20 .../apache/cassandra/utils/TopKSamplerTest.java | 20 12 files changed, 240 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java index aba2631..39bd12c 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.*; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java index 5b18eff..fa6e214 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.List; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/repair/RepairParallelism.java -- diff --git a/src/java/org/apache/cassandra/repair/RepairParallelism.java b/src/java/org/apache/cassandra/repair/RepairParallelism.java index 12c22ca..0afde28 100644 --- a/src/java/org/apache/cassandra/repair/RepairParallelism.java +++ b/src/java/org/a
[01/20] cassandra git commit: ninja fix license headers
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 2f74831f4 -> 615bf3721 refs/heads/cassandra-2.2 c2566d1cf -> dbefa854b refs/heads/cassandra-3.0 70ee4ed49 -> 05bacc756 refs/heads/trunk 4210a25a9 -> 693e52d66 ninja fix license headers Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/615bf372 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/615bf372 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/615bf372 Branch: refs/heads/cassandra-2.1 Commit: 615bf3721e26db5dc1350d2a74d28cf907cffa53 Parents: 2f74831 Author: Carl Yeksigian Authored: Wed Jun 15 10:29:27 2016 -0400 Committer: Carl Yeksigian Committed: Wed Jun 15 10:29:27 2016 -0400 -- .../cql3/statements/SelectionColumnMapping.java | 20 .../cql3/statements/SelectionColumns.java | 20 .../cassandra/repair/RepairParallelism.java | 20 .../utils/RMIServerSocketFactoryImpl.java | 20 .../apache/cassandra/utils/concurrent/Ref.java | 20 .../apache/cassandra/utils/concurrent/Refs.java | 20 .../statements/SelectionColumnMappingTest.java | 20 .../validation/operations/SelectLimitTest.java | 20 .../SelectOrderedPartitionerTest.java | 20 .../gms/ArrayBackedBoundedStatsTest.java| 20 .../cassandra/net/MessagingServiceTest.java | 20 .../apache/cassandra/utils/TopKSamplerTest.java | 20 12 files changed, 240 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java index aba2631..39bd12c 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumnMapping.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.*; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java -- diff --git a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java index 5b18eff..fa6e214 100644 --- a/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java +++ b/src/java/org/apache/cassandra/cql3/statements/SelectionColumns.java @@ -1,3 +1,23 @@ +/* + * + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + * + */ package org.apache.cassandra.cql3.statements; import java.util.List; http://git-wip-us.apache.org/repos/asf/cassandra/blob/615bf372/src/java/org/apache/cassandra/repair/RepairParallelism.java -- diff --git
[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)
[ https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331768#comment-15331768 ] Joshua McKenzie commented on CASSANDRA-8844: bq. That's what is in the code That's a pretty nasty idiom as it leaves a lot of inter-version cruft around. Would be really nice to have versioned schema for these things cleanly abstracted so we could just deserialize a version and construct w/the appropriate method from there. But that's a problem for another day. > Change Data Capture (CDC) > - > > Key: CASSANDRA-8844 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8844 > Project: Cassandra > Issue Type: New Feature > Components: Coordination, Local Write-Read Paths >Reporter: Tupshin Harper >Assignee: Joshua McKenzie >Priority: Critical > Fix For: 3.x > > > "In databases, change data capture (CDC) is a set of software design patterns > used to determine (and track) the data that has changed so that action can be > taken using the changed data. Also, Change data capture (CDC) is an approach > to data integration that is based on the identification, capture and delivery > of the changes made to enterprise data sources." > -Wikipedia > As Cassandra is increasingly being used as the Source of Record (SoR) for > mission critical data in large enterprises, it is increasingly being called > upon to act as the central hub of traffic and data flow to other systems. In > order to try to address the general need, we (cc [~brianmhess]), propose > implementing a simple data logging mechanism to enable per-table CDC patterns. > h2. The goals: > # Use CQL as the primary ingestion mechanism, in order to leverage its > Consistency Level semantics, and in order to treat it as the single > reliable/durable SoR for the data. > # To provide a mechanism for implementing good and reliable > (deliver-at-least-once with possible mechanisms for deliver-exactly-once ) > continuous semi-realtime feeds of mutations going into a Cassandra cluster. > # To eliminate the developmental and operational burden of users so that they > don't have to do dual writes to other systems. > # For users that are currently doing batch export from a Cassandra system, > give them the opportunity to make that realtime with a minimum of coding. > h2. The mechanism: > We propose a durable logging mechanism that functions similar to a commitlog, > with the following nuances: > - Takes place on every node, not just the coordinator, so RF number of copies > are logged. > - Separate log per table. > - Per-table configuration. Only tables that are specified as CDC_LOG would do > any logging. > - Per DC. We are trying to keep the complexity to a minimum to make this an > easy enhancement, but most likely use cases would prefer to only implement > CDC logging in one (or a subset) of the DCs that are being replicated to > - In the critical path of ConsistencyLevel acknowledgment. Just as with the > commitlog, failure to write to the CDC log should fail that node's write. If > that means the requested consistency level was not met, then clients *should* > experience UnavailableExceptions. > - Be written in a Row-centric manner such that it is easy for consumers to > reconstitute rows atomically. > - Written in a simple format designed to be consumed *directly* by daemons > written in non JVM languages > h2. Nice-to-haves > I strongly suspect that the following features will be asked for, but I also > believe that they can be deferred for a subsequent release, and to guage > actual interest. > - Multiple logs per table. This would make it easy to have multiple > "subscribers" to a single table's changes. A workaround would be to create a > forking daemon listener, but that's not a great answer. > - Log filtering. Being able to apply filters, including UDF-based filters > would make Casandra a much more versatile feeder into other systems, and > again, reduce complexity that would otherwise need to be built into the > daemons. > h2. Format and Consumption > - Cassandra would only write to the CDC log, and never delete from it. > - Cleaning up consumed logfiles would be the client daemon's responibility > - Logfile size should probably be configurable. > - Logfiles should be named with a predictable naming schema, making it > triivial to process them in order. > - Daemons should be able to checkpoint their work, and resume from where they > left off. This means they would have to leave some file artifact in the CDC > log's directory. > - A sophisticated daemon should be able to be written that could > -- Catch up, in written-order, even when it is multiple logfiles behind in > processing > -- Be able to continuously "tail" the most recent logfile and get > low-latency(
[jira] [Commented] (CASSANDRA-11988) NullPointerExpception when reading/compacting table
[
https://issues.apache.org/jira/browse/CASSANDRA-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331667#comment-15331667
]
Carl Yeksigian commented on CASSANDRA-11988:
[~marekasf] This is really helpful; it definitely focuses the work on this a
lot. I'll report back when I have something more.
> NullPointerExpception when reading/compacting table
> ---
>
> Key: CASSANDRA-11988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11988
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nimi Wariboko Jr.
>Assignee: Carl Yeksigian
> Fix For: 3.6
>
>
> I have a table that suddenly refuses to be read or compacted. Issuing a read
> on the table causes a NPE.
> On compaction, it returns the error
> {code}
> ERROR [CompactionExecutor:6] 2016-06-09 17:10:15,724 CassandraDaemon.java:213
> - Exception in thread Thread[CompactionExecutor:6,1,main]
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:64)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:264)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_45]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> ~[na:1.8.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_45]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}
> Schema:
> {code}
> CREATE TABLE cmpayments.report_payments (
> reportid timeuuid,
> userid timeuuid,
> adjustedearnings decimal,
> deleted set static,
> earnings map,
> gross map,
> organizationid text,
> payall timestamp static,
> status text,
> PRIMARY KEY (reportid, userid)
> ) WITH CLUSTERING ORDER BY (userid ASC)
> AND bloom_filter_fp_chance = 0.01
> AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
> AND comment = ''
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
> 'max_threshold': '32', 'min_threshold': '4'}
> AND compression = {'chunk_length_in_kb': '64', 'class':
> 'org.apache.cassandra.io.compress.LZ4Compressor'}
> AND crc_check_chance = 1.0
> AND dclocal_read_repair_chance = 0.1
> AND default_time_to_live = 0
> AND gc_grace_seconds = 864000
> AND max_index_interval = 2048
> AND memtable_flush_period_in_ms = 0
> AND min_index_interval = 128
> AND read_repair_chance = 0.0
> AND speculative_retry = '99PERCENTILE';
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8844) Change Data Capture (CDC)
[
https://issues.apache.org/jira/browse/CASSANDRA-8844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331623#comment-15331623
]
Carl Yeksigian commented on CASSANDRA-8844:
---
{quote}
No need to get snippy.
{quote}
Hmm, the text has lost the all-important tone; rather than bemused riffing, it
reads much more as "DO WHAT I SAID NOW!"
{quote}
I have a hard time thinking there's not a better way than:
{code}
.cdc(row.has("cdc") ? row.getBoolean("cdc") : false)
{code}
Is there?
{quote}
That's what is in the code and I've done before; it'd be cleaner if we used
optional, but to get this ticket shipped, let's go with that.
> Change Data Capture (CDC)
> -
>
> Key: CASSANDRA-8844
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8844
> Project: Cassandra
> Issue Type: New Feature
> Components: Coordination, Local Write-Read Paths
>Reporter: Tupshin Harper
>Assignee: Joshua McKenzie
>Priority: Critical
> Fix For: 3.x
>
>
> "In databases, change data capture (CDC) is a set of software design patterns
> used to determine (and track) the data that has changed so that action can be
> taken using the changed data. Also, Change data capture (CDC) is an approach
> to data integration that is based on the identification, capture and delivery
> of the changes made to enterprise data sources."
> -Wikipedia
> As Cassandra is increasingly being used as the Source of Record (SoR) for
> mission critical data in large enterprises, it is increasingly being called
> upon to act as the central hub of traffic and data flow to other systems. In
> order to try to address the general need, we (cc [~brianmhess]), propose
> implementing a simple data logging mechanism to enable per-table CDC patterns.
> h2. The goals:
> # Use CQL as the primary ingestion mechanism, in order to leverage its
> Consistency Level semantics, and in order to treat it as the single
> reliable/durable SoR for the data.
> # To provide a mechanism for implementing good and reliable
> (deliver-at-least-once with possible mechanisms for deliver-exactly-once )
> continuous semi-realtime feeds of mutations going into a Cassandra cluster.
> # To eliminate the developmental and operational burden of users so that they
> don't have to do dual writes to other systems.
> # For users that are currently doing batch export from a Cassandra system,
> give them the opportunity to make that realtime with a minimum of coding.
> h2. The mechanism:
> We propose a durable logging mechanism that functions similar to a commitlog,
> with the following nuances:
> - Takes place on every node, not just the coordinator, so RF number of copies
> are logged.
> - Separate log per table.
> - Per-table configuration. Only tables that are specified as CDC_LOG would do
> any logging.
> - Per DC. We are trying to keep the complexity to a minimum to make this an
> easy enhancement, but most likely use cases would prefer to only implement
> CDC logging in one (or a subset) of the DCs that are being replicated to
> - In the critical path of ConsistencyLevel acknowledgment. Just as with the
> commitlog, failure to write to the CDC log should fail that node's write. If
> that means the requested consistency level was not met, then clients *should*
> experience UnavailableExceptions.
> - Be written in a Row-centric manner such that it is easy for consumers to
> reconstitute rows atomically.
> - Written in a simple format designed to be consumed *directly* by daemons
> written in non JVM languages
> h2. Nice-to-haves
> I strongly suspect that the following features will be asked for, but I also
> believe that they can be deferred for a subsequent release, and to guage
> actual interest.
> - Multiple logs per table. This would make it easy to have multiple
> "subscribers" to a single table's changes. A workaround would be to create a
> forking daemon listener, but that's not a great answer.
> - Log filtering. Being able to apply filters, including UDF-based filters
> would make Casandra a much more versatile feeder into other systems, and
> again, reduce complexity that would otherwise need to be built into the
> daemons.
> h2. Format and Consumption
> - Cassandra would only write to the CDC log, and never delete from it.
> - Cleaning up consumed logfiles would be the client daemon's responibility
> - Logfile size should probably be configurable.
> - Logfiles should be named with a predictable naming schema, making it
> triivial to process them in order.
> - Daemons should be able to checkpoint their work, and resume from where they
> left off. This means they would have to leave some file artifact in the CDC
> log's directory.
> - A sophisticated daemon should be able to be written that could
> -- Catch up, in written-order, even when it i
[jira] [Commented] (CASSANDRA-11961) Nonfatal NPE in CompactionMetrics
[
https://issues.apache.org/jira/browse/CASSANDRA-11961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331484#comment-15331484
]
Robert Stupp commented on CASSANDRA-11961:
--
No, will review (and probably commit) in the next days. (I'm off this week)
> Nonfatal NPE in CompactionMetrics
> -
>
> Key: CASSANDRA-11961
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11961
> Project: Cassandra
> Issue Type: Bug
>Reporter: Robert Stupp
>Assignee: Achal Shah
>Priority: Minor
> Labels: lhf
>
> Just saw the following NPE on trunk. Means, that {{metaData}} from
> {{CFMetaData metaData = compaction.getCompactionInfo().getCFMetaData();}} is
> {{null}}. A simple {{if (metaData == null) continue;}} should fix this.
> {code}
> Caused by: java.lang.NullPointerException: null
> at
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:103)
> ~[main/:na]
> at
> org.apache.cassandra.metrics.CompactionMetrics$2.getValue(CompactionMetrics.java:78)
> ~[main/:na]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11933) Cache local ranges when calculating repair neighbors
[
https://issues.apache.org/jira/browse/CASSANDRA-11933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-11933:
-
Resolution: Fixed
Fix Version/s: 3.0.8
3.8
2.2.7
2.1.15
Status: Resolved (was: Ready to Commit)
Nice stuff!
Committed as 2f74831f4218142f6e118678a3c74c79c1f7e1ed to 2.1 and merged to
trunk (cherry-picked from Paulo's branches).
> Cache local ranges when calculating repair neighbors
>
>
> Key: CASSANDRA-11933
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11933
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Cyril Scetbon
>Assignee: Mahdi Mohammadi
> Fix For: 2.1.15, 2.2.7, 3.8, 3.0.8
>
>
> During a full repair on a ~ 60 nodes cluster, I've been able to see that
> this stage can be significant (up to 60 percent of the whole time) :
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/StorageService.java#L2983-L2997
> It's merely caused by the fact that
> https://github.com/apache/cassandra/blob/cassandra-2.1/src/java/org/apache/cassandra/service/ActiveRepairService.java#L189
> calls {code}ss.getLocalRanges(keyspaceName){code} everytime and that it
> takes more than 99% of the time. This call takes 600ms when there is no load
> on the cluster and more if there is. So for 10k ranges, you can imagine that
> it takes at least 1.5 hours just to compute ranges.
> Underneath it calls
> [ReplicationStrategy.getAddressRanges|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L170]
> which can get pretty inefficient ([~jbellis]'s
> [words|https://github.com/apache/cassandra/blob/3dcbe90e02440e6ee534f643c7603d50ca08482b/src/java/org/apache/cassandra/locator/AbstractReplicationStrategy.java#L165])
> *ss.getLocalRanges(keyspaceName)* should be cached to avoid having to spend
> hours on it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[08/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70ee4ed4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70ee4ed4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70ee4ed4
Branch: refs/heads/cassandra-3.0
Commit: 70ee4ed49a01a6466f8156c69f383ebc1d3b0e29
Parents: 3c531cb c2566d1
Author: Robert Stupp
Authored: Wed Jun 15 11:46:25 2016 +0200
Committer: Robert Stupp
Committed: Wed Jun 15 11:46:25 2016 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/repair/RepairRunnable.java | 7 ++-
.../cassandra/service/ActiveRepairService.java| 7 +--
.../service/ActiveRepairServiceTest.java | 18 ++
4 files changed, 22 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/70ee4ed4/CHANGES.txt
--
diff --cc CHANGES.txt
index 2c42c94,d9afaa3..3eb420d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,5 +1,31 @@@
-2.2.7
+3.0.8
+ * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
+Merged from 2.2:
* StorageService shutdown hook should use a volatile variable
(CASSANDRA-11984)
+Merged from 2.1:
++ * Cache local ranges when calculating repair neighbors (CASSANDRA-11934)
+ * Allow LWT operation on static column with only partition keys
(CASSANDRA-10532)
+ * Create interval tree over canonical sstables to avoid missing sstables
during streaming (CASSANDRA-11886)
+ * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid
corrupting SSL connections (CASSANDRA-11749)
+
+
+3.0.7
+ * Fix legacy serialization of Thrift-generated non-compound range tombstones
+ when communicating with 2.x nodes (CASSANDRA-11930)
+ * Fix Directories instantiations where CFS.initialDirectories should be used
(CASSANDRA-11849)
+ * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912)
+ * Fix sstables not being protected from removal during index build
(CASSANDRA-11905)
+ * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032)
+ * Remove unneeded code to repair index summaries that have
+ been improperly down-sampled (CASSANDRA-11127)
+ * Avoid WriteTimeoutExceptions during commit log replay due to materialized
+ view lock contention (CASSANDRA-11891)
+ * Prevent OOM failures on SSTable corruption, improve tests for corruption
detection (CASSANDRA-9530)
+ * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705)
+ * Allow compaction strategies to disable early open (CASSANDRA-11754)
+ * Refactor Materialized View code (CASSANDRA-11475)
+ * Update Java Driver (CASSANDRA-11615)
+Merged from 2.2:
* Persist local metadata earlier in startup sequence (CASSANDRA-11742)
* Run CommitLog tests with different compression settings (CASSANDRA-9039)
* cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/70ee4ed4/src/java/org/apache/cassandra/repair/RepairRunnable.java
--
diff --cc src/java/org/apache/cassandra/repair/RepairRunnable.java
index 741cada,f92310b..21d0cd6
--- a/src/java/org/apache/cassandra/repair/RepairRunnable.java
+++ b/src/java/org/apache/cassandra/repair/RepairRunnable.java
@@@ -147,19 -146,22 +147,24 @@@ public class RepairRunnable extends Wra
}
final Set allNeighbors = new HashSet<>();
-Map> rangeToNeighbors = new HashMap<>();
+List, ? extends Collection>>>
commonRanges = new ArrayList<>();
+
+ //pre-calculate output of getLocalRanges and pass it to getNeighbors
to increase performance and prevent
+ //calculation multiple times
+ Collection> keyspaceLocalRanges =
storageService.getLocalRanges(keyspace);
+
try
{
for (Range range : options.getRanges())
{
- Set neighbors =
ActiveRepairService.getNeighbors(keyspace, range,
-Set neighbors =
ActiveRepairService.getNeighbors(keyspace, keyspaceLocalRanges,
-
range, options.getDataCenters(),
-
options.getHosts());
-rangeToNeighbors.put(range, neighbors);
-allNeighbors.addAll(neighbors);
++Set neighbors =
ActiveRepairService.getNeighbors(keyspace, keyspaceLocalRanges, range,
+
options.getDataCenters(),
+
[02/10] cassandra git commit: Cache local ranges when calculating repair neighbors
Cache local ranges when calculating repair neighbors
patch by Mahdi Mohammadi; reviewed by Paulo Motta for CASSANDRA-11933
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2f74831f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2f74831f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2f74831f
Branch: refs/heads/cassandra-2.2
Commit: 2f74831f4218142f6e118678a3c74c79c1f7e1ed
Parents: 1d2d074
Author: Mahdi Mohammadi
Authored: Wed Jun 15 11:43:27 2016 +0200
Committer: Robert Stupp
Committed: Wed Jun 15 11:43:27 2016 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/service/ActiveRepairService.java | 8 +---
.../org/apache/cassandra/service/StorageService.java | 6 +-
.../service/AntiEntropyServiceTestAbstract.java | 14 --
4 files changed, 19 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7d70902..ec2b48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.15
+ * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
* Allow LWT operation on static column with only partition keys
(CASSANDRA-10532)
* Create interval tree over canonical sstables to avoid missing sstables
during streaming (CASSANDRA-11886)
* cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting
SSL connections (CASSANDRA-11749)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index f8975f9..4c83c48 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -164,7 +164,8 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
RepairFuture submitArtificialRepairSession(RepairJobDesc desc)
{
Set neighbours = new HashSet<>();
-neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
desc.range, null, null));
+Collection> keyspaceLocalRanges =
StorageService.instance.getLocalRanges(desc.keyspace);
+neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
keyspaceLocalRanges, desc.range, null, null));
RepairSession session = new RepairSession(desc.parentSessionId,
desc.sessionId, desc.range, desc.keyspace, RepairParallelism.PARALLEL,
neighbours, new String[]{desc.columnFamily});
sessions.put(session.getId(), session);
RepairFuture futureTask = new RepairFuture(session);
@@ -176,17 +177,18 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
* Return all of the neighbors with whom we share the provided range.
*
* @param keyspaceName keyspace to repair
+ * @param keyspaceLocalRanges local-range for given keyspaceName
* @param toRepair token to repair
* @param dataCenters the data centers to involve in the repair
*
* @return neighbors with whom we share the provided range
*/
-public static Set getNeighbors(String keyspaceName,
Range toRepair, Collection dataCenters, Collection hosts)
+public static Set getNeighbors(String keyspaceName,
Collection> keyspaceLocalRanges, Range toRepair,
Collection dataCenters, Collection hosts)
{
StorageService ss = StorageService.instance;
Map, List> replicaSets =
ss.getRangeToAddressMap(keyspaceName);
Range rangeSuperSet = null;
-for (Range range : ss.getLocalRanges(keyspaceName))
+for (Range range : keyspaceLocalRanges)
{
if (range.contains(toRepair))
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index eea4556..27939f9 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2978,13 +2978,17 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
return;
}
+//pre-calculate output of getLocalRanges and pass it to
getNeighbors to increase performance and prevent
+
[jira] [Updated] (CASSANDRA-10202) simplify CommitLogSegmentManager
[ https://issues.apache.org/jira/browse/CASSANDRA-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-10202: Status: Open (was: Patch Available) > simplify CommitLogSegmentManager > > > Key: CASSANDRA-10202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10202 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Jonathan Ellis >Assignee: Branimir Lambov >Priority: Minor > > Now that we only keep one active segment around we can simplify this from the > old recycling design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[05/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c2566d1c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c2566d1c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c2566d1c Branch: refs/heads/cassandra-3.0 Commit: c2566d1cf9e239063a530ec08e8e098feffe387b Parents: 52a827b 2f74831 Author: Robert Stupp Authored: Wed Jun 15 11:44:57 2016 +0200 Committer: Robert Stupp Committed: Wed Jun 15 11:44:57 2016 +0200 -- CHANGES.txt | 1 + .../apache/cassandra/repair/RepairRunnable.java | 9 +++-- .../cassandra/service/ActiveRepairService.java| 5 +++-- .../service/ActiveRepairServiceTest.java | 18 ++ 4 files changed, 21 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2566d1c/CHANGES.txt -- diff --cc CHANGES.txt index d0ca37f,ec2b48e..d9afaa3 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,30 -1,5 +1,31 @@@ -2.1.15 +2.2.7 + * StorageService shutdown hook should use a volatile variable (CASSANDRA-11984) + * Persist local metadata earlier in startup sequence (CASSANDRA-11742) + * Run CommitLog tests with different compression settings (CASSANDRA-9039) + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: + * Cache local ranges when calculating repair neighbors (CASSANDRA-11933) * Allow LWT operation on static column with only partition keys (CASSANDRA-10532) * Create interval tree over canonical sstables to avoid missing sstables during streaming (CASSANDRA-11886) * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting SSL connections (CASSANDRA-11749) http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2566d1c/src/java/org/apache/cassandra/repair/RepairRunnable.java -- diff --cc src/java/org/apache/cassandra/repair/RepairRunnable.java index b849cf8,000..f92310b mode 100644,00..100644 --- a/src/java/org/apache/cassandra/repair/RepairRunnable.java +++ b/src/java/org/apache/cassandra/repair/RepairRunnable.java @@@ -1,409 -1,0 +1,414 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.repair; + +import java.net.InetA
[10/10] cassandra git commit: Merge branch 'cassandra-3.0' into trunk
Merge branch 'cassandra-3.0' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4210a25a Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4210a25a Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4210a25a Branch: refs/heads/trunk Commit: 4210a25a909c629c19db5d91de671c2a0f9fa28b Parents: 4ed0060 70ee4ed Author: Robert Stupp Authored: Wed Jun 15 11:47:37 2016 +0200 Committer: Robert Stupp Committed: Wed Jun 15 11:47:37 2016 +0200 -- CHANGES.txt | 2 +- .../apache/cassandra/repair/RepairRunnable.java | 7 ++- .../cassandra/service/ActiveRepairService.java| 7 +-- .../service/ActiveRepairServiceTest.java | 18 ++ 4 files changed, 22 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4210a25a/CHANGES.txt -- diff --cc CHANGES.txt index dbcfa34,3eb420d..c9fa673 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -20,11 -8,8 +21,10 @@@ Merged from 2.1 * Create interval tree over canonical sstables to avoid missing sstables during streaming (CASSANDRA-11886) * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting SSL connections (CASSANDRA-11749) -- -3.0.7 +3.7 + * Support multiple folders for user defined compaction tasks (CASSANDRA-11765) + * Fix race in CompactionStrategyManager's pause/resume (CASSANDRA-11922) +Merged from 3.0: * Fix legacy serialization of Thrift-generated non-compound range tombstones when communicating with 2.x nodes (CASSANDRA-11930) * Fix Directories instantiations where CFS.initialDirectories should be used (CASSANDRA-11849) http://git-wip-us.apache.org/repos/asf/cassandra/blob/4210a25a/src/java/org/apache/cassandra/repair/RepairRunnable.java -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4210a25a/src/java/org/apache/cassandra/service/ActiveRepairService.java --
[03/10] cassandra git commit: Cache local ranges when calculating repair neighbors
Cache local ranges when calculating repair neighbors
patch by Mahdi Mohammadi; reviewed by Paulo Motta for CASSANDRA-11933
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2f74831f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2f74831f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2f74831f
Branch: refs/heads/cassandra-3.0
Commit: 2f74831f4218142f6e118678a3c74c79c1f7e1ed
Parents: 1d2d074
Author: Mahdi Mohammadi
Authored: Wed Jun 15 11:43:27 2016 +0200
Committer: Robert Stupp
Committed: Wed Jun 15 11:43:27 2016 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/service/ActiveRepairService.java | 8 +---
.../org/apache/cassandra/service/StorageService.java | 6 +-
.../service/AntiEntropyServiceTestAbstract.java | 14 --
4 files changed, 19 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7d70902..ec2b48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.15
+ * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
* Allow LWT operation on static column with only partition keys
(CASSANDRA-10532)
* Create interval tree over canonical sstables to avoid missing sstables
during streaming (CASSANDRA-11886)
* cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting
SSL connections (CASSANDRA-11749)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index f8975f9..4c83c48 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -164,7 +164,8 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
RepairFuture submitArtificialRepairSession(RepairJobDesc desc)
{
Set neighbours = new HashSet<>();
-neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
desc.range, null, null));
+Collection> keyspaceLocalRanges =
StorageService.instance.getLocalRanges(desc.keyspace);
+neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
keyspaceLocalRanges, desc.range, null, null));
RepairSession session = new RepairSession(desc.parentSessionId,
desc.sessionId, desc.range, desc.keyspace, RepairParallelism.PARALLEL,
neighbours, new String[]{desc.columnFamily});
sessions.put(session.getId(), session);
RepairFuture futureTask = new RepairFuture(session);
@@ -176,17 +177,18 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
* Return all of the neighbors with whom we share the provided range.
*
* @param keyspaceName keyspace to repair
+ * @param keyspaceLocalRanges local-range for given keyspaceName
* @param toRepair token to repair
* @param dataCenters the data centers to involve in the repair
*
* @return neighbors with whom we share the provided range
*/
-public static Set getNeighbors(String keyspaceName,
Range toRepair, Collection dataCenters, Collection hosts)
+public static Set getNeighbors(String keyspaceName,
Collection> keyspaceLocalRanges, Range toRepair,
Collection dataCenters, Collection hosts)
{
StorageService ss = StorageService.instance;
Map, List> replicaSets =
ss.getRangeToAddressMap(keyspaceName);
Range rangeSuperSet = null;
-for (Range range : ss.getLocalRanges(keyspaceName))
+for (Range range : keyspaceLocalRanges)
{
if (range.contains(toRepair))
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index eea4556..27939f9 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2978,13 +2978,17 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
return;
}
+//pre-calculate output of getLocalRanges and pass it to
getNeighbors to increase performance and prevent
+
[07/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c2566d1c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c2566d1c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c2566d1c Branch: refs/heads/trunk Commit: c2566d1cf9e239063a530ec08e8e098feffe387b Parents: 52a827b 2f74831 Author: Robert Stupp Authored: Wed Jun 15 11:44:57 2016 +0200 Committer: Robert Stupp Committed: Wed Jun 15 11:44:57 2016 +0200 -- CHANGES.txt | 1 + .../apache/cassandra/repair/RepairRunnable.java | 9 +++-- .../cassandra/service/ActiveRepairService.java| 5 +++-- .../service/ActiveRepairServiceTest.java | 18 ++ 4 files changed, 21 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2566d1c/CHANGES.txt -- diff --cc CHANGES.txt index d0ca37f,ec2b48e..d9afaa3 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,30 -1,5 +1,31 @@@ -2.1.15 +2.2.7 + * StorageService shutdown hook should use a volatile variable (CASSANDRA-11984) + * Persist local metadata earlier in startup sequence (CASSANDRA-11742) + * Run CommitLog tests with different compression settings (CASSANDRA-9039) + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: + * Cache local ranges when calculating repair neighbors (CASSANDRA-11933) * Allow LWT operation on static column with only partition keys (CASSANDRA-10532) * Create interval tree over canonical sstables to avoid missing sstables during streaming (CASSANDRA-11886) * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting SSL connections (CASSANDRA-11749) http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2566d1c/src/java/org/apache/cassandra/repair/RepairRunnable.java -- diff --cc src/java/org/apache/cassandra/repair/RepairRunnable.java index b849cf8,000..f92310b mode 100644,00..100644 --- a/src/java/org/apache/cassandra/repair/RepairRunnable.java +++ b/src/java/org/apache/cassandra/repair/RepairRunnable.java @@@ -1,409 -1,0 +1,414 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.repair; + +import java.net.InetAddress;
[04/10] cassandra git commit: Cache local ranges when calculating repair neighbors
Cache local ranges when calculating repair neighbors
patch by Mahdi Mohammadi; reviewed by Paulo Motta for CASSANDRA-11933
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2f74831f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2f74831f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2f74831f
Branch: refs/heads/trunk
Commit: 2f74831f4218142f6e118678a3c74c79c1f7e1ed
Parents: 1d2d074
Author: Mahdi Mohammadi
Authored: Wed Jun 15 11:43:27 2016 +0200
Committer: Robert Stupp
Committed: Wed Jun 15 11:43:27 2016 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/service/ActiveRepairService.java | 8 +---
.../org/apache/cassandra/service/StorageService.java | 6 +-
.../service/AntiEntropyServiceTestAbstract.java | 14 --
4 files changed, 19 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7d70902..ec2b48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.15
+ * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
* Allow LWT operation on static column with only partition keys
(CASSANDRA-10532)
* Create interval tree over canonical sstables to avoid missing sstables
during streaming (CASSANDRA-11886)
* cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting
SSL connections (CASSANDRA-11749)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index f8975f9..4c83c48 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -164,7 +164,8 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
RepairFuture submitArtificialRepairSession(RepairJobDesc desc)
{
Set neighbours = new HashSet<>();
-neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
desc.range, null, null));
+Collection> keyspaceLocalRanges =
StorageService.instance.getLocalRanges(desc.keyspace);
+neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
keyspaceLocalRanges, desc.range, null, null));
RepairSession session = new RepairSession(desc.parentSessionId,
desc.sessionId, desc.range, desc.keyspace, RepairParallelism.PARALLEL,
neighbours, new String[]{desc.columnFamily});
sessions.put(session.getId(), session);
RepairFuture futureTask = new RepairFuture(session);
@@ -176,17 +177,18 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
* Return all of the neighbors with whom we share the provided range.
*
* @param keyspaceName keyspace to repair
+ * @param keyspaceLocalRanges local-range for given keyspaceName
* @param toRepair token to repair
* @param dataCenters the data centers to involve in the repair
*
* @return neighbors with whom we share the provided range
*/
-public static Set getNeighbors(String keyspaceName,
Range toRepair, Collection dataCenters, Collection hosts)
+public static Set getNeighbors(String keyspaceName,
Collection> keyspaceLocalRanges, Range toRepair,
Collection dataCenters, Collection hosts)
{
StorageService ss = StorageService.instance;
Map, List> replicaSets =
ss.getRangeToAddressMap(keyspaceName);
Range rangeSuperSet = null;
-for (Range range : ss.getLocalRanges(keyspaceName))
+for (Range range : keyspaceLocalRanges)
{
if (range.contains(toRepair))
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index eea4556..27939f9 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2978,13 +2978,17 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
return;
}
+//pre-calculate output of getLocalRanges and pass it to
getNeighbors to increase performance and prevent
+
[09/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/70ee4ed4
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/70ee4ed4
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/70ee4ed4
Branch: refs/heads/trunk
Commit: 70ee4ed49a01a6466f8156c69f383ebc1d3b0e29
Parents: 3c531cb c2566d1
Author: Robert Stupp
Authored: Wed Jun 15 11:46:25 2016 +0200
Committer: Robert Stupp
Committed: Wed Jun 15 11:46:25 2016 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/repair/RepairRunnable.java | 7 ++-
.../cassandra/service/ActiveRepairService.java| 7 +--
.../service/ActiveRepairServiceTest.java | 18 ++
4 files changed, 22 insertions(+), 11 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/70ee4ed4/CHANGES.txt
--
diff --cc CHANGES.txt
index 2c42c94,d9afaa3..3eb420d
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,30 -1,5 +1,31 @@@
-2.2.7
+3.0.8
+ * Add TimeWindowCompactionStrategy (CASSANDRA-9666)
+Merged from 2.2:
* StorageService shutdown hook should use a volatile variable
(CASSANDRA-11984)
+Merged from 2.1:
++ * Cache local ranges when calculating repair neighbors (CASSANDRA-11934)
+ * Allow LWT operation on static column with only partition keys
(CASSANDRA-10532)
+ * Create interval tree over canonical sstables to avoid missing sstables
during streaming (CASSANDRA-11886)
+ * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid
corrupting SSL connections (CASSANDRA-11749)
+
+
+3.0.7
+ * Fix legacy serialization of Thrift-generated non-compound range tombstones
+ when communicating with 2.x nodes (CASSANDRA-11930)
+ * Fix Directories instantiations where CFS.initialDirectories should be used
(CASSANDRA-11849)
+ * Avoid referencing DatabaseDescriptor in AbstractType (CASSANDRA-11912)
+ * Fix sstables not being protected from removal during index build
(CASSANDRA-11905)
+ * cqlsh: Suppress stack trace from Read/WriteFailures (CASSANDRA-11032)
+ * Remove unneeded code to repair index summaries that have
+ been improperly down-sampled (CASSANDRA-11127)
+ * Avoid WriteTimeoutExceptions during commit log replay due to materialized
+ view lock contention (CASSANDRA-11891)
+ * Prevent OOM failures on SSTable corruption, improve tests for corruption
detection (CASSANDRA-9530)
+ * Use CFS.initialDirectories when clearing snapshots (CASSANDRA-11705)
+ * Allow compaction strategies to disable early open (CASSANDRA-11754)
+ * Refactor Materialized View code (CASSANDRA-11475)
+ * Update Java Driver (CASSANDRA-11615)
+Merged from 2.2:
* Persist local metadata earlier in startup sequence (CASSANDRA-11742)
* Run CommitLog tests with different compression settings (CASSANDRA-9039)
* cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/70ee4ed4/src/java/org/apache/cassandra/repair/RepairRunnable.java
--
diff --cc src/java/org/apache/cassandra/repair/RepairRunnable.java
index 741cada,f92310b..21d0cd6
--- a/src/java/org/apache/cassandra/repair/RepairRunnable.java
+++ b/src/java/org/apache/cassandra/repair/RepairRunnable.java
@@@ -147,19 -146,22 +147,24 @@@ public class RepairRunnable extends Wra
}
final Set allNeighbors = new HashSet<>();
-Map> rangeToNeighbors = new HashMap<>();
+List, ? extends Collection>>>
commonRanges = new ArrayList<>();
+
+ //pre-calculate output of getLocalRanges and pass it to getNeighbors
to increase performance and prevent
+ //calculation multiple times
+ Collection> keyspaceLocalRanges =
storageService.getLocalRanges(keyspace);
+
try
{
for (Range range : options.getRanges())
{
- Set neighbors =
ActiveRepairService.getNeighbors(keyspace, range,
-Set neighbors =
ActiveRepairService.getNeighbors(keyspace, keyspaceLocalRanges,
-
range, options.getDataCenters(),
-
options.getHosts());
-rangeToNeighbors.put(range, neighbors);
-allNeighbors.addAll(neighbors);
++Set neighbors =
ActiveRepairService.getNeighbors(keyspace, keyspaceLocalRanges, range,
+
options.getDataCenters(),
+
[06/10] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c2566d1c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c2566d1c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c2566d1c Branch: refs/heads/cassandra-2.2 Commit: c2566d1cf9e239063a530ec08e8e098feffe387b Parents: 52a827b 2f74831 Author: Robert Stupp Authored: Wed Jun 15 11:44:57 2016 +0200 Committer: Robert Stupp Committed: Wed Jun 15 11:44:57 2016 +0200 -- CHANGES.txt | 1 + .../apache/cassandra/repair/RepairRunnable.java | 9 +++-- .../cassandra/service/ActiveRepairService.java| 5 +++-- .../service/ActiveRepairServiceTest.java | 18 ++ 4 files changed, 21 insertions(+), 12 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2566d1c/CHANGES.txt -- diff --cc CHANGES.txt index d0ca37f,ec2b48e..d9afaa3 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,30 -1,5 +1,31 @@@ -2.1.15 +2.2.7 + * StorageService shutdown hook should use a volatile variable (CASSANDRA-11984) + * Persist local metadata earlier in startup sequence (CASSANDRA-11742) + * Run CommitLog tests with different compression settings (CASSANDRA-9039) + * cqlsh: fix tab completion for case-sensitive identifiers (CASSANDRA-11664) + * Avoid showing estimated key as -1 in tablestats (CASSANDRA-11587) + * Fix possible race condition in CommitLog.recover (CASSANDRA-11743) + * Enable client encryption in sstableloader with cli options (CASSANDRA-11708) + * Possible memory leak in NIODataInputStream (CASSANDRA-11867) + * Fix commit log replay after out-of-order flush completion (CASSANDRA-9669) + * Add seconds to cqlsh tracing session duration (CASSANDRA-11753) + * Prohibit Reverse Counter type as part of the PK (CASSANDRA-9395) + * cqlsh: correctly handle non-ascii chars in error messages (CASSANDRA-11626) + * Exit JVM if JMX server fails to startup (CASSANDRA-11540) + * Produce a heap dump when exiting on OOM (CASSANDRA-9861) + * Avoid read repairing purgeable tombstones on range slices (CASSANDRA-11427) + * Restore ability to filter on clustering columns when using a 2i (CASSANDRA-11510) + * JSON datetime formatting needs timezone (CASSANDRA-11137) + * Fix is_dense recalculation for Thrift-updated tables (CASSANDRA-11502) + * Remove unnescessary file existence check during anticompaction (CASSANDRA-11660) + * Add missing files to debian packages (CASSANDRA-11642) + * Avoid calling Iterables::concat in loops during ModificationStatement::getFunctions (CASSANDRA-11621) + * cqlsh: COPY FROM should use regular inserts for single statement batches and + report errors correctly if workers processes crash on initialization (CASSANDRA-11474) + * Always close cluster with connection in CqlRecordWriter (CASSANDRA-11553) + * Fix slice queries on ordered COMPACT tables (CASSANDRA-10988) +Merged from 2.1: + * Cache local ranges when calculating repair neighbors (CASSANDRA-11933) * Allow LWT operation on static column with only partition keys (CASSANDRA-10532) * Create interval tree over canonical sstables to avoid missing sstables during streaming (CASSANDRA-11886) * cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting SSL connections (CASSANDRA-11749) http://git-wip-us.apache.org/repos/asf/cassandra/blob/c2566d1c/src/java/org/apache/cassandra/repair/RepairRunnable.java -- diff --cc src/java/org/apache/cassandra/repair/RepairRunnable.java index b849cf8,000..f92310b mode 100644,00..100644 --- a/src/java/org/apache/cassandra/repair/RepairRunnable.java +++ b/src/java/org/apache/cassandra/repair/RepairRunnable.java @@@ -1,409 -1,0 +1,414 @@@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.cassandra.repair; + +import java.net.InetA
[jira] [Commented] (CASSANDRA-10202) simplify CommitLogSegmentManager
[ https://issues.apache.org/jira/browse/CASSANDRA-10202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331458#comment-15331458 ] Branimir Lambov commented on CASSANDRA-10202: - It does have an elegance and is inherently simpler than the previous approach once we consider the complexities that one hides, but I agree that it is not as easy to follow. The expected benefit isn't that high either. Since we have more than one vote against them, I will revert Benedict's changes and rebase. bq. Why is the replay limit stuff necessary? Seems like it replays everything since the limit is set to be higher then anything that is available. Yes, but we also start the allocation thread which can create a segment and stall before listing it among the managed, causing a replay failure which happened way to often during tests. These changes are already committed as CASSANDRA-11743 and will not be present in the rebased version. > simplify CommitLogSegmentManager > > > Key: CASSANDRA-10202 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10202 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Jonathan Ellis >Assignee: Branimir Lambov >Priority: Minor > > Now that we only keep one active segment around we can simplify this from the > old recycling design. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[01/10] cassandra git commit: Cache local ranges when calculating repair neighbors
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 1d2d0749a -> 2f74831f4
refs/heads/cassandra-2.2 52a827b4d -> c2566d1cf
refs/heads/cassandra-3.0 3c531cbec -> 70ee4ed49
refs/heads/trunk 4ed00607d -> 4210a25a9
Cache local ranges when calculating repair neighbors
patch by Mahdi Mohammadi; reviewed by Paulo Motta for CASSANDRA-11933
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2f74831f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2f74831f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2f74831f
Branch: refs/heads/cassandra-2.1
Commit: 2f74831f4218142f6e118678a3c74c79c1f7e1ed
Parents: 1d2d074
Author: Mahdi Mohammadi
Authored: Wed Jun 15 11:43:27 2016 +0200
Committer: Robert Stupp
Committed: Wed Jun 15 11:43:27 2016 +0200
--
CHANGES.txt | 1 +
.../apache/cassandra/service/ActiveRepairService.java | 8 +---
.../org/apache/cassandra/service/StorageService.java | 6 +-
.../service/AntiEntropyServiceTestAbstract.java | 14 --
4 files changed, 19 insertions(+), 10 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 7d70902..ec2b48e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.15
+ * Cache local ranges when calculating repair neighbors (CASSANDRA-11933)
* Allow LWT operation on static column with only partition keys
(CASSANDRA-10532)
* Create interval tree over canonical sstables to avoid missing sstables
during streaming (CASSANDRA-11886)
* cqlsh COPY FROM: shutdown parent cluster after forking, to avoid corrupting
SSL connections (CASSANDRA-11749)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/ActiveRepairService.java
--
diff --git a/src/java/org/apache/cassandra/service/ActiveRepairService.java
b/src/java/org/apache/cassandra/service/ActiveRepairService.java
index f8975f9..4c83c48 100644
--- a/src/java/org/apache/cassandra/service/ActiveRepairService.java
+++ b/src/java/org/apache/cassandra/service/ActiveRepairService.java
@@ -164,7 +164,8 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
RepairFuture submitArtificialRepairSession(RepairJobDesc desc)
{
Set neighbours = new HashSet<>();
-neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
desc.range, null, null));
+Collection> keyspaceLocalRanges =
StorageService.instance.getLocalRanges(desc.keyspace);
+neighbours.addAll(ActiveRepairService.getNeighbors(desc.keyspace,
keyspaceLocalRanges, desc.range, null, null));
RepairSession session = new RepairSession(desc.parentSessionId,
desc.sessionId, desc.range, desc.keyspace, RepairParallelism.PARALLEL,
neighbours, new String[]{desc.columnFamily});
sessions.put(session.getId(), session);
RepairFuture futureTask = new RepairFuture(session);
@@ -176,17 +177,18 @@ public class ActiveRepairService implements
IEndpointStateChangeSubscriber, IFai
* Return all of the neighbors with whom we share the provided range.
*
* @param keyspaceName keyspace to repair
+ * @param keyspaceLocalRanges local-range for given keyspaceName
* @param toRepair token to repair
* @param dataCenters the data centers to involve in the repair
*
* @return neighbors with whom we share the provided range
*/
-public static Set getNeighbors(String keyspaceName,
Range toRepair, Collection dataCenters, Collection hosts)
+public static Set getNeighbors(String keyspaceName,
Collection> keyspaceLocalRanges, Range toRepair,
Collection dataCenters, Collection hosts)
{
StorageService ss = StorageService.instance;
Map, List> replicaSets =
ss.getRangeToAddressMap(keyspaceName);
Range rangeSuperSet = null;
-for (Range range : ss.getLocalRanges(keyspaceName))
+for (Range range : keyspaceLocalRanges)
{
if (range.contains(toRepair))
{
http://git-wip-us.apache.org/repos/asf/cassandra/blob/2f74831f/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index eea4556..27939f9 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2978,13 +2978,17 @@ public class StorageService
[jira] [Comment Edited] (CASSANDRA-11988) NullPointerExpception when reading/compacting table
[
https://issues.apache.org/jira/browse/CASSANDRA-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331441#comment-15331441
]
Marek edited comment on CASSANDRA-11988 at 6/15/16 9:30 AM:
Some more details to identify issue in 3.0.7:
- it happens only with static column
- it might be related to setting null to static column value
- I identified that
org.apache.cassandra.db.ReadCommand$1WithoutPurgeableTombstones transformation
returns null in org.apache.cassandra.db.transform.BaseRows.add(...) method so
staticRow is set to null
- There can be also some data corruption because we lost at least one partition
row after it happened
Here is naive fix to mitigate this issue:
{code}
diff --git a/src/java/org/apache/cassandra/db/transform/BaseRows.java
b/src/java/org/apache/cassandra/db/transform/BaseRows.java
index 78526e8..bc92b41 100644
--- a/src/java/org/apache/cassandra/db/transform/BaseRows.java
+++ b/src/java/org/apache/cassandra/db/transform/BaseRows.java
@@ -1,17 +1,24 @@
package org.apache.cassandra.db.transform;
+import static org.apache.cassandra.utils.Throwables.merge;
+
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.DecoratedKey;
import org.apache.cassandra.db.PartitionColumns;
-import org.apache.cassandra.db.rows.*;
-
-import static org.apache.cassandra.utils.Throwables.merge;
+import org.apache.cassandra.db.rows.BaseRowIterator;
+import org.apache.cassandra.db.rows.RangeTombstoneMarker;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.db.rows.Unfiltered;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public abstract class BaseRows>
extends BaseIterator
implements BaseRowIterator
{
+private static final Logger logger =
LoggerFactory.getLogger(BaseRows.class);
+
private Row staticRow;
public BaseRows(I input)
@@ -82,7 +89,14 @@ implements BaseRowIterator
super.add(transformation);
// transform any existing data
-staticRow = transformation.applyToStatic(staticRow);
+final Row afterTransform = transformation.applyToStatic(staticRow);
+
+if (staticRow != null && afterTransform == null)
+logger.debug("Transformation returns NULL value : " +
transformation);
+
+if (afterTransform != null)
+staticRow = afterTransform;
+
next = applyOne(next, transformation);
}
diff --git a/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
b/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
index 98640ae..e497bef 100644
--- a/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
+++ b/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
@@ -35,6 +35,9 @@ final class UnfilteredRows extends BaseRows i
@Override
public boolean isEmpty()
{
+if (staticRow() == null)
+return true;
+
return staticRow().isEmpty() && partitionLevelDeletion().isLive() &&
!hasNext();
}
}
{code}
was (Author: marekasf):
Some more details to identify issue:
- it happens only with static column
- it might be related to setting null to static column value
- I identified that
org.apache.cassandra.db.ReadCommand$1WithoutPurgeableTombstones transformation
returns null in org.apache.cassandra.db.transform.BaseRows.add(...) method so
staticRow is set to null
- There can be also some data corruption because we lost at least one partition
row after it happened
Here is naive fix to mitigate this issue:
{code}
diff --git a/src/java/org/apache/cassandra/db/transform/BaseRows.java
b/src/java/org/apache/cassandra/db/transform/BaseRows.java
index 78526e8..bc92b41 100644
--- a/src/java/org/apache/cassandra/db/transform/BaseRows.java
+++ b/src/java/org/apache/cassandra/db/transform/BaseRows.java
@@ -1,17 +1,24 @@
package org.apache.cassandra.db.transform;
+import static org.apache.cassandra.utils.Throwables.merge;
+
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.DecoratedKey;
import org.apache.cassandra.db.PartitionColumns;
-import org.apache.cassandra.db.rows.*;
-
-import static org.apache.cassandra.utils.Throwables.merge;
+import org.apache.cassandra.db.rows.BaseRowIterator;
+import org.apache.cassandra.db.rows.RangeTombstoneMarker;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.db.rows.Unfiltered;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public abstract class BaseRows>
extends BaseIterator
implements BaseRowIterator
{
+private static final Logger logger =
LoggerFactory.getLogger(BaseRows.class);
+
private Row staticRow;
public BaseRows(I input)
@@ -82,7 +89,14 @@ implements BaseRowIterator
super.add(transformation);
// transform any existing data
-staticRow = transformation.applyToStatic(staticRow);
+final Row afterTransfor
[jira] [Commented] (CASSANDRA-11988) NullPointerExpception when reading/compacting table
[
https://issues.apache.org/jira/browse/CASSANDRA-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331441#comment-15331441
]
Marek commented on CASSANDRA-11988:
---
Some more details to identify issue:
- it happens only with static column
- it might be related to setting null to static column value
- I identified that
org.apache.cassandra.db.ReadCommand$1WithoutPurgeableTombstones transformation
returns null in org.apache.cassandra.db.transform.BaseRows.add(...) method so
staticRow is set to null
- There can be also some data corruption because we lost at least one partition
row after it happened
Here is naive fix to mitigate this issue:
{code}
diff --git a/src/java/org/apache/cassandra/db/transform/BaseRows.java
b/src/java/org/apache/cassandra/db/transform/BaseRows.java
index 78526e8..bc92b41 100644
--- a/src/java/org/apache/cassandra/db/transform/BaseRows.java
+++ b/src/java/org/apache/cassandra/db/transform/BaseRows.java
@@ -1,17 +1,24 @@
package org.apache.cassandra.db.transform;
+import static org.apache.cassandra.utils.Throwables.merge;
+
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.db.DecoratedKey;
import org.apache.cassandra.db.PartitionColumns;
-import org.apache.cassandra.db.rows.*;
-
-import static org.apache.cassandra.utils.Throwables.merge;
+import org.apache.cassandra.db.rows.BaseRowIterator;
+import org.apache.cassandra.db.rows.RangeTombstoneMarker;
+import org.apache.cassandra.db.rows.Row;
+import org.apache.cassandra.db.rows.Unfiltered;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
public abstract class BaseRows>
extends BaseIterator
implements BaseRowIterator
{
+private static final Logger logger =
LoggerFactory.getLogger(BaseRows.class);
+
private Row staticRow;
public BaseRows(I input)
@@ -82,7 +89,14 @@ implements BaseRowIterator
super.add(transformation);
// transform any existing data
-staticRow = transformation.applyToStatic(staticRow);
+final Row afterTransform = transformation.applyToStatic(staticRow);
+
+if (staticRow != null && afterTransform == null)
+logger.debug("Transformation returns NULL value : " +
transformation);
+
+if (afterTransform != null)
+staticRow = afterTransform;
+
next = applyOne(next, transformation);
}
diff --git a/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
b/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
index 98640ae..e497bef 100644
--- a/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
+++ b/src/java/org/apache/cassandra/db/transform/UnfilteredRows.java
@@ -35,6 +35,9 @@ final class UnfilteredRows extends BaseRows i
@Override
public boolean isEmpty()
{
+if (staticRow() == null)
+return true;
+
return staticRow().isEmpty() && partitionLevelDeletion().isLive() &&
!hasNext();
}
}
{code}
> NullPointerExpception when reading/compacting table
> ---
>
> Key: CASSANDRA-11988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11988
> Project: Cassandra
> Issue Type: Bug
>Reporter: Nimi Wariboko Jr.
>Assignee: Carl Yeksigian
> Fix For: 3.6
>
>
> I have a table that suddenly refuses to be read or compacted. Issuing a read
> on the table causes a NPE.
> On compaction, it returns the error
> {code}
> ERROR [CompactionExecutor:6] 2016-06-09 17:10:15,724 CassandraDaemon.java:213
> - Exception in thread Thread[CompactionExecutor:6,1,main]
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:38)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:64)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:24)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:76)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.CompactionIterator.hasNext(CompactionIterator.java:226)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:182)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:82)
> ~[apache-cassandra-3.6.jar:3.6]
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:60)
> ~[apache-cassandra-3.6.jar:3.6]
[jira] [Updated] (CASSANDRA-11882) Clustering Key with ByteBuffer size > 64k throws Assertion Error
[
https://issues.apache.org/jira/browse/CASSANDRA-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Branimir Lambov updated CASSANDRA-11882:
Status: Ready to Commit (was: Patch Available)
> Clustering Key with ByteBuffer size > 64k throws Assertion Error
>
>
> Key: CASSANDRA-11882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11882
> Project: Cassandra
> Issue Type: Bug
> Components: CQL, Streaming and Messaging
>Reporter: Lerh Chuan Low
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 11882-2.1.txt, 11882-2.2.txt, 11882-3.X.txt
>
>
> Setup:
> {code}
> CREATE KEYSPACE Blues WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 2};
> CREATE TABLE test (a text, b text, PRIMARY KEY ((a), b))
> {code}
> There currently doesn't seem to be an existing check for selecting clustering
> keys that are larger than 64k. So if we proceed to do the following select:
> {code}
> CONSISTENCY ALL;
> SELECT * FROM Blues.test WHERE a = 'foo' AND b = 'something larger than 64k';
> {code}
> An AssertionError is thrown in `ByteBufferUtil` with just a number and an
> error message detailing 'Coordinator node timed out waiting for replica nodes
> responses' . Additionally, because an error extends Throwable (it's not a
> subclass of Exception), it's not caught so the connection between the
> coordinator node and the other nodes which have the replicas seem to be
> 'stuck' until it's restarted. Any other subsequent queries, even if it's just
> SELECT where a = 'foo' and b = 'bar', will always return the Coordinator
> timing out waiting for replica nodes responses'.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11882) Clustering Key with ByteBuffer size > 64k throws Assertion Error
[
https://issues.apache.org/jira/browse/CASSANDRA-11882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331435#comment-15331435
]
Branimir Lambov commented on CASSANDRA-11882:
-
Updated the branches above with the latest version. Tests are now passing (to
the extent that their respective base version are) and I'm happy with the code.
Marking ready to commit.
> Clustering Key with ByteBuffer size > 64k throws Assertion Error
>
>
> Key: CASSANDRA-11882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11882
> Project: Cassandra
> Issue Type: Bug
> Components: CQL, Streaming and Messaging
>Reporter: Lerh Chuan Low
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 11882-2.1.txt, 11882-2.2.txt, 11882-3.X.txt
>
>
> Setup:
> {code}
> CREATE KEYSPACE Blues WITH REPLICATION = { 'class' : 'SimpleStrategy',
> 'replication_factor' : 2};
> CREATE TABLE test (a text, b text, PRIMARY KEY ((a), b))
> {code}
> There currently doesn't seem to be an existing check for selecting clustering
> keys that are larger than 64k. So if we proceed to do the following select:
> {code}
> CONSISTENCY ALL;
> SELECT * FROM Blues.test WHERE a = 'foo' AND b = 'something larger than 64k';
> {code}
> An AssertionError is thrown in `ByteBufferUtil` with just a number and an
> error message detailing 'Coordinator node timed out waiting for replica nodes
> responses' . Additionally, because an error extends Throwable (it's not a
> subclass of Exception), it's not caught so the connection between the
> coordinator node and the other nodes which have the replicas seem to be
> 'stuck' until it's restarted. Any other subsequent queries, even if it's just
> SELECT where a = 'foo' and b = 'bar', will always return the Coordinator
> timing out waiting for replica nodes responses'.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7461) operator functionality in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331392#comment-15331392
]
Benjamin Lerer commented on CASSANDRA-7461:
---
It is good that you mentioned it. :-)
I have added support for parenthesis in the patch I am currently working on but
I did not think about testing {{((someType)?).bar}}.
> operator functionality in CQL
> -
>
> Key: CASSANDRA-7461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7461
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Robert Stupp
>Assignee: Benjamin Lerer
> Labels: cql
>
> Intention: Allow operators in CQL
> Operators could be decimal arithmetics {{+ - * /}} or boolen arithmetics {{|
> & !}} or string 'arithmetics' {{+}}
> {{SELECT tab.label + ' = ' + tab.value FROM foo.tab}}
> {{SELECT * FROM tab WHERE tab.label + ' = ' + tab.value = 'foo = bar'}}
> as well as
> {{CREATE INDEX idx ON tab ( tab.tabel + '=' + tab.value )}}
> or
> {{CREATE INDEX idx ON tab (label) WHERE contains(tab.tabel,
> 'very-important-key')}}
> Operators could be mapped to UDFs like this:
> {{+}} mapped to UDF {{cstarstd::oper_plus(...)}}
> {{-}} mapped to UDF {{cstarstd::oper_minus(...)}}
> or handled directly via {{Cql.g}} in 'special' code
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7461) operator functionality in CQL
[
https://issues.apache.org/jira/browse/CASSANDRA-7461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331340#comment-15331340
]
Sylvain Lebresne commented on CASSANDRA-7461:
-
Btw, mentioning this because I don't think it has been so far, but I think
supporting operators also imply we start supporting parenthesis for terms (or
you get pretty quickly limited). I'll note that having parenthesis in terms is
imo a good idea for clarity anyway (even ignoring operators), as, for instance,
something like {{((someType)?).bar}} looks cleaner than {{(someType)?.bar}}
(even though they will be equivalent) and it would be nice to allow it.
> operator functionality in CQL
> -
>
> Key: CASSANDRA-7461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7461
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL
>Reporter: Robert Stupp
>Assignee: Benjamin Lerer
> Labels: cql
>
> Intention: Allow operators in CQL
> Operators could be decimal arithmetics {{+ - * /}} or boolen arithmetics {{|
> & !}} or string 'arithmetics' {{+}}
> {{SELECT tab.label + ' = ' + tab.value FROM foo.tab}}
> {{SELECT * FROM tab WHERE tab.label + ' = ' + tab.value = 'foo = bar'}}
> as well as
> {{CREATE INDEX idx ON tab ( tab.tabel + '=' + tab.value )}}
> or
> {{CREATE INDEX idx ON tab (label) WHERE contains(tab.tabel,
> 'very-important-key')}}
> Operators could be mapped to UDFs like this:
> {{+}} mapped to UDF {{cstarstd::oper_plus(...)}}
> {{-}} mapped to UDF {{cstarstd::oper_minus(...)}}
> or handled directly via {{Cql.g}} in 'special' code
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11873) Add duration type
[
https://issues.apache.org/jira/browse/CASSANDRA-11873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331333#comment-15331333
]
Benjamin Lerer commented on CASSANDRA-11873:
I am not sure of what {{date1}} and {{date2}} would be as Cassandra do not
support column identifiers at this place in the {{WHERE}} clause but otherwise
the query will be:
{code}
WHERE T >= (NOW() - 1d) + (date1 - date2);
{code}
which seems much more readable to me.
One problem with your proposal is the {{timeunit}} parameter. Functions only
accept arguments associated with a CQL type to be able to deserialize the input.
> Add duration type
> -
>
> Key: CASSANDRA-11873
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11873
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
> Labels: client-impacting, doc-impacting
> Fix For: 3.x
>
>
> For CASSANDRA-11871 or to allow queries with {{WHERE}} clause like:
> {{... WHERE reading_time < now() - 2h}}, we need to support some duration
> type.
> In my opinion, it should be represented internally as a number of
> microseconds.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10783) Allow literal value as parameter of UDF & UDA
[
https://issues.apache.org/jira/browse/CASSANDRA-10783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-10783:
-
Resolution: Fixed
Assignee: Sylvain Lebresne (was: Robert Stupp)
Fix Version/s: (was: 3.x)
3.8
Status: Resolved (was: Ready to Commit)
Committed, thanks
> Allow literal value as parameter of UDF & UDA
> -
>
> Key: CASSANDRA-10783
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10783
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: DOAN DuyHai
>Assignee: Sylvain Lebresne
>Priority: Minor
> Labels: CQL3, UDF, client-impacting, doc-impacting
> Fix For: 3.8
>
>
> I have defined the following UDF
> {code:sql}
> CREATE OR REPLACE FUNCTION maxOf(current int, testValue int) RETURNS NULL ON
> NULL INPUT
> RETURNS int
> LANGUAGE java
> AS 'return Math.max(current,testValue);'
> CREATE TABLE maxValue(id int primary key, val int);
> INSERT INTO maxValue(id, val) VALUES(1, 100);
> SELECT maxOf(val, 101) FROM maxValue WHERE id=1;
> {code}
> I got the following error message:
> {code}
> SyntaxException: message="line 1:19 no viable alternative at input '101' (SELECT maxOf(val1,
> [101]...)">
> {code}
> It would be nice to allow literal value as parameter of UDF and UDA too.
> I was thinking about an use-case for an UDA groupBy() function where the end
> user can *inject* at runtime a literal value to select which aggregation he
> want to display, something similar to GROUP BY ... HAVING
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[4/4] cassandra git commit: Allow terms in selection clauses
Allow terms in selection clauses patch by slebresne; reviewed by blerer for CASSANDRA-10783 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4ed00607 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4ed00607 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4ed00607 Branch: refs/heads/trunk Commit: 4ed00607df060a3cdcc685c4fc0c1967f62b37f1 Parents: 60e5e0e Author: Sylvain Lebresne Authored: Tue May 24 15:49:48 2016 +0200 Committer: Sylvain Lebresne Committed: Wed Jun 15 09:33:06 2016 +0200 -- CHANGES.txt | 1 + NEWS.txt| 10 + doc/cql3/CQL.textile| 8 +- pylib/cqlshlib/cql3handling.py | 1 + src/antlr/Cql.g | 1 + src/antlr/Parser.g | 103 +++--- .../cassandra/config/ColumnDefinition.java | 202 ++- .../apache/cassandra/config/ViewDefinition.java | 4 +- .../apache/cassandra/cql3/AbstractMarker.java | 7 + .../apache/cassandra/cql3/ColumnCondition.java | 40 +-- .../apache/cassandra/cql3/ColumnIdentifier.java | 143 +--- .../org/apache/cassandra/cql3/Constants.java| 24 +- .../apache/cassandra/cql3/FieldIdentifier.java | 98 ++ src/java/org/apache/cassandra/cql3/Json.java| 11 + src/java/org/apache/cassandra/cql3/Lists.java | 13 + src/java/org/apache/cassandra/cql3/Maps.java| 19 +- .../cassandra/cql3/MultiColumnRelation.java | 20 +- .../org/apache/cassandra/cql3/Operation.java| 41 ++- .../org/apache/cassandra/cql3/Relation.java | 27 +- src/java/org/apache/cassandra/cql3/Sets.java| 12 + .../cassandra/cql3/SingleColumnRelation.java| 26 +- src/java/org/apache/cassandra/cql3/Term.java| 14 +- .../apache/cassandra/cql3/TokenRelation.java| 16 +- src/java/org/apache/cassandra/cql3/Tuples.java | 24 ++ .../org/apache/cassandra/cql3/TypeCast.java | 5 + .../org/apache/cassandra/cql3/UserTypes.java| 44 +-- .../cassandra/cql3/functions/FunctionCall.java | 10 + .../selection/AbstractFunctionSelector.java | 23 +- .../selection/AggregateFunctionSelector.java| 4 +- .../cassandra/cql3/selection/FieldSelector.java | 11 +- .../cql3/selection/ScalarFunctionSelector.java | 6 +- .../cassandra/cql3/selection/Selectable.java| 350 ++- .../cassandra/cql3/selection/Selection.java | 36 +- .../cassandra/cql3/selection/Selector.java | 32 +- .../cql3/selection/SelectorFactories.java | 31 +- .../cql3/selection/SimpleSelector.java | 3 +- .../cassandra/cql3/selection/TermSelector.java | 92 + .../cql3/selection/WritetimeOrTTLSelector.java | 3 +- .../cql3/statements/AlterTableStatement.java| 23 +- .../statements/AlterTableStatementColumn.java | 24 +- .../cql3/statements/AlterTypeStatement.java | 44 +-- .../cql3/statements/CreateTypeStatement.java| 18 +- .../cql3/statements/CreateViewStatement.java| 53 ++- .../cql3/statements/DeleteStatement.java| 2 +- .../cassandra/cql3/statements/IndexTarget.java | 22 +- .../cql3/statements/ModificationStatement.java | 24 +- .../cql3/statements/SelectStatement.java| 84 ++--- .../cql3/statements/UpdateStatement.java| 16 +- .../cassandra/db/marshal/AbstractType.java | 24 +- .../apache/cassandra/db/marshal/TypeParser.java | 7 +- .../apache/cassandra/db/marshal/UserType.java | 41 +-- src/java/org/apache/cassandra/db/view/View.java | 4 +- .../exceptions/UnrecognizedEntityException.java | 49 --- .../cassandra/schema/LegacySchemaMigrator.java | 5 +- .../apache/cassandra/schema/SchemaKeyspace.java | 2 +- src/java/org/apache/cassandra/schema/Types.java | 5 +- .../apache/cassandra/transport/DataType.java| 11 +- .../cassandra/cql3/CQL3TypeLiteralTest.java | 4 +- .../org/apache/cassandra/cql3/CQLTester.java| 5 + .../cql3/selection/TermSelectionTest.java | 338 ++ .../validation/operations/AggregationTest.java | 6 +- .../cql3/validation/operations/DeleteTest.java | 16 +- .../cql3/validation/operations/InsertTest.java | 16 +- .../SelectMultiColumnRelationTest.java | 12 +- .../SelectOrderedPartitionerTest.java | 8 +- .../SelectSingleColumnRelationTest.java | 22 +- .../cql3/validation/operations/SelectTest.java | 8 +- .../cql3/validation/operations/UpdateTest.java | 12 +- .../schema/LegacySchemaMigratorTest.java| 22 +- .../cassandra/transport/SerDeserTest.java | 18 +- 70 files changed, 1631 insertions(+), 829 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ed00607/CHANGES.txt --
[2/4] cassandra git commit: Allow terms in selection clauses
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ed00607/src/java/org/apache/cassandra/cql3/statements/DeleteStatement.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/DeleteStatement.java
b/src/java/org/apache/cassandra/cql3/statements/DeleteStatement.java
index 7b329ac..26b25de 100644
--- a/src/java/org/apache/cassandra/cql3/statements/DeleteStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/DeleteStatement.java
@@ -122,7 +122,7 @@ public class DeleteStatement extends ModificationStatement
Attributes.Raw attrs,
List deletions,
WhereClause whereClause,
- List>
conditions,
+ List>
conditions,
boolean ifExists)
{
super(name, StatementType.DELETE, attrs, conditions, false,
ifExists);
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ed00607/src/java/org/apache/cassandra/cql3/statements/IndexTarget.java
--
diff --git a/src/java/org/apache/cassandra/cql3/statements/IndexTarget.java
b/src/java/org/apache/cassandra/cql3/statements/IndexTarget.java
index 8cdf2c8..9756a4c 100644
--- a/src/java/org/apache/cassandra/cql3/statements/IndexTarget.java
+++ b/src/java/org/apache/cassandra/cql3/statements/IndexTarget.java
@@ -68,36 +68,36 @@ public class IndexTarget
public static class Raw
{
-private final ColumnIdentifier.Raw column;
+private final ColumnDefinition.Raw column;
private final Type type;
-private Raw(ColumnIdentifier.Raw column, Type type)
+private Raw(ColumnDefinition.Raw column, Type type)
{
this.column = column;
this.type = type;
}
-public static Raw simpleIndexOn(ColumnIdentifier.Raw c)
+public static Raw simpleIndexOn(ColumnDefinition.Raw c)
{
return new Raw(c, Type.SIMPLE);
}
-public static Raw valuesOf(ColumnIdentifier.Raw c)
+public static Raw valuesOf(ColumnDefinition.Raw c)
{
return new Raw(c, Type.VALUES);
}
-public static Raw keysOf(ColumnIdentifier.Raw c)
+public static Raw keysOf(ColumnDefinition.Raw c)
{
return new Raw(c, Type.KEYS);
}
-public static Raw keysAndValuesOf(ColumnIdentifier.Raw c)
+public static Raw keysAndValuesOf(ColumnDefinition.Raw c)
{
return new Raw(c, Type.KEYS_AND_VALUES);
}
-public static Raw fullCollection(ColumnIdentifier.Raw c)
+public static Raw fullCollection(ColumnDefinition.Raw c)
{
return new Raw(c, Type.FULL);
}
@@ -109,13 +109,9 @@ public class IndexTarget
// same syntax as an index on a regular column (i.e. the 'values'
in
// 'CREATE INDEX on table(values(collection));' is optional). So
we correct the target type
// when the target column is a collection & the target type is
SIMPLE.
-ColumnIdentifier colId = column.prepare(cfm);
-ColumnDefinition columnDef = cfm.getColumnDefinition(colId);
-if (columnDef == null)
-throw new InvalidRequestException("No column definition found
for column " + colId);
-
+ColumnDefinition columnDef = column.prepare(cfm);
Type actualType = (type == Type.SIMPLE &&
columnDef.type.isCollection()) ? Type.VALUES : type;
-return new IndexTarget(colId, actualType);
+return new IndexTarget(columnDef.name, actualType);
}
}
http://git-wip-us.apache.org/repos/asf/cassandra/blob/4ed00607/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
--
diff --git
a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
index af89ba8..0a9be0c 100644
--- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
+++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java
@@ -27,9 +27,9 @@ import org.slf4j.LoggerFactory;
import org.apache.cassandra.auth.Permission;
import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.ColumnDefinition;
+import org.apache.cassandra.config.ColumnDefinition.Raw;
import org.apache.cassandra.config.ViewDefinition;
import org.apache.cassandra.cql3.*;
-import org.apache.cassandra.cql3.ColumnIdentifier.Raw;
import org.apache.cassandra.cql3.functions.Function;
import org.apache.cassandra.cql3.restrictions.StatementRestrictions;
import org.apache.cassandra.cql3.selection.Selection;
@@ -549,10 +549,10 @@ public abstract class ModificationStatement imple
