[jira] [Commented] (CASSANDRA-7576) DateType columns not properly converted to TimestampType when in ReversedType columns.
[
https://issues.apache.org/jira/browse/CASSANDRA-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068328#comment-14068328
]
Ben Hood commented on CASSANDRA-7576:
-
FWIW, you could verify this patch by removing this workaround
(https://github.com/gocql/gocql/pull/154) that we put into place to deal with
this issue in gocql.
> DateType columns not properly converted to TimestampType when in ReversedType
> columns.
> --
>
> Key: CASSANDRA-7576
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7576
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Karl Rieb
> Attachments: DataType_CASSANDRA_7576.patch
>
> Original Estimate: 0.25h
> Remaining Estimate: 0.25h
>
> The {{org.apache.cassandra.transport.DataType.fromType(AbstractType)}} method
> has a bug that prevents sending the correct Protocol ID for reversed
> {{DateType}} columns. This results in clients receiving Protocol ID {{0}},
> which maps to a {{CUSTOM}} type, for timestamp columns that are clustered in
> reverse order.
> Some clients can handle this properly since they recognize the
> {{"org.apache.cassandra.db.marshal.DateType"}} marshaling type, however the
> native Datastax java-driver does not. It will produce errors like the one
> below when trying to prepare queries against such tables:
> {noformat}
> com.datastax.driver.core.exceptions.InvalidTypeException: Invalid type
> for value 2 of CQL type 'org.apache.cassandra.db.marshal.DateType', expecting
> class java.nio.ByteBuffer but class java.util.Date provided
> at com.datastax.driver.core.BoundStatement.bind(BoundStatement.java:190)
> at
> com.datastax.driver.core.DefaultPreparedStatement.bind(DefaultPreparedStatement.java:103)
> {noformat}
> On the Cassandra side, there is a check for {{DateType}} columns that is
> supposed to convert these columns to TimestampType. However, the check is
> skipped when the column is also reversed. Specifically:
> {code:title=DataType.java|borderStyle=solid}
> public static Pair fromType(AbstractType type)
> {
> // For CQL3 clients, ReversedType is an implementation detail and they
> // shouldn't have to care about it.
> if (type instanceof ReversedType)
> type = ((ReversedType)type).baseType;
> // For compatibility sake, we still return DateType as the timestamp type
> in resultSet metadata (#5723)
> else if (type instanceof DateType)
> type = TimestampType.instance;
> // ...
> {code}
> The *else if* should be changed to just an *if*, like so:
> {code:title=DataType.java|borderStyle=solid}
> public static Pair fromType(AbstractType type)
> {
> // For CQL3 clients, ReversedType is an implementation detail and they
> // shouldn't have to care about it.
> if (type instanceof ReversedType)
> type = ((ReversedType)type).baseType;
> // For compatibility sake, we still return DateType as the timestamp type
> in resultSet metadata (#5723)
> if (type instanceof DateType)
> type = TimestampType.instance;
> // ...
> {code}
> This bug is preventing us from upgrading our 1.2.11 cluster to 2.0.9 because
> our clients keep throwing exceptions trying to read or write data to tables
> with reversed timestamp columns. This issue can be reproduced by creating a
> CQL table in Cassandra 1.2.11 that clusters on a timestamp in reverse, then
> upgrading the node to 2.0.9. When querying the metadata for the table, the
> node will return Protocol ID 0 (CUSTOM) instead of Protocol ID 11 (TIMESTAMP).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7454) NPE When Prepared Statement ID is not Found
[
https://issues.apache.org/jira/browse/CASSANDRA-7454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068366#comment-14068366
]
Ben Hood commented on CASSANDRA-7454:
-
I can confirm that this fix (as applied to 2.1-rc4) has resolved part of the an
issue reported in https://issues.apache.org/jira/browse/CASSANDRA-7566.
When I run the the TestReprepareStatement test case from the gocql integration
suite against the 2.1-rc3 tarball, it fails:
{code}
bool:gocql 0x6e6562$ go test . -v -test.run=TestReprepareStatement
=== RUN TestReprepareStatement
--- FAIL: TestReprepareStatement (0.31 seconds)
cassandra_test.go:588: Failed to execute query for reprepare statement:
java.lang.NullPointerException
FAIL
exit status 1
{code}
But when I run it against the 2.1-rc4 tarball, it passes:
{code}
bool:gocql 0x6e6562$ go test . -v -test.run=TestReprepareStatement
=== RUN TestReprepareStatement
--- PASS: TestReprepareStatement (0.25 seconds)
PASS
ok github.com/relops/gocql 0.267s
{code}
> NPE When Prepared Statement ID is not Found
> ---
>
> Key: CASSANDRA-7454
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7454
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Tyler Hobbs
> Fix For: 2.1.0
>
> Attachments: 7475.txt
>
>
> CASSANDRA-6855 introduced a NullPointerException when an unknown prepared
> statement ID is used.
> You'll see a stack trace like this:
> {noformat}
> ERROR [SharedPool-Worker-4] 2014-06-26 15:02:04,911 ErrorMessage.java:218 -
> Unexpected exception during request
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.transport.messages.ExecuteMessage.execute(ExecuteMessage.java:105)
> ~[main/:na]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:412)
> [main/:na]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:309)
> [main/:na]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:103)
> [netty-all-4.0.20.Final.jar:4.0.20.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:332)
> [netty-all-4.0.20.Final.jar:4.0.20.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:31)
> [netty-all-4.0.20.Final.jar:4.0.20.Final]
> at
> io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:323)
> [netty-all-4.0.20.Final.jar:4.0.20.Final]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> [na:1.7.0_40]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:162)
> [main/:na]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:103)
> [main/:na]
> at java.lang.Thread.run(Thread.java:724) [na:1.7.0_40]
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[ https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068388#comment-14068388 ] Benedict commented on CASSANDRA-7507: - Unfortunately it looks like we have a *lot* of places where we catch IOException and do something specific. I'd say the safest thing to do is create a special method for handling special exceptions that we call any time we hit an IOException, which will ordinarily do nothing, but in case of 'too many open files' (or anything else we later decide to include) performs a sys exit > OOM creates unreliable state - die instantly better > --- > > Key: CASSANDRA-7507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7507 > Project: Cassandra > Issue Type: New Feature >Reporter: Karl Mueller >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7507_v1.txt > > > I had a cassandra node run OOM. My heap had enough headroom, there was just > something which either was a bug or some unfortunate amount of short-term > memory utilization. This resulted in the following error: > WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java > (line 1713) Some hints were not written before shutdown. This is not > supposed to happen. You should (a) run repair, and (b) file a bug report > There are no other messages of relevance besides the OOM error about 90 > minutes earlier. > My (limited) understanding of the JVM and Cassandra says that when it goes > OOM, it will attempt to signal cassandra to shut down "cleanly". The problem, > in my view, is that with an OOM situation, nothing is guaranteed anymore. I > believe it's impossible to reliably "cleanly shut down" at this point, and > therefore it's wrong to even try. > Yes, ideally things could be written out, flushed to disk, memory messages > written, other nodes notified, etc. but why is there any reason to believe > any of those steps could happen? Would happen? Couldn't bad data be written > at this point to disk rather than good data? Some network messages delivered, > but not others? > I think Cassandra should have the option to (and possibly default) to kill > itself immediately upon the OOM condition happening in a hard way, and not > rely on the java-based clean shutdown process. Cassandra already handles > recovery from unclean shutdown, and it's not a big deal. My node, for > example, kept in a sort-of alive state for 90 minutes where who knows what it > was doing or not doing. > I don't know enough about the JVM and options for it to know the best exact > implementation of "die instantly on OOM", but it should be something that's > possible either with some flags or a C library (which doesn't rely on java > memory to do something which it may not be able to get!) > Short version: a kill -9 of all C* processes in that instance without needing > more java memory, when OOM is raised -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6851) Improve anticompaction after incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068435#comment-14068435 ] Marcus Eriksson commented on CASSANDRA-6851: in general this looks good to me, pushed a branch with a few updates: https://github.com/krummas/cassandra/commits/russellspitzer/6851 it fixes a few nits (style mostly) and tries to pick better sstables by sorting them by first key in the sstable for standard grouping we should probably target trunk for this as well > Improve anticompaction after incremental repair > --- > > Key: CASSANDRA-6851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6851 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Marcus Eriksson >Assignee: Russell Alexander Spitzer >Priority: Minor > Labels: compaction, lhf > Fix For: 2.1.1 > > > After an incremental repair we iterate over all sstables and split them in > two parts, one containing the repaired data and one the unrepaired. We could > in theory double the number of sstables on a node. > To avoid this we could make anticompaction also do a compaction, for example, > if we are to anticompact 10 sstables, we could anticompact those to 2. > Note that we need to avoid creating too big sstables though, if we > anticompact all sstables on a node it would essentially be a major compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6851) Improve anticompaction after incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-6851: --- Fix Version/s: (was: 2.1.1) 3.0 > Improve anticompaction after incremental repair > --- > > Key: CASSANDRA-6851 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6851 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: Marcus Eriksson >Assignee: Russell Alexander Spitzer >Priority: Minor > Labels: compaction, lhf > Fix For: 3.0 > > > After an incremental repair we iterate over all sstables and split them in > two parts, one containing the repaired data and one the unrepaired. We could > in theory double the number of sstables on a node. > To avoid this we could make anticompaction also do a compaction, for example, > if we are to anticompact 10 sstables, we could anticompact those to 2. > Note that we need to avoid creating too big sstables though, if we > anticompact all sstables on a node it would essentially be a major compaction. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068437#comment-14068437 ] Marcus Eriksson commented on CASSANDRA-6434: Blocks on CASSANDRA-5839 - once we have the repair history we can figure out actual repair times for keys > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (CASSANDRA-5839) Save repair data to system table
[ https://issues.apache.org/jira/browse/CASSANDRA-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson reassigned CASSANDRA-5839: -- Assignee: Marcus Eriksson (was: Jimmy Mårdell) > Save repair data to system table > > > Key: CASSANDRA-5839 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5839 > Project: Cassandra > Issue Type: New Feature > Components: Core, Tools >Reporter: Jonathan Ellis >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.0.10 > > Attachments: 2.0.4-5839-draft.patch, 2.0.6-5839-v2.patch > > > As noted in CASSANDRA-2405, it would be useful to store repair results, > particularly with sub-range repair available (CASSANDRA-5280). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7443) SSTable Pluggability v2
[ https://issues.apache.org/jira/browse/CASSANDRA-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068458#comment-14068458 ] Benedict commented on CASSANDRA-7443: - First impression is pretty positive - this is looking a lot cleaner than I might have hoped. A few initial comments, obviously bearing in mind it's still in progress: * I'm not super keen on using Class.getConstructor() for instantiating a reader/writer. Would prefer a factory method / class; using reflection for long type parameter lists worries me. * Would prefer DatabaseDescriptor.getDefaultSSTableFormat, and permit overrides per Table * LazilyCompactedRow is still tightly coupled with Big format (relates to AbstractCompactedRow as you mentioned) * Streaming _may_ need a bit more work/thought - seems that compression format of sstables is coupled quite tightly with compression stream writer, and also assumes we can stream a single file range. Might not want to impose that requirement (we probably don't for 3.0 format - a set of ranges is more likely, but I would prefer to abstract the concept of a stream Chunk to be format specific anyway, to remove the coupling) > SSTable Pluggability v2 > --- > > Key: CASSANDRA-7443 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7443 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: T Jake Luciani >Assignee: T Jake Luciani > Fix For: 3.0 > > > As part of a wider effort to improve the performance of our storage engine we > will need to support basic pluggability of the SSTable reader/writer. We > primarily need this to support the current SSTable format and new SSTable > format in the same version. This will also let us encapsulate the changes in > a single layer vs forcing the whole engine to change at once. > We previously discussed how to accomplish this in CASSANDRA-3067 > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-6751) Setting -Dcassandra.fd_initial_value_ms Results in NPE
[
https://issues.apache.org/jira/browse/CASSANDRA-6751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-6751:
Fix Version/s: 2.0.8
> Setting -Dcassandra.fd_initial_value_ms Results in NPE
> --
>
> Key: CASSANDRA-6751
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6751
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Tyler Hobbs
>Assignee: Dave Brosius
>Priority: Minor
> Fix For: 1.2.17, 2.0.8
>
> Attachments: 6751.txt
>
>
> Start Cassandra with {{-Dcassandra.fd_initial_value_ms=1000}} and you'll get
> the following stacktrace:
> {noformat}
> INFO [main] 2014-02-21 14:45:57,731 StorageService.java (line 617) Starting
> up server gossip
> ERROR [main] 2014-02-21 14:45:57,736 CassandraDaemon.java (line 464)
> Exception encountered during startup
> java.lang.ExceptionInInitializerError
> at org.apache.cassandra.gms.Gossiper.(Gossiper.java:178)
> at org.apache.cassandra.gms.Gossiper.(Gossiper.java:71)
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:618)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:583)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:480)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:348)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:447)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:490)
> Caused by: java.lang.NullPointerException
> at
> org.apache.cassandra.gms.FailureDetector.getInitialValue(FailureDetector.java:81)
> at
> org.apache.cassandra.gms.FailureDetector.(FailureDetector.java:48)
> ... 8 more
> ERROR [StorageServiceShutdownHook] 2014-02-21 14:45:57,754
> CassandraDaemon.java (line 191) Exception in thread
> Thread[StorageServiceShutdownHook,5,main]
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.cassandra.gms.Gossiper
> at
> org.apache.cassandra.service.StorageService$1.runMayThrow(StorageService.java:550)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at java.lang.Thread.run(Thread.java:724)
> {noformat}
> Glancing at the code, this is because the FailureDetector logger isn't
> initialized when the static initialization of {{INITIAL_VALUE}} happens.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[06/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095
Branch: refs/heads/cassandra-2.1
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams
Committed: Mon Jul 21 07:49:13 2014 -0500
--
conf/cassandra-env.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
java_ver_output=`"${JAVA:-java}" -version 2>&1`
-jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
+jvmver=`echo "$java_ver_output" | grep 'java version' | awk -F'"' 'NR==1
{print $2}'`
JVM_VERSION=${jvmver%_*}
JVM_PATCH_VERSION=${jvmver#*_}
@@ -100,7 +100,7 @@ if [ "$JVM_VERSION" \< "1.7" ] ; then
fi
-jvm=`echo "$java_ver_output" | awk 'NR==2 {print $1}'`
+jvm=`echo "$java_ver_output" | grep -A 1 'java version' | awk 'NR==2 {print
$1}'`
case "$jvm" in
OpenJDK)
JVM_VENDOR=OpenJDK
[11/12] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3b14f6a5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3b14f6a5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3b14f6a5 Branch: refs/heads/cassandra-2.1 Commit: 3b14f6a5dbc91b3e8d2efa95840a61604dd6fbcb Parents: 82fa5d5 df7bd7f Author: Brandon Williams Authored: Mon Jul 21 07:51:10 2014 -0500 Committer: Brandon Williams Committed: Mon Jul 21 07:51:10 2014 -0500 -- CHANGES.txt | 2 +- build.xml | 2 +- conf/cassandra-env.sh | 4 ++-- debian/changelog | 6 ++ 4 files changed, 10 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/CHANGES.txt -- diff --cc CHANGES.txt index 8fa96f6,5ba30cc..2aa2efa --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,15 -1,4 +1,15 @@@ +2.1.1 + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Fail to start if commit log replay detects a problem (CASSANDRA-7125) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) + * Fix native protocol drop user type notification (CASSANDRA-7571) + + - 2.1.0-final + 2.1.0-rc4 * Updated memtable_cleanup_threshold and memtable_flush_writers defaults (CASSANDRA-7551) * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505) http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/build.xml -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/conf/cassandra-env.sh --
[09/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0
Merge branch 'cassandra-2.0' into cassandra-2.1.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df7bd7f7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df7bd7f7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df7bd7f7 Branch: refs/heads/trunk Commit: df7bd7f725f6df5102730de89647418dd9f9473d Parents: d872e2c 8754809 Author: Brandon Williams Authored: Mon Jul 21 07:50:18 2014 -0500 Committer: Brandon Williams Committed: Mon Jul 21 07:50:18 2014 -0500 -- conf/cassandra-env.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/df7bd7f7/conf/cassandra-env.sh --
[07/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0
Merge branch 'cassandra-2.0' into cassandra-2.1.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df7bd7f7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df7bd7f7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df7bd7f7 Branch: refs/heads/cassandra-2.1 Commit: df7bd7f725f6df5102730de89647418dd9f9473d Parents: d872e2c 8754809 Author: Brandon Williams Authored: Mon Jul 21 07:50:18 2014 -0500 Committer: Brandon Williams Committed: Mon Jul 21 07:50:18 2014 -0500 -- conf/cassandra-env.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/df7bd7f7/conf/cassandra-env.sh --
[02/12] git commit: update versioning for 2.1.0-rc4 release
update versioning for 2.1.0-rc4 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d872e2c7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d872e2c7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d872e2c7 Branch: refs/heads/trunk Commit: d872e2c74685c0c8c019b2ef46aefa443efa95b9 Parents: 8fd8832 Author: Eric Evans Authored: Fri Jul 18 13:42:11 2014 -0500 Committer: Eric Evans Committed: Fri Jul 18 13:42:11 2014 -0500 -- CHANGES.txt | 2 +- build.xml| 2 +- debian/changelog | 6 ++ 3 files changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ff417b1..5ba30cc 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,4 @@ -2.1.0-final +2.1.0-rc4 * Updated memtable_cleanup_threshold and memtable_flush_writers defaults (CASSANDRA-7551) * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/build.xml -- diff --git a/build.xml b/build.xml index fd0c72b..ca973cc 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 6f8e3ec..7a38d2d 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (2.1.0~rc4) unstable; urgency=medium + + * New RC release + + -- Eric Evans Fri, 18 Jul 2014 13:40:48 -0500 + cassandra (2.1.0~rc3) unstable; urgency=medium * New RC release
[03/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095
Branch: refs/heads/cassandra-2.1.0
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams
Committed: Mon Jul 21 07:49:13 2014 -0500
--
conf/cassandra-env.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
java_ver_output=`"${JAVA:-java}" -version 2>&1`
-jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
+jvmver=`echo "$java_ver_output" | grep 'java version' | awk -F'"' 'NR==1
{print $2}'`
JVM_VERSION=${jvmver%_*}
JVM_PATCH_VERSION=${jvmver#*_}
@@ -100,7 +100,7 @@ if [ "$JVM_VERSION" \< "1.7" ] ; then
fi
-jvm=`echo "$java_ver_output" | awk 'NR==2 {print $1}'`
+jvm=`echo "$java_ver_output" | grep -A 1 'java version' | awk 'NR==2 {print
$1}'`
case "$jvm" in
OpenJDK)
JVM_VENDOR=OpenJDK
[01/12] git commit: update versioning for 2.1.0-rc4 release
Repository: cassandra Updated Branches: refs/heads/cassandra-2.0 337e4a829 -> 875480958 refs/heads/cassandra-2.1 82fa5d507 -> 3b14f6a5d refs/heads/cassandra-2.1.0 d872e2c74 -> df7bd7f72 refs/heads/trunk 8d8fed522 -> 2093270c7 update versioning for 2.1.0-rc4 release Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d872e2c7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d872e2c7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d872e2c7 Branch: refs/heads/cassandra-2.1 Commit: d872e2c74685c0c8c019b2ef46aefa443efa95b9 Parents: 8fd8832 Author: Eric Evans Authored: Fri Jul 18 13:42:11 2014 -0500 Committer: Eric Evans Committed: Fri Jul 18 13:42:11 2014 -0500 -- CHANGES.txt | 2 +- build.xml| 2 +- debian/changelog | 6 ++ 3 files changed, 8 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ff417b1..5ba30cc 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,4 @@ -2.1.0-final +2.1.0-rc4 * Updated memtable_cleanup_threshold and memtable_flush_writers defaults (CASSANDRA-7551) * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505) http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/build.xml -- diff --git a/build.xml b/build.xml index fd0c72b..ca973cc 100644 --- a/build.xml +++ b/build.xml @@ -25,7 +25,7 @@ - + http://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=tree"/> http://git-wip-us.apache.org/repos/asf/cassandra/blob/d872e2c7/debian/changelog -- diff --git a/debian/changelog b/debian/changelog index 6f8e3ec..7a38d2d 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +cassandra (2.1.0~rc4) unstable; urgency=medium + + * New RC release + + -- Eric Evans Fri, 18 Jul 2014 13:40:48 -0500 + cassandra (2.1.0~rc3) unstable; urgency=medium * New RC release
[10/12] git commit: Merge branch 'cassandra-2.1.0' into cassandra-2.1
Merge branch 'cassandra-2.1.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3b14f6a5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3b14f6a5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3b14f6a5 Branch: refs/heads/trunk Commit: 3b14f6a5dbc91b3e8d2efa95840a61604dd6fbcb Parents: 82fa5d5 df7bd7f Author: Brandon Williams Authored: Mon Jul 21 07:51:10 2014 -0500 Committer: Brandon Williams Committed: Mon Jul 21 07:51:10 2014 -0500 -- CHANGES.txt | 2 +- build.xml | 2 +- conf/cassandra-env.sh | 4 ++-- debian/changelog | 6 ++ 4 files changed, 10 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/CHANGES.txt -- diff --cc CHANGES.txt index 8fa96f6,5ba30cc..2aa2efa --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,15 -1,4 +1,15 @@@ +2.1.1 + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Fail to start if commit log replay detects a problem (CASSANDRA-7125) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) + * Fix native protocol drop user type notification (CASSANDRA-7571) + + - 2.1.0-final + 2.1.0-rc4 * Updated memtable_cleanup_threshold and memtable_flush_writers defaults (CASSANDRA-7551) * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505) http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/build.xml -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3b14f6a5/conf/cassandra-env.sh --
[08/12] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1.0
Merge branch 'cassandra-2.0' into cassandra-2.1.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df7bd7f7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df7bd7f7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df7bd7f7 Branch: refs/heads/cassandra-2.1.0 Commit: df7bd7f725f6df5102730de89647418dd9f9473d Parents: d872e2c 8754809 Author: Brandon Williams Authored: Mon Jul 21 07:50:18 2014 -0500 Committer: Brandon Williams Committed: Mon Jul 21 07:50:18 2014 -0500 -- conf/cassandra-env.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/df7bd7f7/conf/cassandra-env.sh --
[05/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095
Branch: refs/heads/cassandra-2.0
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams
Committed: Mon Jul 21 07:49:13 2014 -0500
--
conf/cassandra-env.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
java_ver_output=`"${JAVA:-java}" -version 2>&1`
-jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
+jvmver=`echo "$java_ver_output" | grep 'java version' | awk -F'"' 'NR==1
{print $2}'`
JVM_VERSION=${jvmver%_*}
JVM_PATCH_VERSION=${jvmver#*_}
@@ -100,7 +100,7 @@ if [ "$JVM_VERSION" \< "1.7" ] ; then
fi
-jvm=`echo "$java_ver_output" | awk 'NR==2 {print $1}'`
+jvm=`echo "$java_ver_output" | grep -A 1 'java version' | awk 'NR==2 {print
$1}'`
case "$jvm" in
OpenJDK)
JVM_VENDOR=OpenJDK
[04/12] git commit: Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Fix jvm version detection when JAVA_TOOL_OPTIONS in environment
Patch by Gregory Ramsperger, reviewed by brandonwilliams for
CASSANDRA-7572
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/87548095
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/87548095
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/87548095
Branch: refs/heads/trunk
Commit: 87548095859c210e7be8cc8bbb4a5aa1f857a6ad
Parents: 337e4a8
Author: Brandon Williams
Authored: Mon Jul 21 07:49:13 2014 -0500
Committer: Brandon Williams
Committed: Mon Jul 21 07:49:13 2014 -0500
--
conf/cassandra-env.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/87548095/conf/cassandra-env.sh
--
diff --git a/conf/cassandra-env.sh b/conf/cassandra-env.sh
index 7604918..d4e6590 100644
--- a/conf/cassandra-env.sh
+++ b/conf/cassandra-env.sh
@@ -90,7 +90,7 @@ calculate_heap_sizes()
java_ver_output=`"${JAVA:-java}" -version 2>&1`
-jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
+jvmver=`echo "$java_ver_output" | grep 'java version' | awk -F'"' 'NR==1
{print $2}'`
JVM_VERSION=${jvmver%_*}
JVM_PATCH_VERSION=${jvmver#*_}
@@ -100,7 +100,7 @@ if [ "$JVM_VERSION" \< "1.7" ] ; then
fi
-jvm=`echo "$java_ver_output" | awk 'NR==2 {print $1}'`
+jvm=`echo "$java_ver_output" | grep -A 1 'java version' | awk 'NR==2 {print
$1}'`
case "$jvm" in
OpenJDK)
JVM_VENDOR=OpenJDK
[12/12] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/2093270c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/2093270c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/2093270c Branch: refs/heads/trunk Commit: 2093270c7c51700996577fd679f644bcfb17239e Parents: 8d8fed5 3b14f6a Author: Brandon Williams Authored: Mon Jul 21 07:51:21 2014 -0500 Committer: Brandon Williams Committed: Mon Jul 21 07:51:21 2014 -0500 -- CHANGES.txt | 2 +- build.xml | 2 +- conf/cassandra-env.sh | 4 ++-- debian/changelog | 6 ++ 4 files changed, 10 insertions(+), 4 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2093270c/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/2093270c/build.xml --
[jira] [Updated] (CASSANDRA-5839) Save repair data to system table
[ https://issues.apache.org/jira/browse/CASSANDRA-5839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-5839: --- Fix Version/s: (was: 2.0.10) 3.0 > Save repair data to system table > > > Key: CASSANDRA-5839 > URL: https://issues.apache.org/jira/browse/CASSANDRA-5839 > Project: Cassandra > Issue Type: New Feature > Components: Core, Tools >Reporter: Jonathan Ellis >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 3.0 > > Attachments: 2.0.4-5839-draft.patch, 2.0.6-5839-v2.patch > > > As noted in CASSANDRA-2405, it would be useful to store repair results, > particularly with sub-range repair available (CASSANDRA-5280). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (CASSANDRA-7572) cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in environment
[
https://issues.apache.org/jira/browse/CASSANDRA-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams resolved CASSANDRA-7572.
-
Resolution: Fixed
Fix Version/s: 2.1.0
2.0.10
Reviewer: Brandon Williams
bq. Parsing this is not guaranteed to work as there is no contract for this
output. The recommended way of dealing with this is via a Java application
(which will have access to each of these pieces).
While I totally agree with you on this point, we've been using this method for
years without any problem until now, so I committed the change with grep.
> cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in
> environment
> ---
>
> Key: CASSANDRA-7572
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7572
> Project: Cassandra
> Issue Type: Bug
> Components: Config
> Environment: Linux/darwin/etc
>Reporter: Gregory Ramsperger
> Fix For: 2.0.10, 2.1.0
>
>
> Parsing of Java version is done in cassandra-env.sh via
> {code}
> java_ver_output=`"${JAVA:-java}" -version 2>&1`
> jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
> {code}
> This fails if the first line of output from `java -version` does not contain
> the version. When JAVA_TOOL_OPTIONS is set as the output is
> {quote}
> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
> java version "1.7.0_40"
> Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
> Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
> {quote}
> Parsing this is not guaranteed to work as there is no contract for this
> output. The recommended way of dealing with this is via a Java application
> (which will have access to each of these pieces).
> If you want to keep with this model of doing things, you can apply a very
> small but hacky patch:
> {code}
> -jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
> +jvmver=`echo "$java_ver_output" | grep 'java version' | awk -F'"' 'NR==1
> {print $2}'`
> JVM_VERSION=${jvmver%_*}
> JVM_PATCH_VERSION=${jvmver#*_}
> @@ -100,7 +100,7 @@
> fi
> -jvm=`echo "$java_ver_output" | awk 'NR==2 {print $1}'`
> +jvm=`echo "$java_ver_output" | grep -A 1 'java version' | awk 'NR==2 {print
> $1}'`
> case "$jvm" in
> OpenJDK)
> {code}
> Otherwise, you need to write a java app, compiled for older versions, that
> returns this data via the java.specification.version and a combination of
> java.vm.name and java.vm.vendor.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-7572) cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in environment
[
https://issues.apache.org/jira/browse/CASSANDRA-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-7572:
---
Assignee: Gregory Ramsperger
> cassandra-env.sh fails to find Java version if JAVA_TOOL_OPTIONS in
> environment
> ---
>
> Key: CASSANDRA-7572
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7572
> Project: Cassandra
> Issue Type: Bug
> Components: Config
> Environment: Linux/darwin/etc
>Reporter: Gregory Ramsperger
>Assignee: Gregory Ramsperger
> Fix For: 2.0.10, 2.1.0
>
>
> Parsing of Java version is done in cassandra-env.sh via
> {code}
> java_ver_output=`"${JAVA:-java}" -version 2>&1`
> jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
> {code}
> This fails if the first line of output from `java -version` does not contain
> the version. When JAVA_TOOL_OPTIONS is set as the output is
> {quote}
> Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8
> java version "1.7.0_40"
> Java(TM) SE Runtime Environment (build 1.7.0_40-b43)
> Java HotSpot(TM) 64-Bit Server VM (build 24.0-b56, mixed mode)
> {quote}
> Parsing this is not guaranteed to work as there is no contract for this
> output. The recommended way of dealing with this is via a Java application
> (which will have access to each of these pieces).
> If you want to keep with this model of doing things, you can apply a very
> small but hacky patch:
> {code}
> -jvmver=`echo "$java_ver_output" | awk -F'"' 'NR==1 {print $2}'`
> +jvmver=`echo "$java_ver_output" | grep 'java version' | awk -F'"' 'NR==1
> {print $2}'`
> JVM_VERSION=${jvmver%_*}
> JVM_PATCH_VERSION=${jvmver#*_}
> @@ -100,7 +100,7 @@
> fi
> -jvm=`echo "$java_ver_output" | awk 'NR==2 {print $1}'`
> +jvm=`echo "$java_ver_output" | grep -A 1 'java version' | awk 'NR==2 {print
> $1}'`
> case "$jvm" in
> OpenJDK)
> {code}
> Otherwise, you need to write a java app, compiled for older versions, that
> returns this data via the java.specification.version and a combination of
> java.vm.name and java.vm.vendor.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes
Mike Adamson created CASSANDRA-7578:
---
Summary: Non-superusers can't login to tools because of no access
to system.schema_usertypes
Key: CASSANDRA-7578
URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Priority: Blocker
Fix For: 2.1.0
When I try to login to cqlsh as a non-superuser I get the following error:
{noformat}
ErrorMessage code=2100 [Unauthorized] message="User mike has no SELECT
permission on or any of its parents"
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/478 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: portunus_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 2093270c7c51700996577fd679f644bcfb17239e Blamelist: Brandon Williams ,Eric Evans Build succeeded! sincerely, -The Buildbot
[jira] [Updated] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes
[
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-7578:
Attachment: 7578.patch
> Non-superusers can't login to tools because of no access to
> system.schema_usertypes
> ---
>
> Key: CASSANDRA-7578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Priority: Blocker
> Fix For: 2.1.0
>
> Attachments: 7578.patch
>
>
> When I try to login to cqlsh as a non-superuser I get the following error:
> {noformat}
> ErrorMessage code=2100 [Unauthorized] message="User mike has no SELECT
> permission on or any of its parents"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
buildbot failure in ASF Buildbot on cassandra-trunk
The Buildbot has detected a new failure on builder cassandra-trunk while building libcloud. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/479 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: portunus_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 075383509e507f88b2c105b4f5fde5b4d4af5e68 Blamelist: Tomaz Muraus BUILD FAILED: failed sincerely, -The Buildbot
[jira] [Updated] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes
[
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-7578:
Reviewer: Aleksey Yeschenko
> Non-superusers can't login to tools because of no access to
> system.schema_usertypes
> ---
>
> Key: CASSANDRA-7578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Priority: Blocker
> Fix For: 2.1.0
>
> Attachments: 7578.patch
>
>
> When I try to login to cqlsh as a non-superuser I get the following error:
> {noformat}
> ErrorMessage code=2100 [Unauthorized] message="User mike has no SELECT
> permission on or any of its parents"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7546: Attachment: 7546.20.txt Attaching my suggested alternative approach, which I think is a little simpler to reason about: if we loop for any reason we immediately take the monitor, if unsafe is available to us; if it isn't we don't benefit from the optimisation. We don't change the logic for updating, as taking the monitor doesn't guarantee we have exclusive access, it only guarantees we have exclusive access versus those writes that have themselves spun once, thereby capping the amount of competition. For 2.1 I suggest sticking with the same approach, then in 3.0/.1 we'll take a look at the more complex approach of making writes lazy under competition. > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[ https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068588#comment-14068588 ] Jonathan Ellis commented on CASSANDRA-7507: --- Let's keep the scope of this ticket to dealing with OOM. Open files can be another ticket. > OOM creates unreliable state - die instantly better > --- > > Key: CASSANDRA-7507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7507 > Project: Cassandra > Issue Type: New Feature >Reporter: Karl Mueller >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7507_v1.txt > > > I had a cassandra node run OOM. My heap had enough headroom, there was just > something which either was a bug or some unfortunate amount of short-term > memory utilization. This resulted in the following error: > WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java > (line 1713) Some hints were not written before shutdown. This is not > supposed to happen. You should (a) run repair, and (b) file a bug report > There are no other messages of relevance besides the OOM error about 90 > minutes earlier. > My (limited) understanding of the JVM and Cassandra says that when it goes > OOM, it will attempt to signal cassandra to shut down "cleanly". The problem, > in my view, is that with an OOM situation, nothing is guaranteed anymore. I > believe it's impossible to reliably "cleanly shut down" at this point, and > therefore it's wrong to even try. > Yes, ideally things could be written out, flushed to disk, memory messages > written, other nodes notified, etc. but why is there any reason to believe > any of those steps could happen? Would happen? Couldn't bad data be written > at this point to disk rather than good data? Some network messages delivered, > but not others? > I think Cassandra should have the option to (and possibly default) to kill > itself immediately upon the OOM condition happening in a hard way, and not > rely on the java-based clean shutdown process. Cassandra already handles > recovery from unclean shutdown, and it's not a big deal. My node, for > example, kept in a sort-of alive state for 90 minutes where who knows what it > was doing or not doing. > I don't know enough about the JVM and options for it to know the best exact > implementation of "die instantly on OOM", but it should be something that's > possible either with some flags or a C library (which doesn't rely on java > memory to do something which it may not be able to get!) > Short version: a kill -9 of all C* processes in that instance without needing > more java memory, when OOM is raised -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6434) Repair-aware gc grace period
[ https://issues.apache.org/jira/browse/CASSANDRA-6434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068592#comment-14068592 ] Jonathan Ellis commented on CASSANDRA-6434: --- We could approximate it as max(repairtime) for any sstable that may contain the key. (And we already have the logic that says, "checking bloom filters is expensive so avoid it until the last minute.") > Repair-aware gc grace period > - > > Key: CASSANDRA-6434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6434 > Project: Cassandra > Issue Type: New Feature > Components: Core >Reporter: sankalp kohli >Assignee: Marcus Eriksson > Fix For: 3.0 > > > Since the reason for gcgs is to ensure that we don't purge tombstones until > every replica has been notified, it's redundant in a world where we're > tracking repair times per sstable (and repairing frequentily), i.e., a world > where we default to incremental repair a la CASSANDRA-5351. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[ https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068616#comment-14068616 ] Joshua McKenzie commented on CASSANDRA-7507: Was planning on creating another ticket for that effort and doing a little tidying up on this patch to make it more general rather than oom-specific on naming. I should be able to get to that today. > OOM creates unreliable state - die instantly better > --- > > Key: CASSANDRA-7507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7507 > Project: Cassandra > Issue Type: New Feature >Reporter: Karl Mueller >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7507_v1.txt > > > I had a cassandra node run OOM. My heap had enough headroom, there was just > something which either was a bug or some unfortunate amount of short-term > memory utilization. This resulted in the following error: > WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java > (line 1713) Some hints were not written before shutdown. This is not > supposed to happen. You should (a) run repair, and (b) file a bug report > There are no other messages of relevance besides the OOM error about 90 > minutes earlier. > My (limited) understanding of the JVM and Cassandra says that when it goes > OOM, it will attempt to signal cassandra to shut down "cleanly". The problem, > in my view, is that with an OOM situation, nothing is guaranteed anymore. I > believe it's impossible to reliably "cleanly shut down" at this point, and > therefore it's wrong to even try. > Yes, ideally things could be written out, flushed to disk, memory messages > written, other nodes notified, etc. but why is there any reason to believe > any of those steps could happen? Would happen? Couldn't bad data be written > at this point to disk rather than good data? Some network messages delivered, > but not others? > I think Cassandra should have the option to (and possibly default) to kill > itself immediately upon the OOM condition happening in a hard way, and not > rely on the java-based clean shutdown process. Cassandra already handles > recovery from unclean shutdown, and it's not a big deal. My node, for > example, kept in a sort-of alive state for 90 minutes where who knows what it > was doing or not doing. > I don't know enough about the JVM and options for it to know the best exact > implementation of "die instantly on OOM", but it should be something that's > possible either with some flags or a C library (which doesn't rely on java > memory to do something which it may not be able to get!) > Short version: a kill -9 of all C* processes in that instance without needing > more java memory, when OOM is raised -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[ https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068730#comment-14068730 ] Benedict commented on CASSANDRA-7507: - It's worth noting then that this applies also to catch (Throwable), although it looks a little less widespread. > OOM creates unreliable state - die instantly better > --- > > Key: CASSANDRA-7507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7507 > Project: Cassandra > Issue Type: New Feature >Reporter: Karl Mueller >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7507_v1.txt > > > I had a cassandra node run OOM. My heap had enough headroom, there was just > something which either was a bug or some unfortunate amount of short-term > memory utilization. This resulted in the following error: > WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java > (line 1713) Some hints were not written before shutdown. This is not > supposed to happen. You should (a) run repair, and (b) file a bug report > There are no other messages of relevance besides the OOM error about 90 > minutes earlier. > My (limited) understanding of the JVM and Cassandra says that when it goes > OOM, it will attempt to signal cassandra to shut down "cleanly". The problem, > in my view, is that with an OOM situation, nothing is guaranteed anymore. I > believe it's impossible to reliably "cleanly shut down" at this point, and > therefore it's wrong to even try. > Yes, ideally things could be written out, flushed to disk, memory messages > written, other nodes notified, etc. but why is there any reason to believe > any of those steps could happen? Would happen? Couldn't bad data be written > at this point to disk rather than good data? Some network messages delivered, > but not others? > I think Cassandra should have the option to (and possibly default) to kill > itself immediately upon the OOM condition happening in a hard way, and not > rely on the java-based clean shutdown process. Cassandra already handles > recovery from unclean shutdown, and it's not a big deal. My node, for > example, kept in a sort-of alive state for 90 minutes where who knows what it > was doing or not doing. > I don't know enough about the JVM and options for it to know the best exact > implementation of "die instantly on OOM", but it should be something that's > possible either with some flags or a C library (which doesn't rely on java > memory to do something which it may not be able to get!) > Short version: a kill -9 of all C* processes in that instance without needing > more java memory, when OOM is raised -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] graham sanderson updated CASSANDRA-7546: Attachment: 7546.20_alt.txt Thanks for taking a look Benedict - I am in the process of actually measuring the effect via cassandra stress in the real world for code paths that might hit this (actual user initiated contended write on a partition, hint based contented write, and low cardinality secondary indexes). I had initially shied away from the non obviously paired monitor usage because a) it requires unsafe, and b) whilst monitorenter and monitorexit do exist as separate byte codes, I seem to recall that there is code in the monitor fast path (especially w.r.t. biased locking which we probably wan't to avoid) that expects them to be trivially paired... c) on the negative side this forces two loops under contention, but that is better than unbounded obviously. I'm not sure if b) is an actual issue (or what effect it has - does it matter if the monitor is always inflated for example). As for c) I guess I'll have to measure and see. Note I attached the current version of the 2.0 patch I was testing with, which pushed the loop body into a class similar to ColumnUpdater in 2.1 (which I would update accordingly)... again any attempt to further simplify the three separate loops, didn't really make things any simpler looking. I'm not sure my code (whilst not blazingly pretty) is insanely hard to reason about... clearly someone always makes progress (since any change in state is dependent on a CAS success or failure due to someone else succeeding); the un-contended and highly contended cases are pretty simple, and any flip-flopping between the "count" and "raw" state again always means we made progress AND must (still) be under low contention anyway. In any case, I'll defer to your judgement as to what you'd like to see in the code base... as I say some actual numbers outside of a synthetic test should help > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_alt.txt, suggestion1.txt, > suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7579) File descriptor exhaustion can lead to unreliable state in exception condition
Joshua McKenzie created CASSANDRA-7579: -- Summary: File descriptor exhaustion can lead to unreliable state in exception condition Key: CASSANDRA-7579 URL: https://issues.apache.org/jira/browse/CASSANDRA-7579 Project: Cassandra Issue Type: New Feature Reporter: Joshua McKenzie Priority: Minor If the JVM runs out of file descriptors we can get into an unreliable state (similar to CASSANDRA-7507 on OOM) where we cannot trust our shutdown hook to run successfully to completion. We need to check IOExceptions and appropriate Throwable to see if we have a FileNotFoundException w/message "Too many files open" and forcefully shutdown the Daemon in these cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[ https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua McKenzie updated CASSANDRA-7507: --- Attachment: 7507_v2.txt Attaching a v2 w/naming that's not OOM-centric and exposes an external method for others to call to cause a hardExit (i.e. CASSANDRA-7579) > OOM creates unreliable state - die instantly better > --- > > Key: CASSANDRA-7507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7507 > Project: Cassandra > Issue Type: New Feature >Reporter: Karl Mueller >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7507_v1.txt, 7507_v2.txt > > > I had a cassandra node run OOM. My heap had enough headroom, there was just > something which either was a bug or some unfortunate amount of short-term > memory utilization. This resulted in the following error: > WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java > (line 1713) Some hints were not written before shutdown. This is not > supposed to happen. You should (a) run repair, and (b) file a bug report > There are no other messages of relevance besides the OOM error about 90 > minutes earlier. > My (limited) understanding of the JVM and Cassandra says that when it goes > OOM, it will attempt to signal cassandra to shut down "cleanly". The problem, > in my view, is that with an OOM situation, nothing is guaranteed anymore. I > believe it's impossible to reliably "cleanly shut down" at this point, and > therefore it's wrong to even try. > Yes, ideally things could be written out, flushed to disk, memory messages > written, other nodes notified, etc. but why is there any reason to believe > any of those steps could happen? Would happen? Couldn't bad data be written > at this point to disk rather than good data? Some network messages delivered, > but not others? > I think Cassandra should have the option to (and possibly default) to kill > itself immediately upon the OOM condition happening in a hard way, and not > rely on the java-based clean shutdown process. Cassandra already handles > recovery from unclean shutdown, and it's not a big deal. My node, for > example, kept in a sort-of alive state for 90 minutes where who knows what it > was doing or not doing. > I don't know enough about the JVM and options for it to know the best exact > implementation of "die instantly on OOM", but it should be something that's > possible either with some flags or a C library (which doesn't rely on java > memory to do something which it may not be able to get!) > Short version: a kill -9 of all C* processes in that instance without needing > more java memory, when OOM is raised -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068803#comment-14068803 ] Benedict commented on CASSANDRA-7546: - bq. I'm not sure my code (whilst not blazingly pretty) is insanely hard to reason about... I'm not suggesting it is by any means abhorrent, only that we can achieve the desired goal with fewer changes, so unless there's a lot of evidence that the extra complexity is worth it, we should stick with the simpler approach (this also means less pollution of the instruction cache in a very hot part of the codebase, which is a good thing). I would suggest running a stress workload with a fixed number of threads, with increasing numbers of partitions (from 1 up to > number of threads) and see how the curve changes, if you want to benchmark this closely. As to (b): since we only ever acquire the lock when we are contending, it must always be inflated anyway, so this shouldn't be an issue. > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_alt.txt, suggestion1.txt, > suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[ https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14068856#comment-14068856 ] Benedict commented on CASSANDRA-7507: - Starting a thread is sometimes impossible after an OOM (since OOM can be thrown for having exhausted our permitted thread count), so we should probably execute our shutdown directly in the thread that caught the exception. Also, we need to (IMO) deal with all the catch (Throwable) instances that do not propagate the exception - esp. any worker threads in SEP, DebuggableExecutor etc, but there are other places we could get bitten as well. > OOM creates unreliable state - die instantly better > --- > > Key: CASSANDRA-7507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7507 > Project: Cassandra > Issue Type: New Feature >Reporter: Karl Mueller >Assignee: Joshua McKenzie >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7507_v1.txt, 7507_v2.txt > > > I had a cassandra node run OOM. My heap had enough headroom, there was just > something which either was a bug or some unfortunate amount of short-term > memory utilization. This resulted in the following error: > WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java > (line 1713) Some hints were not written before shutdown. This is not > supposed to happen. You should (a) run repair, and (b) file a bug report > There are no other messages of relevance besides the OOM error about 90 > minutes earlier. > My (limited) understanding of the JVM and Cassandra says that when it goes > OOM, it will attempt to signal cassandra to shut down "cleanly". The problem, > in my view, is that with an OOM situation, nothing is guaranteed anymore. I > believe it's impossible to reliably "cleanly shut down" at this point, and > therefore it's wrong to even try. > Yes, ideally things could be written out, flushed to disk, memory messages > written, other nodes notified, etc. but why is there any reason to believe > any of those steps could happen? Would happen? Couldn't bad data be written > at this point to disk rather than good data? Some network messages delivered, > but not others? > I think Cassandra should have the option to (and possibly default) to kill > itself immediately upon the OOM condition happening in a hard way, and not > rely on the java-based clean shutdown process. Cassandra already handles > recovery from unclean shutdown, and it's not a big deal. My node, for > example, kept in a sort-of alive state for 90 minutes where who knows what it > was doing or not doing. > I don't know enough about the JVM and options for it to know the best exact > implementation of "die instantly on OOM", but it should be something that's > possible either with some flags or a C library (which doesn't rely on java > memory to do something which it may not be able to get!) > Short version: a kill -9 of all C* processes in that instance without needing > more java memory, when OOM is raised -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7580) Super user cannot list permissions with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Adamson updated CASSANDRA-7580:
Description:
On a system configured with:
* PasswordAuthenticator
* CassandraAuthorizer
If you login as a super user and issue:
{noformat}
list all permissions;
{noformat}
where permissions have been granted to a user you get back:
{noformat}
Keyspace None not found.
{noformat}
was:
On a system configured with:
* PasswordAuthenticator
* CassandraAuthorizer
If you login as a super user and issue:
{noformat}
list all permissions;
{noformat}
where permissions have been granted to a user you get back:
{noformat}
Keyspace None not found.
{noformat}
> Super user cannot list permissions with cqlsh
> -
>
> Key: CASSANDRA-7580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Priority: Critical
> Fix For: 2.1.0
>
>
> On a system configured with:
> * PasswordAuthenticator
> * CassandraAuthorizer
> If you login as a super user and issue:
> {noformat}
> list all permissions;
> {noformat}
> where permissions have been granted to a user you get back:
> {noformat}
> Keyspace None not found.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7580) Super user cannot list permissions with cqlsh
Mike Adamson created CASSANDRA-7580:
---
Summary: Super user cannot list permissions with cqlsh
Key: CASSANDRA-7580
URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
Project: Cassandra
Issue Type: Bug
Components: Tools
Reporter: Mike Adamson
Priority: Critical
Fix For: 2.1.0
On a system configured with:
* PasswordAuthenticator
* CassandraAuthorizer
If you login as a super user and issue:
{noformat}
list all permissions;
{noformat}
where permissions have been granted to a user you get back:
{noformat}
Keyspace None not found.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7580) Super user cannot list permissions with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-7580:
Attachment: 7580.txt
Patch against 2.1.0
> Super user cannot list permissions with cqlsh
> -
>
> Key: CASSANDRA-7580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Priority: Critical
> Fix For: 2.1.0
>
> Attachments: 7580.txt
>
>
> On a system configured with:
> * PasswordAuthenticator
> * CassandraAuthorizer
> If you login as a super user and issue:
> {noformat}
> list all permissions;
> {noformat}
> where permissions have been granted to a user you get back:
> {noformat}
> Keyspace None not found.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7580) Super user cannot list permissions with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-7580:
Reviewer: Tyler Hobbs
Priority: Minor (was: Critical)
> Super user cannot list permissions with cqlsh
> -
>
> Key: CASSANDRA-7580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Brandon Williams
>Priority: Minor
> Fix For: 2.1.0
>
> Attachments: 7580.txt
>
>
> On a system configured with:
> * PasswordAuthenticator
> * CassandraAuthorizer
> If you login as a super user and issue:
> {noformat}
> list all permissions;
> {noformat}
> where permissions have been granted to a user you get back:
> {noformat}
> Keyspace None not found.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (CASSANDRA-7580) Super user cannot list permissions with cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-7580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-7580:
---
Assignee: Brandon Williams
> Super user cannot list permissions with cqlsh
> -
>
> Key: CASSANDRA-7580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7580
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Brandon Williams
>Priority: Critical
> Fix For: 2.1.0
>
> Attachments: 7580.txt
>
>
> On a system configured with:
> * PasswordAuthenticator
> * CassandraAuthorizer
> If you login as a super user and issue:
> {noformat}
> list all permissions;
> {noformat}
> where permissions have been granted to a user you get back:
> {noformat}
> Keyspace None not found.
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] graham sanderson updated CASSANDRA-7546: Attachment: 7546.20_2.txt Yes, I agree with you; simpler is better - I didn't like the fact that it was hard to disentangle my code. So I have attached a different patch 7546.20_2.txt, which is about the same as yours except that it tracks a very minimal state heuristic to avoid the first unlocked loop when it thinks it is contended... this helps under load where it attempts about 1.11 loop per mutation in the heavily contented case vs 1.93 with 7546.20.txt (or about 18 without any patch at all). It turns out to be rather hard to measure this with cassandra-stress in isolation - basically the looping causes very high memory allocation rates which may or may not be a problem based on the current state of memory in the JVM (e.g. currently resident recent memtables) and other activity. Also, I may open a separate ticket to allow batch (thrift and maybe CQL) inserts in cassandra-stress to bump up the load a little more. That said, clearly this race is undesirable, and we have certainly observed its effects in practice in the hinting case (which is doubly unfortunate since you are already now doing more work per node than you thought, and you are going down a contention path you weren't expecting based on your own partitioning). Note when it goes wrong it goes really wrong! So, I think settling for a patch with minimal code or uncontended path impact, that caps us nicely is the right way to go > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, > suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069059#comment-14069059
]
graham sanderson commented on CASSANDRA-7546:
-
FYI here are the same synthetic test results for 7546.20_2.txt
{code}
[junit] --
[junit] 1 THREAD; ELEMENT SIZE 64
[junit]
[junit] Threads = 1 elements = 10 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 993ms maxConcurrency = 1
[junit] GC for PS Scavenge: 34 ms for 3 collections
[junit] Approx allocation = 553MB vs 8MB; ratio to raw data size =
69.13799428571429
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 761ms maxConcurrency = 1
[junit] GC for PS Scavenge: 34 ms for 3 collections
[junit] Approx allocation = 579MB vs 8MB; ratio to raw data size =
72.31675047619048
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 10 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 780ms maxConcurrency = 1
[junit] GC for PS Scavenge: 25 ms for 2 collections
[junit] Approx allocation = 436MB vs 8MB; ratio to raw data size =
54.48992095238095
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 671ms maxConcurrency = 1
[junit] GC for PS Scavenge: 24 ms for 2 collections
[junit] Approx allocation = 477MB vs 8MB; ratio to raw data size =
59.545997142857146
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 10 (of size 64) partitions = 256
[junit] original code:
[junit] Duration = 452ms maxConcurrency = 1
[junit] GC for PS Scavenge: 11 ms for 1 collections
[junit] Approx allocation = 321MB vs 8MB; ratio to raw data size =
40.14510761904762
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 460ms maxConcurrency = 1
[junit] GC for PS Scavenge: 10 ms for 1 collections
[junit] Approx allocation = 341MB vs 8MB; ratio to raw data size =
42.63770857142857
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] Threads = 1 elements = 10 (of size 64) partitions = 1024
[junit] original code:
[junit] Duration = 462ms maxConcurrency = 1
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 264MB vs 8MB; ratio to raw data size =
32.99879142857143
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 543ms maxConcurrency = 1
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 272MB vs 8MB; ratio to raw data size =
34.047360952380956
[junit] loopRatio (closest to 1 best) 1.0 raw 10/10 counted 0/0
sync 0/0 up 0 down 0
[junit]
[junit]
[junit] --
[junit] 100 THREADS; ELEMENT SIZE 64
[junit]
[junit] Threads = 100 elements = 10 (of size 64) partitions = 1
[junit] original code:
[junit] Duration = 2318ms maxConcurrency = 100
[junit] GC for PS Scavenge: 119 ms for 32 collections
[junit] Approx allocation = 10547MB vs 8MB; ratio to raw data size =
1316.62704
[junit] loopRatio (closest to 1 best) 18.35448 raw 10/1835448 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 1315ms maxConcurrency = 100
[junit] GC for PS Scavenge: 14 ms for 1 collections
[junit] Approx allocation = 629MB vs 8MB; ratio to raw data size =
78.62949142857143
[junit] loopRatio (closest to 1 best) 1.11563 raw 13653/13653 counted 0/0
sync 88223/97910 up 0 down 0
[junit]
[junit]
[junit] Threads = 100 elements = 10 (of size 64) partitions = 16
[junit] original code:
[junit] Duration = 215ms maxConcurrency = 100
[junit] GC for PS Scavenge: 23 ms for 2 collections
[junit] Approx allocation = 776MB vs 8MB; ratio to raw data size =
96.92138285714286
[junit] loopRatio (closest to 1 best) 1.95927 raw 10/195927 counted
0/0 sync 0/0 up 0 down 0
[junit]
[junit] modified code:
[junit] Duration = 201ms maxConcurrency = 99
[junit] GC for PS Scavenge: 9 ms for 1 collections
[juni
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069065#comment-14069065 ] Benedict commented on CASSANDRA-7546: - I'll take a look at your patch shortly, but in the meantime it's worth pointing out cassandra-stress does now support fairly complex CQL inserts including various sizes of batch updates, with fine grained control over how large a partition to generate, and what percentage of that total partition to update at any point. Take a look at the sample stress profiles under the tools hierarchy on latest 2.1 > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, > suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joshua McKenzie updated CASSANDRA-7507:
---
Attachment: exceptionHandlingResults.txt
findSwallowedExceptions.py
Some quick and dirty parsing of the code base gives me:
{code:title=Parse_Results}
Total Exceptions caught and rethrown as something other than RuntimeException:
69
Total Exceptions caught and rethrown as RuntimeException: 76
Total Exceptions Swallowed: 89
Total Exceptions analyzed: 234
{code}
While incredibly rough and not handling a bunch of edge-cases in the exception
handling blocks, as a general metric of how pervasive these blanket catches are
within the code-base this data should be helpful to get a general feel for how
much work we're proposing here.
Attaching findSwallowedExceptions.py and exceptionHandlingResults.txt which is
output from the script.
While I agree that this is something we need to look into, I'm not sure 2.1.1
and this ticket is the right place to make changes that are this extensive.
3.0 and another ticket to specifically go through the code-base and clean up
blanket exception catch blocks to not swallow and be very specific on the logic
surrounding rethrows would work for me. WDYT [~jbellis]?
> OOM creates unreliable state - die instantly better
> ---
>
> Key: CASSANDRA-7507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Karl Mueller
>Assignee: Joshua McKenzie
>Priority: Minor
> Fix For: 2.1.1
>
> Attachments: 7507_v1.txt, 7507_v2.txt, exceptionHandlingResults.txt,
> findSwallowedExceptions.py
>
>
> I had a cassandra node run OOM. My heap had enough headroom, there was just
> something which either was a bug or some unfortunate amount of short-term
> memory utilization. This resulted in the following error:
> WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java
> (line 1713) Some hints were not written before shutdown. This is not
> supposed to happen. You should (a) run repair, and (b) file a bug report
> There are no other messages of relevance besides the OOM error about 90
> minutes earlier.
> My (limited) understanding of the JVM and Cassandra says that when it goes
> OOM, it will attempt to signal cassandra to shut down "cleanly". The problem,
> in my view, is that with an OOM situation, nothing is guaranteed anymore. I
> believe it's impossible to reliably "cleanly shut down" at this point, and
> therefore it's wrong to even try.
> Yes, ideally things could be written out, flushed to disk, memory messages
> written, other nodes notified, etc. but why is there any reason to believe
> any of those steps could happen? Would happen? Couldn't bad data be written
> at this point to disk rather than good data? Some network messages delivered,
> but not others?
> I think Cassandra should have the option to (and possibly default) to kill
> itself immediately upon the OOM condition happening in a hard way, and not
> rely on the java-based clean shutdown process. Cassandra already handles
> recovery from unclean shutdown, and it's not a big deal. My node, for
> example, kept in a sort-of alive state for 90 minutes where who knows what it
> was doing or not doing.
> I don't know enough about the JVM and options for it to know the best exact
> implementation of "die instantly on OOM", but it should be something that's
> possible either with some flags or a C library (which doesn't rely on java
> memory to do something which it may not be able to get!)
> Short version: a kill -9 of all C* processes in that instance without needing
> more java memory, when OOM is raised
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069173#comment-14069173 ] graham sanderson commented on CASSANDRA-7546: - Excellent - I will take a look in the 2.1 branch - I was wondering if there were some sample profiles. The main problem we have in 2.0.x is that if we are under relatively heavy sustained write load, so we are allocating memtable slabs along with all the small short lived objects in the commit log and write path... you add to that hinting which means more memtable slabs, and now because of single partition for hints, much larger snap trees (whose somewhat contentious lazy-copy-on-write may or may not make things worse, I don't know)... under that allocation rate we spill huge numbers of small (possibly snap tree nodes) objects into the tenured gen along with the slabs, which tends to lead to promotion failure and need for compaction. I'll have to play around, but I don't think it is easy to capture the effect of excessive (intended to be) temporary object allocation in a stress test as opposed to excessive CPU because the GC can cope really well until it doesn't. Note my belief is your new tree in 2.1 probably mitigates the problem quite a bit (no contention in the tree, wider nodes, less rebalancing etc), though I suggest we still fix the CAS loop allocation there too. > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, > suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7578) Non-superusers can't login to tools because of no access to system.schema_usertypes
[
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069218#comment-14069218
]
Aleksey Yeschenko commented on CASSANDRA-7578:
--
Not available to anyone without access to table.schema_usertypes, only when
both authn and authz are enabled. Not blocker-worthy.
> Non-superusers can't login to tools because of no access to
> system.schema_usertypes
> ---
>
> Key: CASSANDRA-7578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Priority: Blocker
> Fix For: 2.1.0
>
> Attachments: 7578.patch
>
>
> When I try to login to cqlsh as a non-superuser I get the following error:
> {noformat}
> ErrorMessage code=2100 [Unauthorized] message="User mike has no SELECT
> permission on or any of its parents"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7578) Add system.schema_usertypes to the readable-by-default whitelist
[
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-7578:
-
Priority: Minor (was: Blocker)
Fix Version/s: (was: 2.1.0)
2.1.1
Assignee: Mike Adamson
Summary: Add system.schema_usertypes to the readable-by-default
whitelist (was: Non-superusers can't login to tools because of no access to
system.schema_usertypes)
> Add system.schema_usertypes to the readable-by-default whitelist
>
>
> Key: CASSANDRA-7578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 2.1.1
>
> Attachments: 7578.patch
>
>
> When I try to login to cqlsh as a non-superuser I get the following error:
> {noformat}
> ErrorMessage code=2100 [Unauthorized] message="User mike has no SELECT
> permission on or any of its parents"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7468) Add time-based execution to cassandra-stress
[ https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Kennedy updated CASSANDRA-7468: Attachment: trunk-7468-rebase.patch > Add time-based execution to cassandra-stress > > > Key: CASSANDRA-7468 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7468 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Matt Kennedy >Assignee: Matt Kennedy >Priority: Minor > Fix For: 2.1.1 > > Attachments: trunk-7468-rebase.patch, trunk-7468.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7468) Add time-based execution to cassandra-stress
[ https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069268#comment-14069268 ] Matt Kennedy commented on CASSANDRA-7468: - Rebased to trunk. Changed '-d' parameter to '-duration'. Note, running this without the latest DataStax Java driver (2.1-beta2) results in some seemingly extraneous stack traces, but they don't seem to affect functionality. > Add time-based execution to cassandra-stress > > > Key: CASSANDRA-7468 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7468 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Matt Kennedy >Assignee: Matt Kennedy >Priority: Minor > Fix For: 2.1.1 > > Attachments: trunk-7468-rebase.patch, trunk-7468.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069309#comment-14069309 ] sankalp kohli commented on CASSANDRA-2434: -- This will be very nice to have in 2.0 cc [~brandon.williams] > range movements can violate consistency > --- > > Key: CASSANDRA-2434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Assignee: T Jake Luciani > Fix For: 2.1 beta2 > > Attachments: 2434-3.patch.txt, 2434-testery.patch.txt > > > My reading (a while ago) of the code indicates that there is no logic > involved during bootstrapping that avoids consistency level violations. If I > recall correctly it just grabs neighbors that are currently up. > There are at least two issues I have with this behavior: > * If I have a cluster where I have applications relying on QUORUM with RF=3, > and bootstrapping complete based on only one node, I have just violated the > supposedly guaranteed consistency semantics of the cluster. > * Nodes can flap up and down at any time, so even if a human takes care to > look at which nodes are up and things about it carefully before > bootstrapping, there's no guarantee. > A complication is that not only does it depend on use-case where this is an > issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster > which is otherwise used for QUORUM operations you may wish to accept > less-than-quorum nodes during bootstrap in various emergency situations. > A potential easy fix is to have bootstrap take an argument which is the > number of hosts to bootstrap from, or to assume QUORUM if none is given. > (A related concern is bootstrapping across data centers. You may *want* to > bootstrap to a local node and then do a repair to avoid sending loads of data > across DC:s while still achieving consistency. Or even if you don't care > about the consistency issues, I don't think there is currently a way to > bootstrap from local nodes only.) > Thoughts? -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7468) Add time-based execution to cassandra-stress
[ https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict updated CASSANDRA-7468: Attachment: 7468v2.txt Attached a slightly tweaked version that uses Uninterruptibles.sleepUninterruptibly instead of Thread.sleep(), and also asks that the duration units be specified, for the hell of it (supporting s, m and h)... "duration" seemed a bit strange without it :-) For the record, and posterity, the switch is actually duration=, not -duration If you review the changes I'll commit to trunk + 2.1 > Add time-based execution to cassandra-stress > > > Key: CASSANDRA-7468 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7468 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Matt Kennedy >Assignee: Matt Kennedy >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7468v2.txt, trunk-7468-rebase.patch, trunk-7468.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (CASSANDRA-7581) Stop read requests for partitions which don't belong to a node
sankalp kohli created CASSANDRA-7581: Summary: Stop read requests for partitions which don't belong to a node Key: CASSANDRA-7581 URL: https://issues.apache.org/jira/browse/CASSANDRA-7581 Project: Cassandra Issue Type: Bug Reporter: sankalp kohli Priority: Minor A C* instance will serve CF level reads with null values or old values to a co-ordinator even if the instance is not responsible for the data. This is bad if co-ordinator has a wrong view of the ring and will return wrong results back to the client. A solution to this problem could be to not respond or return error to such read requests. Tricky part here is to respond to read requests when the instance has just lost ownership for a few ring delay. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-6851) Improve anticompaction after incremental repair
[
https://issues.apache.org/jira/browse/CASSANDRA-6851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069324#comment-14069324
]
Russell Alexander Spitzer commented on CASSANDRA-6851:
--
The only thing I worry about is changing the signature on,
{code}
public Collection>
groupSSTablesForAntiCompaction(Collection sstablesToGroup, int
groupSize)
{code}
I was wondering if we might want multiple implementations, some of which may
not have a set group size. I know this is bikesheding, but maybe we want to
change the signature to maxGroupSize?
I'm +1 with everything else
> Improve anticompaction after incremental repair
> ---
>
> Key: CASSANDRA-6851
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6851
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Marcus Eriksson
>Assignee: Russell Alexander Spitzer
>Priority: Minor
> Labels: compaction, lhf
> Fix For: 3.0
>
>
> After an incremental repair we iterate over all sstables and split them in
> two parts, one containing the repaired data and one the unrepaired. We could
> in theory double the number of sstables on a node.
> To avoid this we could make anticompaction also do a compaction, for example,
> if we are to anticompact 10 sstables, we could anticompact those to 2.
> Note that we need to avoid creating too big sstables though, if we
> anticompact all sstables on a node it would essentially be a major compaction.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6572) Workload recording / playback
[ https://issues.apache.org/jira/browse/CASSANDRA-6572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069328#comment-14069328 ] Lyuben Todorov commented on CASSANDRA-6572: --- variablesClone was used for debugging the values being stored for replay so it was indeed redundant, pushing [6228df6|https://github.com/lyubent/cassandra/commit/6228df6e5f047b29476744b1012802de37edda37] that makes replay of prepared statements functional, now string statements and prepared statements can be mixed and replayed. The commit also addressed the above issues. Next up is logging of thread & session ids. > Workload recording / playback > - > > Key: CASSANDRA-6572 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6572 > Project: Cassandra > Issue Type: New Feature > Components: Core, Tools >Reporter: Jonathan Ellis >Assignee: Lyuben Todorov > Fix For: 2.1.1 > > Attachments: 6572-trunk.diff > > > "Write sample mode" gets us part way to testing new versions against a real > world workload, but we need an easy way to test the query side as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: no need to enforce case, when using equalsIgnoreCase
Repository: cassandra
Updated Branches:
refs/heads/trunk 2093270c7 -> 453292f3c
no need to enforce case, when using equalsIgnoreCase
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/453292f3
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/453292f3
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/453292f3
Branch: refs/heads/trunk
Commit: 453292f3c6a443aed72a4b53203f1a46b26d9aaf
Parents: 2093270
Author: Dave Brosius
Authored: Mon Jul 21 17:41:43 2014 -0400
Committer: Dave Brosius
Committed: Mon Jul 21 17:41:43 2014 -0400
--
tools/stress/src/org/apache/cassandra/stress/StressProfile.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/453292f3/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
--
diff --git a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
index 13f26a2..8037ba5 100644
--- a/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
+++ b/tools/stress/src/org/apache/cassandra/stress/StressProfile.java
@@ -492,7 +492,7 @@ public class StressProfile implements Serializable
while (iter.hasNext())
{
Map.Entry e = iter.next();
-if (!e.getKey().toLowerCase().equalsIgnoreCase(e.getKey()))
+if (!e.getKey().equalsIgnoreCase(e.getKey()))
{
reinsert.add(e);
iter.remove();
[jira] [Commented] (CASSANDRA-2434) range movements can violate consistency
[ https://issues.apache.org/jira/browse/CASSANDRA-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069351#comment-14069351 ] Brandon Williams commented on CASSANDRA-2434: - Does seem like mostly new code we could just add with the flag defaulting to off to give 2.0 the option. > range movements can violate consistency > --- > > Key: CASSANDRA-2434 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2434 > Project: Cassandra > Issue Type: Bug >Reporter: Peter Schuller >Assignee: T Jake Luciani > Fix For: 2.1 beta2 > > Attachments: 2434-3.patch.txt, 2434-testery.patch.txt > > > My reading (a while ago) of the code indicates that there is no logic > involved during bootstrapping that avoids consistency level violations. If I > recall correctly it just grabs neighbors that are currently up. > There are at least two issues I have with this behavior: > * If I have a cluster where I have applications relying on QUORUM with RF=3, > and bootstrapping complete based on only one node, I have just violated the > supposedly guaranteed consistency semantics of the cluster. > * Nodes can flap up and down at any time, so even if a human takes care to > look at which nodes are up and things about it carefully before > bootstrapping, there's no guarantee. > A complication is that not only does it depend on use-case where this is an > issue (if all you ever do you do at CL.ONE, it's fine); even in a cluster > which is otherwise used for QUORUM operations you may wish to accept > less-than-quorum nodes during bootstrap in various emergency situations. > A potential easy fix is to have bootstrap take an argument which is the > number of hosts to bootstrap from, or to assume QUORUM if none is given. > (A related concern is bootstrapping across data centers. You may *want* to > bootstrap to a local node and then do a repair to avoid sending loads of data > across DC:s while still achieving consistency. Or even if you don't care > about the consistency issues, I don't think there is currently a way to > bootstrap from local nodes only.) > Thoughts? -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: remove unbound method template parm
Repository: cassandra
Updated Branches:
refs/heads/trunk 453292f3c -> 19b7d56fe
remove unbound method template parm
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/19b7d56f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/19b7d56f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/19b7d56f
Branch: refs/heads/trunk
Commit: 19b7d56fece687b108e1b6ee024e0f09592a529e
Parents: 453292f
Author: Dave Brosius
Authored: Mon Jul 21 17:46:45 2014 -0400
Committer: Dave Brosius
Committed: Mon Jul 21 17:46:45 2014 -0400
--
src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/19b7d56f/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
--
diff --git a/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
b/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
index 92fabf1..9469a9c 100644
--- a/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
+++ b/src/java/org/apache/cassandra/metrics/KeyspaceMetrics.java
@@ -223,7 +223,7 @@ public class KeyspaceMetrics
* @param MetricValue
* @return Gauge>Long> that computes sum of MetricValue.getValue()
*/
-private Gauge createKeyspaceGauge(String name,
final MetricValue extractor)
+private Gauge createKeyspaceGauge(String name, final MetricValue
extractor)
{
allMetrics.add(name);
return Metrics.newGauge(factory.createMetricName(name), new
Gauge()
buildbot success in ASF Buildbot on cassandra-trunk
The Buildbot has detected a restored build on builder cassandra-trunk while building cassandra. Full details are available at: http://ci.apache.org/builders/cassandra-trunk/builds/480 Buildbot URL: http://ci.apache.org/ Buildslave for this Build: portunus_ubuntu Build Reason: scheduler Build Source Stamp: [branch trunk] 453292f3c6a443aed72a4b53203f1a46b26d9aaf Blamelist: Dave Brosius Build succeeded! sincerely, -The Buildbot
[jira] [Commented] (CASSANDRA-7468) Add time-based execution to cassandra-stress
[ https://issues.apache.org/jira/browse/CASSANDRA-7468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069387#comment-14069387 ] Matt Kennedy commented on CASSANDRA-7468: - Thanks for the review, units are a welcome addition. I'm also relieved you got rid of the countInSeconds boolean to do it. I felt cheap doing it that way :-) Everything else looks good to me. > Add time-based execution to cassandra-stress > > > Key: CASSANDRA-7468 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7468 > Project: Cassandra > Issue Type: Improvement > Components: Tools >Reporter: Matt Kennedy >Assignee: Matt Kennedy >Priority: Minor > Fix For: 2.1.1 > > Attachments: 7468v2.txt, trunk-7468-rebase.patch, trunk-7468.patch > > -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069396#comment-14069396 ] Benedict commented on CASSANDRA-7546: - My concern with the approach you've outlined is that we're barely a hair's breadth from a lock: as soon as we hit _any_ contention, we inflate to locking behaviour. This is good for large partitions, and most likely bad for small ones, and more to the point seems barely worth the complexity of not just making it a lock in the first place. On further consideration, I think I would perhaps prefer to run this lock-inflation behaviour based on the size of the aborted changes, so if the amount of work we've wasted exceeds some threshold we decide it's high time all threads were stopped to let us finish. We could in this scenario flip a switch that requires all modifications to acquire the monitor once we hit this threshold once; I would be fine with this behaviour, and it would be simple. I do wonder how much of a problem this is in 2.1, though. I wonder if the largest problem with these racy modifications isn't actually the massive amounts of memtable arena allocations they incur in 2.0 with all their transformation.apply() calls (which reallocate the mutation on the arena), which is most likely what causes the promotion failures, as they cannot be collected. I wonder if we shouldn't simply backport the logic to only allocate these once, or at most twice (the first time we race). It seems much more likely to me that this is where the pain is being felt. > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, > suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7559) Switch Stress from using math3.pair because it is unserializable
[
https://issues.apache.org/jira/browse/CASSANDRA-7559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069424#comment-14069424
]
Russell Alexander Spitzer commented on CASSANDRA-7559:
--
[~tjake], This is the patch we were talking about a few days ago for fixing
StressD. I would be grateful if you could review and make sure I haven't done
anything destructive.
> Switch Stress from using math3.pair because it is unserializable
> -
>
> Key: CASSANDRA-7559
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7559
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Russell Alexander Spitzer
>Assignee: Russell Alexander Spitzer
> Attachments: CASSANDRA-7559-2.1.patch
>
>
> Stress uses org.apache.commons.math3.util.Pair to hold information in
> settings because eventually it is used in commons.math3.distributions. This
> makes the settings unserializable so we can't run with StressDemon.
> {code}
> /bin/cassandra-stress user no_warmup "ops(insert=1)" n=1
> profile=cqlstress-example.yaml -sendto 127.0.0.1
> Exception in thread "main" java.io.NotSerializableException:
> org.apache.commons.math3.util.Pair
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1183)
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
> at java.util.ArrayList.writeObject(ArrayList.java:742)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:988)
> at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1495)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1547)
> at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1508)
> at
> java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1431)
> at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177)
> at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347)
> at org.apache.cassandra.stress.Stress.main(Stress.java:78)
> Control-C caught. Canceling running action and shutting down...
> {code}
> To fix this we can pass around serializable pairs and convert to
> commons.math3 pairs before we actually pass the objects to the distribution
> code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
git commit: Give read access to system.schema_usertypes to all authenticated users
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 3b14f6a5d -> 3d2da7987 Give read access to system.schema_usertypes to all authenticated users patch by Mike Adamson; reviewed by Aleksey Yeschenko for CASSANDRA-7578 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d2da798 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d2da798 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d2da798 Branch: refs/heads/cassandra-2.1 Commit: 3d2da7987e3a6d0f1e088af9c661068335cc9f67 Parents: 3b14f6a Author: Mike Adamson Authored: Mon Jul 21 14:36:01 2014 +0100 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:27:22 2014 +0300 -- CHANGES.txt| 2 ++ src/java/org/apache/cassandra/service/ClientState.java | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 2aa2efa..f05c98e 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -7,6 +7,8 @@ * Configurable client timeout for cqlsh (CASSANDRA-7516) * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) * Fix native protocol drop user type notification (CASSANDRA-7571) + * Give read access to system.schema_usertypes to all authenticated users + (CASSANDRA-7578) 2.1.0-rc4 http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/src/java/org/apache/cassandra/service/ClientState.java -- diff --git a/src/java/org/apache/cassandra/service/ClientState.java b/src/java/org/apache/cassandra/service/ClientState.java index be3b895..c0396cb 100644 --- a/src/java/org/apache/cassandra/service/ClientState.java +++ b/src/java/org/apache/cassandra/service/ClientState.java @@ -67,7 +67,8 @@ public class ClientState SystemKeyspace.PEERS_CF, SystemKeyspace.SCHEMA_KEYSPACES_CF, SystemKeyspace.SCHEMA_COLUMNFAMILIES_CF, - SystemKeyspace.SCHEMA_COLUMNS_CF }; + SystemKeyspace.SCHEMA_COLUMNS_CF, + SystemKeyspace.SCHEMA_USER_TYPES_CF}; for (String cf : cfs) READABLE_SYSTEM_RESOURCES.add(DataResource.columnFamily(Keyspace.SYSTEM_KS, cf));
[1/2] git commit: Give read access to system.schema_usertypes to all authenticated users
Repository: cassandra Updated Branches: refs/heads/trunk 19b7d56fe -> 9c9fa1498 Give read access to system.schema_usertypes to all authenticated users patch by Mike Adamson; reviewed by Aleksey Yeschenko for CASSANDRA-7578 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3d2da798 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3d2da798 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3d2da798 Branch: refs/heads/trunk Commit: 3d2da7987e3a6d0f1e088af9c661068335cc9f67 Parents: 3b14f6a Author: Mike Adamson Authored: Mon Jul 21 14:36:01 2014 +0100 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:27:22 2014 +0300 -- CHANGES.txt| 2 ++ src/java/org/apache/cassandra/service/ClientState.java | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index 2aa2efa..f05c98e 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -7,6 +7,8 @@ * Configurable client timeout for cqlsh (CASSANDRA-7516) * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) * Fix native protocol drop user type notification (CASSANDRA-7571) + * Give read access to system.schema_usertypes to all authenticated users + (CASSANDRA-7578) 2.1.0-rc4 http://git-wip-us.apache.org/repos/asf/cassandra/blob/3d2da798/src/java/org/apache/cassandra/service/ClientState.java -- diff --git a/src/java/org/apache/cassandra/service/ClientState.java b/src/java/org/apache/cassandra/service/ClientState.java index be3b895..c0396cb 100644 --- a/src/java/org/apache/cassandra/service/ClientState.java +++ b/src/java/org/apache/cassandra/service/ClientState.java @@ -67,7 +67,8 @@ public class ClientState SystemKeyspace.PEERS_CF, SystemKeyspace.SCHEMA_KEYSPACES_CF, SystemKeyspace.SCHEMA_COLUMNFAMILIES_CF, - SystemKeyspace.SCHEMA_COLUMNS_CF }; + SystemKeyspace.SCHEMA_COLUMNS_CF, + SystemKeyspace.SCHEMA_USER_TYPES_CF}; for (String cf : cfs) READABLE_SYSTEM_RESOURCES.add(DataResource.columnFamily(Keyspace.SYSTEM_KS, cf));
[2/2] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9c9fa149 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9c9fa149 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9c9fa149 Branch: refs/heads/trunk Commit: 9c9fa1498096c51db7692a453f4af77f9276adfa Parents: 19b7d56 3d2da79 Author: Aleksey Yeschenko Authored: Tue Jul 22 01:30:10 2014 +0300 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:30:10 2014 +0300 -- CHANGES.txt| 2 ++ src/java/org/apache/cassandra/service/ClientState.java | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9c9fa149/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9c9fa149/src/java/org/apache/cassandra/service/ClientState.java --
[jira] [Updated] (CASSANDRA-7578) Give read access to system.schema_usertypes to all authenticated users
[
https://issues.apache.org/jira/browse/CASSANDRA-7578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-7578:
-
Summary: Give read access to system.schema_usertypes to all authenticated
users (was: Add system.schema_usertypes to the readable-by-default whitelist)
> Give read access to system.schema_usertypes to all authenticated users
> --
>
> Key: CASSANDRA-7578
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7578
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Minor
> Fix For: 2.1.1
>
> Attachments: 7578.patch
>
>
> When I try to login to cqlsh as a non-superuser I get the following error:
> {noformat}
> ErrorMessage code=2100 [Unauthorized] message="User mike has no SELECT
> permission on or any of its parents"
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069450#comment-14069450 ] Alex Liu commented on CASSANDRA-6454: - cassandra_pig.yaml uses Murmur3Partitioner instead of ByteOrderedPartitioner. ByteOrderedPartitioner is used for other unit tests. > Pig support for hadoop CqlInputFormat > - > > Key: CASSANDRA-6454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6454 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > Fix For: 2.0.10 > > Attachments: 6454-2.0-branch.txt, 6454-v2-2.0-branch.txt, > 6454-v3-2.0-branch.txt, 6454-v3-2.1-branch.txt > > > CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it -- This message was sent by Atlassian JIRA (v6.2#6252)
git commit: Fix ReversedType(DateType) mapping to native protocol
Repository: cassandra Updated Branches: refs/heads/cassandra-2.0 875480958 -> ff211a4c4 Fix ReversedType(DateType) mapping to native protocol patch by Ben Hood; reviewed by Aleksey Yeschenko for CASSANDRA-7576 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff211a4c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff211a4c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff211a4c Branch: refs/heads/cassandra-2.0 Commit: ff211a4c445cd74859aabf00f3ec81a5e69473f2 Parents: 8754809 Author: Aleksey Yeschenko Authored: Tue Jul 22 01:42:24 2014 +0300 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:42:24 2014 +0300 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/transport/DataType.java | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index d339309..9909760 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.0.10 + * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576) * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541) * Fix range merging when DES scores are zero (CASSANDRA-7535) * Warn when SSL certificates have expired (CASSANDRA-7528) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/src/java/org/apache/cassandra/transport/DataType.java -- diff --git a/src/java/org/apache/cassandra/transport/DataType.java b/src/java/org/apache/cassandra/transport/DataType.java index 1656d24..8cd7194 100644 --- a/src/java/org/apache/cassandra/transport/DataType.java +++ b/src/java/org/apache/cassandra/transport/DataType.java @@ -146,8 +146,9 @@ public enum DataType implements OptionCodec.Codecable // shouldn't have to care about it. if (type instanceof ReversedType) type = ((ReversedType)type).baseType; + // For compatibility sake, we still return DateType as the timestamp type in resultSet metadata (#5723) -else if (type instanceof DateType) +if (type instanceof DateType) type = TimestampType.instance; DataType dt = dataTypeMap.get(type);
[3/3] git commit: Merge branch 'cassandra-2.1' into trunk
Merge branch 'cassandra-2.1' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/3a7242ec Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/3a7242ec Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/3a7242ec Branch: refs/heads/trunk Commit: 3a7242eca51e703c33ed8a4ca7d61a66018f7a81 Parents: 9c9fa14 f9af6ab Author: Aleksey Yeschenko Authored: Tue Jul 22 01:46:32 2014 +0300 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:46:32 2014 +0300 -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/transport/DataType.java | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/3a7242ec/CHANGES.txt --
[jira] [Commented] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069459#comment-14069459 ] Brandon Williams commented on CASSANDRA-6454: - Is there a reason we need to switch to it for this? (not that I disagree with it at all) > Pig support for hadoop CqlInputFormat > - > > Key: CASSANDRA-6454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6454 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > Fix For: 2.0.10 > > Attachments: 6454-2.0-branch.txt, 6454-v2-2.0-branch.txt, > 6454-v3-2.0-branch.txt, 6454-v3-2.1-branch.txt > > > CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it -- This message was sent by Atlassian JIRA (v6.2#6252)
[1/2] git commit: Fix ReversedType(DateType) mapping to native protocol
Repository: cassandra Updated Branches: refs/heads/cassandra-2.1 3d2da7987 -> f9af6ab58 Fix ReversedType(DateType) mapping to native protocol patch by Ben Hood; reviewed by Aleksey Yeschenko for CASSANDRA-7576 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff211a4c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff211a4c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff211a4c Branch: refs/heads/cassandra-2.1 Commit: ff211a4c445cd74859aabf00f3ec81a5e69473f2 Parents: 8754809 Author: Aleksey Yeschenko Authored: Tue Jul 22 01:42:24 2014 +0300 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:42:24 2014 +0300 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/transport/DataType.java | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index d339309..9909760 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.0.10 + * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576) * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541) * Fix range merging when DES scores are zero (CASSANDRA-7535) * Warn when SSL certificates have expired (CASSANDRA-7528) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/src/java/org/apache/cassandra/transport/DataType.java -- diff --git a/src/java/org/apache/cassandra/transport/DataType.java b/src/java/org/apache/cassandra/transport/DataType.java index 1656d24..8cd7194 100644 --- a/src/java/org/apache/cassandra/transport/DataType.java +++ b/src/java/org/apache/cassandra/transport/DataType.java @@ -146,8 +146,9 @@ public enum DataType implements OptionCodec.Codecable // shouldn't have to care about it. if (type instanceof ReversedType) type = ((ReversedType)type).baseType; + // For compatibility sake, we still return DateType as the timestamp type in resultSet metadata (#5723) -else if (type instanceof DateType) +if (type instanceof DateType) type = TimestampType.instance; DataType dt = dataTypeMap.get(type);
[1/3] git commit: Fix ReversedType(DateType) mapping to native protocol
Repository: cassandra Updated Branches: refs/heads/trunk 9c9fa1498 -> 3a7242eca Fix ReversedType(DateType) mapping to native protocol patch by Ben Hood; reviewed by Aleksey Yeschenko for CASSANDRA-7576 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/ff211a4c Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/ff211a4c Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/ff211a4c Branch: refs/heads/trunk Commit: ff211a4c445cd74859aabf00f3ec81a5e69473f2 Parents: 8754809 Author: Aleksey Yeschenko Authored: Tue Jul 22 01:42:24 2014 +0300 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:42:24 2014 +0300 -- CHANGES.txt | 1 + src/java/org/apache/cassandra/transport/DataType.java | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index d339309..9909760 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 2.0.10 + * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576) * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541) * Fix range merging when DES scores are zero (CASSANDRA-7535) * Warn when SSL certificates have expired (CASSANDRA-7528) http://git-wip-us.apache.org/repos/asf/cassandra/blob/ff211a4c/src/java/org/apache/cassandra/transport/DataType.java -- diff --git a/src/java/org/apache/cassandra/transport/DataType.java b/src/java/org/apache/cassandra/transport/DataType.java index 1656d24..8cd7194 100644 --- a/src/java/org/apache/cassandra/transport/DataType.java +++ b/src/java/org/apache/cassandra/transport/DataType.java @@ -146,8 +146,9 @@ public enum DataType implements OptionCodec.Codecable // shouldn't have to care about it. if (type instanceof ReversedType) type = ((ReversedType)type).baseType; + // For compatibility sake, we still return DateType as the timestamp type in resultSet metadata (#5723) -else if (type instanceof DateType) +if (type instanceof DateType) type = TimestampType.instance; DataType dt = dataTypeMap.get(type);
[jira] [Commented] (CASSANDRA-7570) CqlPagingRecordReader is broken
[ https://issues.apache.org/jira/browse/CASSANDRA-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069460#comment-14069460 ] Alex Liu commented on CASSANDRA-7570: - It can be fixed at CPRR by changing > to >= (< to <=), then filter the first row if it's equal to the last row fetched. Since we have new CRR, we should depreciate CPRR. Should we fix this issue or just warn user that compact table is broken, should use the new CRR instead. > CqlPagingRecordReader is broken > --- > > Key: CASSANDRA-7570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7570 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Brandon Williams >Assignee: Alex Liu > Fix For: 2.0.10 > > > As mentioned on CASSANDRA-7059, it broke CPRR. It's not quite as simple as > changing the greater than to a greater than equal, either, since that makes > the task run forever. -- This message was sent by Atlassian JIRA (v6.2#6252)
[2/2] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f9af6ab5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f9af6ab5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f9af6ab5 Branch: refs/heads/cassandra-2.1 Commit: f9af6ab5874f4d33e32af1af8934bd7c0bed1724 Parents: 3d2da79 ff211a4 Author: Aleksey Yeschenko Authored: Tue Jul 22 01:46:11 2014 +0300 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:46:11 2014 +0300 -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/transport/DataType.java | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/CHANGES.txt -- diff --cc CHANGES.txt index f05c98e,9909760..028128d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,29 -1,5 +1,31 @@@ -2.0.10 +2.1.1 + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Fail to start if commit log replay detects a problem (CASSANDRA-7125) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) + * Fix native protocol drop user type notification (CASSANDRA-7571) + * Give read access to system.schema_usertypes to all authenticated users + (CASSANDRA-7578) ++Merged from 2.0: + * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576) + + +2.1.0-rc4 + * Updated memtable_cleanup_threshold and memtable_flush_writers defaults + (CASSANDRA-7551) + * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505) + * Anti-compaction proceeds if any part of the repair failed (CASANDRA-7521) + * Add missing table name to DROP INDEX responses and notifications (CASSANDRA-7539) + * Bump CQL version to 3.2.0 and update CQL documentation (CASSANDRA-7527) + * Fix configuration error message when running nodetool ring (CASSANDRA-7508) + * Support conditional updates, tuple type, and the v3 protocol in cqlsh (CASSANDRA-7509) + * Handle queries on multiple secondary index types (CASSANDRA-7525) + * Fix cqlsh authentication with v3 native protocol (CASSANDRA-7564) + * Fix NPE when unknown prepared statement ID is used (CASSANDRA-7454) +Merged from 2.0: * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541) * Fix range merging when DES scores are zero (CASSANDRA-7535) * Warn when SSL certificates have expired (CASSANDRA-7528) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/src/java/org/apache/cassandra/transport/DataType.java --
[2/3] git commit: Merge branch 'cassandra-2.0' into cassandra-2.1
Merge branch 'cassandra-2.0' into cassandra-2.1 Conflicts: CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f9af6ab5 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f9af6ab5 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f9af6ab5 Branch: refs/heads/trunk Commit: f9af6ab5874f4d33e32af1af8934bd7c0bed1724 Parents: 3d2da79 ff211a4 Author: Aleksey Yeschenko Authored: Tue Jul 22 01:46:11 2014 +0300 Committer: Aleksey Yeschenko Committed: Tue Jul 22 01:46:11 2014 +0300 -- CHANGES.txt | 2 ++ src/java/org/apache/cassandra/transport/DataType.java | 3 ++- 2 files changed, 4 insertions(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/CHANGES.txt -- diff --cc CHANGES.txt index f05c98e,9909760..028128d --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,29 -1,5 +1,31 @@@ -2.0.10 +2.1.1 + * Add listen_interface and rpc_interface options (CASSANDRA-7417) + * Fail to start if commit log replay detects a problem (CASSANDRA-7125) + * Improve schema merge performance (CASSANDRA-7444) + * Adjust MT depth based on # of partition validating (CASSANDRA-5263) + * Optimise NativeCell comparisons (CASSANDRA-6755) + * Configurable client timeout for cqlsh (CASSANDRA-7516) + * Include snippet of CQL query near syntax error in messages (CASSANDRA-7111) + * Fix native protocol drop user type notification (CASSANDRA-7571) + * Give read access to system.schema_usertypes to all authenticated users + (CASSANDRA-7578) ++Merged from 2.0: + * Fix ReversedType(DateType) mapping to native protocol (CASSANDRA-7576) + + +2.1.0-rc4 + * Updated memtable_cleanup_threshold and memtable_flush_writers defaults + (CASSANDRA-7551) + * (Windows) fix startup when WMI memory query fails (CASSANDRA-7505) + * Anti-compaction proceeds if any part of the repair failed (CASANDRA-7521) + * Add missing table name to DROP INDEX responses and notifications (CASSANDRA-7539) + * Bump CQL version to 3.2.0 and update CQL documentation (CASSANDRA-7527) + * Fix configuration error message when running nodetool ring (CASSANDRA-7508) + * Support conditional updates, tuple type, and the v3 protocol in cqlsh (CASSANDRA-7509) + * Handle queries on multiple secondary index types (CASSANDRA-7525) + * Fix cqlsh authentication with v3 native protocol (CASSANDRA-7564) + * Fix NPE when unknown prepared statement ID is used (CASSANDRA-7454) +Merged from 2.0: * (Windows) force range-based repair to non-sequential mode (CASSANDRA-7541) * Fix range merging when DES scores are zero (CASSANDRA-7535) * Warn when SSL certificates have expired (CASSANDRA-7528) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f9af6ab5/src/java/org/apache/cassandra/transport/DataType.java --
[jira] [Commented] (CASSANDRA-7570) CqlPagingRecordReader is broken
[ https://issues.apache.org/jira/browse/CASSANDRA-7570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069463#comment-14069463 ] Brandon Williams commented on CASSANDRA-7570: - We probably at least need to fix it in 2.0, since that's where we broke it, but it's also a bit late to deprecate in 2.1. > CqlPagingRecordReader is broken > --- > > Key: CASSANDRA-7570 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7570 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Brandon Williams >Assignee: Alex Liu > Fix For: 2.0.10 > > > As mentioned on CASSANDRA-7059, it broke CPRR. It's not quite as simple as > changing the greater than to a greater than equal, either, since that makes > the task run forever. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069504#comment-14069504
]
graham sanderson commented on CASSANDRA-7546:
-
{quote}
I do wonder how much of a problem this is in 2.1, though. I wonder if the
largest problem with these racy modifications isn't actually the massive
amounts of memtable arena allocations they incur in 2.0 with all their
transformation.apply() calls (which reallocate the mutation on the arena),
which is most likely what causes the promotion failures, as they cannot be
collected. I wonder if we shouldn't simply backport the logic to only allocate
these once, or at most twice (the first time we race). It seems much more
likely to me that this is where the pain is being felt.
{quote}
I'm not sure which changes you are talking about back-porting and whether the
"at most twice" refers to looping once then locking. Certainly avoiding any
repeated cloning of the cells is good, however I'm still pretty sure based on
PrintFLSStatistics that the slabs themselves are not the biggest problem (I
suspect SnapTreeMap nodes, combined with high rebalancing cost of huge trees in
the hint case since the keys are almost entirely sorted).
Are you suggesting a one way switch per Atomic*Columns instance that flips
after a number waster "operations"? That sounds reasonable... I'd expect that a
partition for a table is either likely to have high contention or not based on
the schema design/use case. I have no idea how long these instances hang around
in practice (presumably not insanely long)
> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -
>
> Key: CASSANDRA-7546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: graham sanderson
>Assignee: graham sanderson
> Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt,
> suggestion1.txt, suggestion1_21.txt
>
>
> In order to preserve atomicity, this code attempts to read, clone/update,
> then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some
> fairly staggering memory growth (the more cores on your machine the worst it
> gets).
> Whilst many usage patterns don't do highly concurrent updates to the same
> partition, hinting today, does, and in this case wild (order(s) of magnitude
> more than expected) memory allocation rates can be seen (especially when the
> updates being hinted are small updates to different partitions which can
> happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation
> whilst not slowing down the very common un-contended case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[
https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069504#comment-14069504
]
graham sanderson edited comment on CASSANDRA-7546 at 7/21/14 11:28 PM:
---
{quote}
I do wonder how much of a problem this is in 2.1, though. I wonder if the
largest problem with these racy modifications isn't actually the massive
amounts of memtable arena allocations they incur in 2.0 with all their
transformation.apply() calls (which reallocate the mutation on the arena),
which is most likely what causes the promotion failures, as they cannot be
collected. I wonder if we shouldn't simply backport the logic to only allocate
these once, or at most twice (the first time we race). It seems much more
likely to me that this is where the pain is being felt.
{quote}
I'm not sure which changes you are talking about back-porting and whether the
"at most twice" refers to looping once then locking, and what is reasonable to
modify in 2.0.x now. Certainly avoiding any repeated cloning of the cells is
good, however I'm still pretty sure based on PrintFLSStatistics that the slabs
themselves are not the biggest problem (I suspect SnapTreeMap nodes, combined
with high rebalancing cost of huge trees in the hint case since the keys are
almost entirely sorted).
Are you suggesting a one way switch per Atomic*Columns instance that flips
after a number waster "operations"? That sounds reasonable... I'd expect that a
partition for a table is either likely to have high contention or not based on
the schema design/use case. I have no idea how long these instances hang around
in practice (presumably not insanely long) at least if they are being actively
used since I assume they get flushed eventually in that case, and if they
aren't it doesn't really matter anyway
was (Author: graham sanderson):
{quote}
I do wonder how much of a problem this is in 2.1, though. I wonder if the
largest problem with these racy modifications isn't actually the massive
amounts of memtable arena allocations they incur in 2.0 with all their
transformation.apply() calls (which reallocate the mutation on the arena),
which is most likely what causes the promotion failures, as they cannot be
collected. I wonder if we shouldn't simply backport the logic to only allocate
these once, or at most twice (the first time we race). It seems much more
likely to me that this is where the pain is being felt.
{quote}
I'm not sure which changes you are talking about back-porting and whether the
"at most twice" refers to looping once then locking. Certainly avoiding any
repeated cloning of the cells is good, however I'm still pretty sure based on
PrintFLSStatistics that the slabs themselves are not the biggest problem (I
suspect SnapTreeMap nodes, combined with high rebalancing cost of huge trees in
the hint case since the keys are almost entirely sorted).
Are you suggesting a one way switch per Atomic*Columns instance that flips
after a number waster "operations"? That sounds reasonable... I'd expect that a
partition for a table is either likely to have high contention or not based on
the schema design/use case. I have no idea how long these instances hang around
in practice (presumably not insanely long)
> AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
> -
>
> Key: CASSANDRA-7546
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7546
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: graham sanderson
>Assignee: graham sanderson
> Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt,
> suggestion1.txt, suggestion1_21.txt
>
>
> In order to preserve atomicity, this code attempts to read, clone/update,
> then CAS the state of the partition.
> Under heavy contention for updating a single partition this can cause some
> fairly staggering memory growth (the more cores on your machine the worst it
> gets).
> Whilst many usage patterns don't do highly concurrent updates to the same
> partition, hinting today, does, and in this case wild (order(s) of magnitude
> more than expected) memory allocation rates can be seen (especially when the
> updates being hinted are small updates to different partitions which can
> happen very fast on their own) - see CASSANDRA-7545
> It would be best to eliminate/reduce/limit the spinning memory allocation
> whilst not slowing down the very common un-contended case.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7576) DateType columns not properly converted to TimestampType when in ReversedType columns.
[
https://issues.apache.org/jira/browse/CASSANDRA-7576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069557#comment-14069557
]
Karl Rieb commented on CASSANDRA-7576:
--
Thanks [~iamaleksey]! No problems with the 'patch by', it's fine by me.
Do you know approximately how long before *2.0.10* is released? I'm guessing
some time around August?
> DateType columns not properly converted to TimestampType when in ReversedType
> columns.
> --
>
> Key: CASSANDRA-7576
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7576
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Karl Rieb
>Assignee: Karl Rieb
> Fix For: 2.0.10, 2.1.1
>
> Attachments: DataType_CASSANDRA_7576.patch
>
> Original Estimate: 0.25h
> Remaining Estimate: 0.25h
>
> The {{org.apache.cassandra.transport.DataType.fromType(AbstractType)}} method
> has a bug that prevents sending the correct Protocol ID for reversed
> {{DateType}} columns. This results in clients receiving Protocol ID {{0}},
> which maps to a {{CUSTOM}} type, for timestamp columns that are clustered in
> reverse order.
> Some clients can handle this properly since they recognize the
> {{"org.apache.cassandra.db.marshal.DateType"}} marshaling type, however the
> native Datastax java-driver does not. It will produce errors like the one
> below when trying to prepare queries against such tables:
> {noformat}
> com.datastax.driver.core.exceptions.InvalidTypeException: Invalid type
> for value 2 of CQL type 'org.apache.cassandra.db.marshal.DateType', expecting
> class java.nio.ByteBuffer but class java.util.Date provided
> at com.datastax.driver.core.BoundStatement.bind(BoundStatement.java:190)
> at
> com.datastax.driver.core.DefaultPreparedStatement.bind(DefaultPreparedStatement.java:103)
> {noformat}
> On the Cassandra side, there is a check for {{DateType}} columns that is
> supposed to convert these columns to TimestampType. However, the check is
> skipped when the column is also reversed. Specifically:
> {code:title=DataType.java|borderStyle=solid}
> public static Pair fromType(AbstractType type)
> {
> // For CQL3 clients, ReversedType is an implementation detail and they
> // shouldn't have to care about it.
> if (type instanceof ReversedType)
> type = ((ReversedType)type).baseType;
> // For compatibility sake, we still return DateType as the timestamp type
> in resultSet metadata (#5723)
> else if (type instanceof DateType)
> type = TimestampType.instance;
> // ...
> {code}
> The *else if* should be changed to just an *if*, like so:
> {code:title=DataType.java|borderStyle=solid}
> public static Pair fromType(AbstractType type)
> {
> // For CQL3 clients, ReversedType is an implementation detail and they
> // shouldn't have to care about it.
> if (type instanceof ReversedType)
> type = ((ReversedType)type).baseType;
> // For compatibility sake, we still return DateType as the timestamp type
> in resultSet metadata (#5723)
> if (type instanceof DateType)
> type = TimestampType.instance;
> // ...
> {code}
> This bug is preventing us from upgrading our 1.2.11 cluster to 2.0.9 because
> our clients keep throwing exceptions trying to read or write data to tables
> with reversed timestamp columns. This issue can be reproduced by creating a
> CQL table in Cassandra 1.2.11 that clusters on a timestamp in reverse, then
> upgrading the node to 2.0.9. When querying the metadata for the table, the
> node will return Protocol ID 0 (CUSTOM) instead of Protocol ID 11 (TIMESTAMP).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069620#comment-14069620 ] Alex Liu commented on CASSANDRA-6454: - Most of Cassandra deployment uses Murmur3Partitioner, testing on Murmur3Partitioner covers the general case. Some unit tests still use old ByteOrderedPartitioner, so just update the cassanra.yaml to Murmur3Partitioner breaks other unit tests. That's the reason I create a new yaml file. > Pig support for hadoop CqlInputFormat > - > > Key: CASSANDRA-6454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6454 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > Fix For: 2.0.10 > > Attachments: 6454-2.0-branch.txt, 6454-v2-2.0-branch.txt, > 6454-v3-2.0-branch.txt, 6454-v3-2.1-branch.txt > > > CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069666#comment-14069666 ] graham sanderson commented on CASSANDRA-7546: - Alternatively if you are saying, let each thread keep working while they still believe they can win, or while they have something to do that can be reused if they lose, then maybe give them one last shot to try again if they lose and haven't done anything reusable, then make them block... I'm okay with that. (of course on 2.0.x. today, that pretty much boils down to your patch!) > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, > suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7582) 2.1 multi-dc upgrade errors
[
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-7582:
Assignee: Russ Hatch
> 2.1 multi-dc upgrade errors
> ---
>
> Key: CASSANDRA-7582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Russ Hatch
>
> Multi-dc upgrade [was working from 2.0 -> 2.1 fairly
> recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
> but is currently failing.
> Running
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
> I get the following errors when starting 2.1 upgraded from 2.0:
> {code}
> ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay
> failed due to replaying a mutation for a missing table. This error can be
> ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on
> the command line
> ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception
> encountered during startup
> java.lang.RuntimeException:
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
> cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546)
> [main/:na]
> Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't
> find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115)
> ~[main/:na]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (CASSANDRA-7582) 2.1 multi-dc upgrade errors
Ryan McGuire created CASSANDRA-7582:
---
Summary: 2.1 multi-dc upgrade errors
Key: CASSANDRA-7582
URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
Project: Cassandra
Issue Type: Bug
Reporter: Ryan McGuire
Multi-dc upgrade [was working from 2.0 -> 2.1 fairly
recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
but is currently failing.
Running
upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
I get the following errors when starting 2.1 upgraded from 2.0:
{code}
ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay
failed due to replaying a mutation for a missing table. This error can be
ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on the
command line
ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception
encountered during startup
java.lang.RuntimeException:
org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
[main/:na]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546)
[main/:na]
Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
at
org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
~[main/:na]
at
org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
~[main/:na]
at
org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
~[main/:na]
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
~[main/:na]
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
~[main/:na]
at
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137)
~[main/:na]
at
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115)
~[main/:na]
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7582) 2.1 multi-dc upgrade errors
[
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069788#comment-14069788
]
Ryan McGuire commented on CASSANDRA-7582:
-
[~rhatch] can you repro and bisect this? It was passing last week.
> 2.1 multi-dc upgrade errors
> ---
>
> Key: CASSANDRA-7582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Russ Hatch
>
> Multi-dc upgrade [was working from 2.0 -> 2.1 fairly
> recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
> but is currently failing.
> Running
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
> I get the following errors when starting 2.1 upgraded from 2.0:
> {code}
> ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay
> failed due to replaying a mutation for a missing table. This error can be
> ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on
> the command line
> ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception
> encountered during startup
> java.lang.RuntimeException:
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
> cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546)
> [main/:na]
> Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't
> find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115)
> ~[main/:na]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7582) 2.1 multi-dc upgrade errors
[
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryan McGuire updated CASSANDRA-7582:
Since Version: 2.1 rc3
> 2.1 multi-dc upgrade errors
> ---
>
> Key: CASSANDRA-7582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ryan McGuire
>Assignee: Russ Hatch
>
> Multi-dc upgrade [was working from 2.0 -> 2.1 fairly
> recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
> but is currently failing.
> Running
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
> I get the following errors when starting 2.1 upgraded from 2.0:
> {code}
> ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay
> failed due to replaying a mutation for a missing table. This error can be
> ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on
> the command line
> ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception
> encountered during startup
> java.lang.RuntimeException:
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
> cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546)
> [main/:na]
> Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't
> find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115)
> ~[main/:na]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (CASSANDRA-7370) Create a new system table "node_config" to load cassandra.yaml config data.
[ https://issues.apache.org/jira/browse/CASSANDRA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7370: Attachment: (was: 7370-v1-prototype.diff) > Create a new system table "node_config" to load cassandra.yaml config data. > --- > > Key: CASSANDRA-7370 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7370 > Project: Cassandra > Issue Type: Wish > Components: Config >Reporter: Hayato Shimizu >Assignee: Robert Stupp >Priority: Minor > Labels: ponies > Attachments: 7370-v3.txt > > > Currently the node configuration information specified in cassandra.yaml can > only be viewed via JMX or by looking at the file on individual machines. > As an administrator, it would be extremely useful to be able to execute > queries like the following example; > select concurrent_reads from system.node_config; > which will list all the concurrent_reads value from all of the nodes in a > cluster. > This will require a new table in the system keyspace and the data to be > loaded (if required) during the bootstrap, and updated when MBeans attribute > value updates are performed. The data from other nodes in the cluster is also > required in the table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7370) Create a new system table "node_config" to load cassandra.yaml config data.
[ https://issues.apache.org/jira/browse/CASSANDRA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7370: Attachment: 7370-v3.txt merge trunk > Create a new system table "node_config" to load cassandra.yaml config data. > --- > > Key: CASSANDRA-7370 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7370 > Project: Cassandra > Issue Type: Wish > Components: Config >Reporter: Hayato Shimizu >Assignee: Robert Stupp >Priority: Minor > Labels: ponies > Attachments: 7370-v3.txt > > > Currently the node configuration information specified in cassandra.yaml can > only be viewed via JMX or by looking at the file on individual machines. > As an administrator, it would be extremely useful to be able to execute > queries like the following example; > select concurrent_reads from system.node_config; > which will list all the concurrent_reads value from all of the nodes in a > cluster. > This will require a new table in the system keyspace and the data to be > loaded (if required) during the bootstrap, and updated when MBeans attribute > value updates are performed. The data from other nodes in the cluster is also > required in the table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7370) Create a new system table "node_config" to load cassandra.yaml config data.
[ https://issues.apache.org/jira/browse/CASSANDRA-7370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Stupp updated CASSANDRA-7370: Attachment: (was: 7370-v2-with-jmx.diff) > Create a new system table "node_config" to load cassandra.yaml config data. > --- > > Key: CASSANDRA-7370 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7370 > Project: Cassandra > Issue Type: Wish > Components: Config >Reporter: Hayato Shimizu >Assignee: Robert Stupp >Priority: Minor > Labels: ponies > Attachments: 7370-v3.txt > > > Currently the node configuration information specified in cassandra.yaml can > only be viewed via JMX or by looking at the file on individual machines. > As an administrator, it would be extremely useful to be able to execute > queries like the following example; > select concurrent_reads from system.node_config; > which will list all the concurrent_reads value from all of the nodes in a > cluster. > This will require a new table in the system keyspace and the data to be > loaded (if required) during the bootstrap, and updated when MBeans attribute > value updates are performed. The data from other nodes in the cluster is also > required in the table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (CASSANDRA-7582) 2.1 multi-dc upgrade errors
[
https://issues.apache.org/jira/browse/CASSANDRA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Ellis updated CASSANDRA-7582:
--
Component/s: Core
Priority: Critical (was: Major)
Fix Version/s: 2.1.0
> 2.1 multi-dc upgrade errors
> ---
>
> Key: CASSANDRA-7582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7582
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: Ryan McGuire
>Assignee: Russ Hatch
>Priority: Critical
> Fix For: 2.1.0
>
>
> Multi-dc upgrade [was working from 2.0 -> 2.1 fairly
> recently|http://cassci.datastax.com/job/cassandra_upgrade_dtest/55/testReport/upgrade_through_versions_test/TestUpgrade_from_cassandra_2_0_latest_tag_to_cassandra_2_1_HEAD/],
> but is currently failing.
> Running
> upgrade_through_versions_test.py:TestUpgrade_from_cassandra_2_0_HEAD_to_cassandra_2_1_HEAD.bootstrap_multidc_test
> I get the following errors when starting 2.1 upgraded from 2.0:
> {code}
> ERROR [main] 2014-07-21 23:54:20,862 CommitLog.java:143 - Commit log replay
> failed due to replaying a mutation for a missing table. This error can be
> ignored by providing -Dcassandra.commitlog.stop_on_missing_tables=false on
> the command line
> ERROR [main] 2014-07-21 23:54:20,869 CassandraDaemon.java:474 - Exception
> encountered during startup
> java.lang.RuntimeException:
> org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't find
> cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:300)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:457)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:546)
> [main/:na]
> Caused by: org.apache.cassandra.db.UnknownColumnFamilyException: Couldn't
> find cfId=a1b676f3-0c5d-3276-bfd5-07cf43397004
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeCfId(ColumnFamilySerializer.java:164)
> ~[main/:na]
> at
> org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:97)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserializeOneCf(Mutation.java:353)
> ~[main/:na]
> at
> org.apache.cassandra.db.Mutation$MutationSerializer.deserialize(Mutation.java:333)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:365)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:98)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:137)
> ~[main/:na]
> at
> org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:115)
> ~[main/:na]
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069862#comment-14069862 ] Vijay commented on CASSANDRA-7438: -- Attached patch makes the off heap/Serialization Cache configurable. (the default is still SerializationCache). Regarding performance, the performance of the new cache is obviously better when the JNI overhead is less than the GC overhead, but for the smaller sized caches which can fit in memory the performance is little lower which is understandable (but both of them out perform pagecache performance by a large number). Here are the numbers. OffheapCacheProvider Running READ with 1200 threads for 1000 iterations ops ,op/s, key/s,mean, med, .95, .99,.999, max, time, stderr 2030355 , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8, 3014.5,1.0, 0.0 2395480 , 202845, 202845, 5.8, 5.4, 5.8,20.2, 522.4, 545.9,2.8, 0.0 2638600 , 221368, 221368, 5.4, 5.3, 5.8,16.3,78.8, 131.5,3.9, 0.57860 2891705 , 221976, 221976, 5.4, 5.3, 5.6, 6.2,15.2, 19.2,5.0, 0.60478 3147747 , 222527, 222527, 5.4, 5.3, 5.6, 6.1,15.4, 18.2,6.2, 0.58659 3394999 , 221527, 221527, 5.4, 5.3, 5.6, 6.6,15.9, 19.4,7.3, 0.55884 3663559 , 226114, 226114, 5.3, 5.2, 5.6,15.0,84.4, 110.7,8.5, 0.52924 3911154 , 223831, 223831, 5.4, 5.3, 5.6, 6.1,15.6, 20.0,9.6, 0.50018 4152946 , 223246, 223246, 5.4, 5.3, 5.6, 6.1,15.7, 18.8, 10.7, 0.47323 4403162 , 228532, 228532, 5.2, 5.2, 5.6,23.2, 107.4, 121.4, 11.8, 0.44856 4641021 , 225196, 225196, 5.3, 5.2, 5.6, 5.9,15.3, 18.4, 12.8, 0.42557 4889523 , 222826, 222826, 5.4, 5.3, 5.6, 6.3,16.2, 22.0, 13.9, 0.40476 5124891 , 223203, 223203, 5.4, 5.3, 5.6, 5.8, 6.2, 14.8, 15.0, 0.38602 5375262 , 221222, 221222, 5.4, 5.2, 5.6,18.4,94.2, 115.1, 16.1, 0.36899 5616470 , 224022, 224022, 5.4, 5.3, 5.6, 5.9,14.3, 17.8, 17.2, 0.35349 5866825 , 223000, 223000, 5.4, 5.3, 5.6, 6.1,15.5, 18.2, 18.3, 0.33882 6125601 , 225757, 225757, 5.2, 5.3, 5.6, 9.6,49.4, 72.0, 19.5, 0.32535 6348030 , 192703, 192703, 6.3, 5.3, 9.3,14.4,77.1, 91.5, 20.6, 0.31282 6483574 , 128520, 128520, 9.3, 8.4,10.9,19.5,88.7, 99.0, 21.7, 0.30329 6626176 , 137199, 137199, 8.7, 8.4,10.6,14.0,32.7, 40.9, 22.7, 0.29771 6768401 , 136860, 136860, 8.8, 8.4,10.3,14.1,35.1, 40.8, 23.8, 0.29213 6911785 , 138204, 138204, 8.7, 8.3,10.2,13.7,34.1, 37.8, 24.8, 0.28669 7055951 , 138633, 138633, 8.7, 8.3,10.5,32.0,40.5, 46.9, 25.8, 0.28130 7199084 , 137731, 137731, 8.7, 8.4,10.2,14.0,33.4, 40.9, 26.9, 0.27623 7338032 , 133201, 133201, 9.0, 8.4,10.9,34.0,39.4, 43.8, 27.9, 0.27116 7480439 , 137059, 137059, 8.8, 8.4,10.2,13.9,35.9, 39.5, 29.0, 0.26663 7647810 , 161209, 161209, 7.5, 7.8, 9.6,13.4,33.9, 77.9, 30.0, 0.26185 7898882 , 226498, 226498, 5.3, 5.2, 5.6,19.7, 108.5, 119.3, 31.1, 0.25629 8136305 , 223840, 223840, 5.4, 5.3, 5.6, 5.9,17.3, 23.2, 32.2, 0.24838 8372076 , 223790, 223790, 5.4, 5.3, 5.6, 6.0,15.2, 20.0, 33.2, 0.24095 8633758 , 232914, 232914, 5.1, 5.2, 5.6,17.5, 138.4, 182.0, 34.4, 0.23397 8869214 , 43, 43, 5.4, 5.3, 5.6, 6.0,15.2, 17.9, 35.4, 0.22717 9121652 , 223037, 223037, 5.4, 5.3, 5.6, 5.9,15.4, 18.8, 36.5, 0.22105 9360286 , 225070, 225070, 5.3, 5.3, 5.6,14.8,82.7, 92.1, 37.6, 0.21524 9609676 , 224089, 224089, 5.4, 5.3, 5.6, 5.8, 6.2, 14.3, 38.7, 0.20967 9848551 , 222123, 222123, 5.4, 5.3, 5.6, 5.9,24.2, 27.2, 39.8, 0.20440 1000 , 229511, 229511, 5.0, 5.2, 5.8,60.0,74.3, 132.0, 40.5, 0.19935 Results: real op rate : 247211 adjusted op rate stderr : 0 key rate : 247211 latency mean : 5.4 latency median: 3.5 latency 95th percentile : 5.5 latency 99th percentile : 6.1 latency 99.9th percentile : 83.4 latency max : 3014.5 Total operation
[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069862#comment-14069862
]
Vijay edited comment on CASSANDRA-7438 at 7/22/14 5:49 AM:
---
Attached patch makes the off heap/Serialization Cache configurable. (the
default is still SerializationCache).
Regarding performance, the performance of the new cache is obviously better
when the JNI overhead is less than the GC overhead, but for the smaller sized
caches which can fit in memory the performance is little lower which is
understandable (but both of them out perform pagecache performance by a large
number). Here are the numbers.
*OffheapCacheProvider*
{panel}
Running READ with 1200 threads for 1000 iterations
ops ,op/s, key/s,mean, med, .95, .99,.999,
max, time, stderr
2030355 , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8,
3014.5,1.0, 0.0
2395480 , 202845, 202845, 5.8, 5.4, 5.8,20.2, 522.4,
545.9,2.8, 0.0
2638600 , 221368, 221368, 5.4, 5.3, 5.8,16.3,78.8,
131.5,3.9, 0.57860
2891705 , 221976, 221976, 5.4, 5.3, 5.6, 6.2,15.2,
19.2,5.0, 0.60478
3147747 , 222527, 222527, 5.4, 5.3, 5.6, 6.1,15.4,
18.2,6.2, 0.58659
3394999 , 221527, 221527, 5.4, 5.3, 5.6, 6.6,15.9,
19.4,7.3, 0.55884
3663559 , 226114, 226114, 5.3, 5.2, 5.6,15.0,84.4,
110.7,8.5, 0.52924
3911154 , 223831, 223831, 5.4, 5.3, 5.6, 6.1,15.6,
20.0,9.6, 0.50018
4152946 , 223246, 223246, 5.4, 5.3, 5.6, 6.1,15.7,
18.8, 10.7, 0.47323
4403162 , 228532, 228532, 5.2, 5.2, 5.6,23.2, 107.4,
121.4, 11.8, 0.44856
4641021 , 225196, 225196, 5.3, 5.2, 5.6, 5.9,15.3,
18.4, 12.8, 0.42557
4889523 , 222826, 222826, 5.4, 5.3, 5.6, 6.3,16.2,
22.0, 13.9, 0.40476
5124891 , 223203, 223203, 5.4, 5.3, 5.6, 5.8, 6.2,
14.8, 15.0, 0.38602
5375262 , 221222, 221222, 5.4, 5.2, 5.6,18.4,94.2,
115.1, 16.1, 0.36899
5616470 , 224022, 224022, 5.4, 5.3, 5.6, 5.9,14.3,
17.8, 17.2, 0.35349
5866825 , 223000, 223000, 5.4, 5.3, 5.6, 6.1,15.5,
18.2, 18.3, 0.33882
6125601 , 225757, 225757, 5.2, 5.3, 5.6, 9.6,49.4,
72.0, 19.5, 0.32535
6348030 , 192703, 192703, 6.3, 5.3, 9.3,14.4,77.1,
91.5, 20.6, 0.31282
6483574 , 128520, 128520, 9.3, 8.4,10.9,19.5,88.7,
99.0, 21.7, 0.30329
6626176 , 137199, 137199, 8.7, 8.4,10.6,14.0,32.7,
40.9, 22.7, 0.29771
6768401 , 136860, 136860, 8.8, 8.4,10.3,14.1,35.1,
40.8, 23.8, 0.29213
6911785 , 138204, 138204, 8.7, 8.3,10.2,13.7,34.1,
37.8, 24.8, 0.28669
7055951 , 138633, 138633, 8.7, 8.3,10.5,32.0,40.5,
46.9, 25.8, 0.28130
7199084 , 137731, 137731, 8.7, 8.4,10.2,14.0,33.4,
40.9, 26.9, 0.27623
7338032 , 133201, 133201, 9.0, 8.4,10.9,34.0,39.4,
43.8, 27.9, 0.27116
7480439 , 137059, 137059, 8.8, 8.4,10.2,13.9,35.9,
39.5, 29.0, 0.26663
7647810 , 161209, 161209, 7.5, 7.8, 9.6,13.4,33.9,
77.9, 30.0, 0.26185
7898882 , 226498, 226498, 5.3, 5.2, 5.6,19.7, 108.5,
119.3, 31.1, 0.25629
8136305 , 223840, 223840, 5.4, 5.3, 5.6, 5.9,17.3,
23.2, 32.2, 0.24838
8372076 , 223790, 223790, 5.4, 5.3, 5.6, 6.0,15.2,
20.0, 33.2, 0.24095
8633758 , 232914, 232914, 5.1, 5.2, 5.6,17.5, 138.4,
182.0, 34.4, 0.23397
8869214 , 43, 43, 5.4, 5.3, 5.6, 6.0,15.2,
17.9, 35.4, 0.22717
9121652 , 223037, 223037, 5.4, 5.3, 5.6, 5.9,15.4,
18.8, 36.5, 0.22105
9360286 , 225070, 225070, 5.3, 5.3, 5.6,14.8,82.7,
92.1, 37.6, 0.21524
9609676 , 224089, 224089, 5.4, 5.3, 5.6, 5.8, 6.2,
14.3, 38.7, 0.20967
9848551 , 222123, 222123, 5.4, 5.3, 5.6, 5.9,24.2,
27.2, 39.8, 0.20440
1000 , 229511, 229511, 5.0, 5.2, 5.8,60.0,74.3,
132.0, 40.5, 0.19935
Results:
real op rate : 247211
adjusted op rate stderr : 0
key rate : 247211
latency mean : 5.4
latency median: 3.5
latency 95th percentile : 5.5
latency 99th percentile : 6.1
latency 99.9th percentile : 83.4
latency max
[jira] [Updated] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[ https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay updated CASSANDRA-7438: - Attachment: 0001-CASSANDRA-7438.patch > Serializing Row cache alternative (Fully off heap) > -- > > Key: CASSANDRA-7438 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7438 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Linux >Reporter: Vijay >Assignee: Vijay > Labels: performance > Fix For: 3.0 > > Attachments: 0001-CASSANDRA-7438.patch > > > Currently SerializingCache is partially off heap, keys are still stored in > JVM heap as BB, > * There is a higher GC costs for a reasonably big cache. > * Some users have used the row cache efficiently in production for better > results, but this requires careful tunning. > * Overhead in Memory for the cache entries are relatively high. > So the proposal for this ticket is to move the LRU cache logic completely off > heap and use JNI to interact with cache. We might want to ensure that the new > implementation match the existing API's (ICache), and the implementation > needs to have safe memory access, low overhead in memory and less memcpy's > (As much as possible). > We might also want to make this cache configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069862#comment-14069862
]
Vijay edited comment on CASSANDRA-7438 at 7/22/14 5:51 AM:
---
Attached patch makes the off heap/Serialization Cache configurable. (the
default is still SerializationCache).
Regarding performance, the performance of the new cache is obviously better
when the JNI overhead is less than the GC overhead, but for the smaller sized
caches which can fit in memory the performance is little lower which is
understandable (but both of them out perform pagecache performance by a large
number). Here are the numbers.
*OffheapCacheProvider*
{panel}
Running READ with 1200 threads for 1000 iterations
ops ,op/s, key/s,mean, med, .95, .99,.999,
max, time, stderr
2030355 , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8,
3014.5,1.0, 0.0
2395480 , 202845, 202845, 5.8, 5.4, 5.8,20.2, 522.4,
545.9,2.8, 0.0
2638600 , 221368, 221368, 5.4, 5.3, 5.8,16.3,78.8,
131.5,3.9, 0.57860
2891705 , 221976, 221976, 5.4, 5.3, 5.6, 6.2,15.2,
19.2,5.0, 0.60478
3147747 , 222527, 222527, 5.4, 5.3, 5.6, 6.1,15.4,
18.2,6.2, 0.58659
3394999 , 221527, 221527, 5.4, 5.3, 5.6, 6.6,15.9,
19.4,7.3, 0.55884
3663559 , 226114, 226114, 5.3, 5.2, 5.6,15.0,84.4,
110.7,8.5, 0.52924
3911154 , 223831, 223831, 5.4, 5.3, 5.6, 6.1,15.6,
20.0,9.6, 0.50018
4152946 , 223246, 223246, 5.4, 5.3, 5.6, 6.1,15.7,
18.8, 10.7, 0.47323
4403162 , 228532, 228532, 5.2, 5.2, 5.6,23.2, 107.4,
121.4, 11.8, 0.44856
4641021 , 225196, 225196, 5.3, 5.2, 5.6, 5.9,15.3,
18.4, 12.8, 0.42557
4889523 , 222826, 222826, 5.4, 5.3, 5.6, 6.3,16.2,
22.0, 13.9, 0.40476
5124891 , 223203, 223203, 5.4, 5.3, 5.6, 5.8, 6.2,
14.8, 15.0, 0.38602
5375262 , 221222, 221222, 5.4, 5.2, 5.6,18.4,94.2,
115.1, 16.1, 0.36899
5616470 , 224022, 224022, 5.4, 5.3, 5.6, 5.9,14.3,
17.8, 17.2, 0.35349
5866825 , 223000, 223000, 5.4, 5.3, 5.6, 6.1,15.5,
18.2, 18.3, 0.33882
6125601 , 225757, 225757, 5.2, 5.3, 5.6, 9.6,49.4,
72.0, 19.5, 0.32535
6348030 , 192703, 192703, 6.3, 5.3, 9.3,14.4,77.1,
91.5, 20.6, 0.31282
6483574 , 128520, 128520, 9.3, 8.4,10.9,19.5,88.7,
99.0, 21.7, 0.30329
6626176 , 137199, 137199, 8.7, 8.4,10.6,14.0,32.7,
40.9, 22.7, 0.29771
6768401 , 136860, 136860, 8.8, 8.4,10.3,14.1,35.1,
40.8, 23.8, 0.29213
6911785 , 138204, 138204, 8.7, 8.3,10.2,13.7,34.1,
37.8, 24.8, 0.28669
7055951 , 138633, 138633, 8.7, 8.3,10.5,32.0,40.5,
46.9, 25.8, 0.28130
7199084 , 137731, 137731, 8.7, 8.4,10.2,14.0,33.4,
40.9, 26.9, 0.27623
7338032 , 133201, 133201, 9.0, 8.4,10.9,34.0,39.4,
43.8, 27.9, 0.27116
7480439 , 137059, 137059, 8.8, 8.4,10.2,13.9,35.9,
39.5, 29.0, 0.26663
7647810 , 161209, 161209, 7.5, 7.8, 9.6,13.4,33.9,
77.9, 30.0, 0.26185
7898882 , 226498, 226498, 5.3, 5.2, 5.6,19.7, 108.5,
119.3, 31.1, 0.25629
8136305 , 223840, 223840, 5.4, 5.3, 5.6, 5.9,17.3,
23.2, 32.2, 0.24838
8372076 , 223790, 223790, 5.4, 5.3, 5.6, 6.0,15.2,
20.0, 33.2, 0.24095
8633758 , 232914, 232914, 5.1, 5.2, 5.6,17.5, 138.4,
182.0, 34.4, 0.23397
8869214 , 43, 43, 5.4, 5.3, 5.6, 6.0,15.2,
17.9, 35.4, 0.22717
9121652 , 223037, 223037, 5.4, 5.3, 5.6, 5.9,15.4,
18.8, 36.5, 0.22105
9360286 , 225070, 225070, 5.3, 5.3, 5.6,14.8,82.7,
92.1, 37.6, 0.21524
9609676 , 224089, 224089, 5.4, 5.3, 5.6, 5.8, 6.2,
14.3, 38.7, 0.20967
9848551 , 222123, 222123, 5.4, 5.3, 5.6, 5.9,24.2,
27.2, 39.8, 0.20440
1000 , 229511, 229511, 5.0, 5.2, 5.8,60.0,74.3,
132.0, 40.5, 0.19935
Results:
real op rate : 247211
adjusted op rate stderr : 0
key rate : 247211
latency mean : 5.4
latency median: 3.5
latency 95th percentile : 5.5
latency 99th percentile : 6.1
latency 99.9th percentile : 83.4
latency max
[jira] [Comment Edited] (CASSANDRA-7438) Serializing Row cache alternative (Fully off heap)
[
https://issues.apache.org/jira/browse/CASSANDRA-7438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069862#comment-14069862
]
Vijay edited comment on CASSANDRA-7438 at 7/22/14 5:54 AM:
---
Attached patch makes the off heap/Serialization Cache configurable. (the
default is still SerializationCache).
Regarding performance, the performance of the new cache is obviously better
when the JNI overhead is less than the GC overhead. For smaller size cache that
can fit in the JVM heap, the performance is a little lower understandablely
(but both of them out perform pagecache performance by a large number). Here
are the numbers.
*OffheapCacheProvider*
{panel}
Running READ with 1200 threads for 1000 iterations
ops ,op/s, key/s,mean, med, .95, .99,.999,
max, time, stderr
2030355 , 2029531, 2029531, 3.1, 3.1, 5.4, 5.7,61.8,
3014.5,1.0, 0.0
2395480 , 202845, 202845, 5.8, 5.4, 5.8,20.2, 522.4,
545.9,2.8, 0.0
2638600 , 221368, 221368, 5.4, 5.3, 5.8,16.3,78.8,
131.5,3.9, 0.57860
2891705 , 221976, 221976, 5.4, 5.3, 5.6, 6.2,15.2,
19.2,5.0, 0.60478
3147747 , 222527, 222527, 5.4, 5.3, 5.6, 6.1,15.4,
18.2,6.2, 0.58659
3394999 , 221527, 221527, 5.4, 5.3, 5.6, 6.6,15.9,
19.4,7.3, 0.55884
3663559 , 226114, 226114, 5.3, 5.2, 5.6,15.0,84.4,
110.7,8.5, 0.52924
3911154 , 223831, 223831, 5.4, 5.3, 5.6, 6.1,15.6,
20.0,9.6, 0.50018
4152946 , 223246, 223246, 5.4, 5.3, 5.6, 6.1,15.7,
18.8, 10.7, 0.47323
4403162 , 228532, 228532, 5.2, 5.2, 5.6,23.2, 107.4,
121.4, 11.8, 0.44856
4641021 , 225196, 225196, 5.3, 5.2, 5.6, 5.9,15.3,
18.4, 12.8, 0.42557
4889523 , 222826, 222826, 5.4, 5.3, 5.6, 6.3,16.2,
22.0, 13.9, 0.40476
5124891 , 223203, 223203, 5.4, 5.3, 5.6, 5.8, 6.2,
14.8, 15.0, 0.38602
5375262 , 221222, 221222, 5.4, 5.2, 5.6,18.4,94.2,
115.1, 16.1, 0.36899
5616470 , 224022, 224022, 5.4, 5.3, 5.6, 5.9,14.3,
17.8, 17.2, 0.35349
5866825 , 223000, 223000, 5.4, 5.3, 5.6, 6.1,15.5,
18.2, 18.3, 0.33882
6125601 , 225757, 225757, 5.2, 5.3, 5.6, 9.6,49.4,
72.0, 19.5, 0.32535
6348030 , 192703, 192703, 6.3, 5.3, 9.3,14.4,77.1,
91.5, 20.6, 0.31282
6483574 , 128520, 128520, 9.3, 8.4,10.9,19.5,88.7,
99.0, 21.7, 0.30329
6626176 , 137199, 137199, 8.7, 8.4,10.6,14.0,32.7,
40.9, 22.7, 0.29771
6768401 , 136860, 136860, 8.8, 8.4,10.3,14.1,35.1,
40.8, 23.8, 0.29213
6911785 , 138204, 138204, 8.7, 8.3,10.2,13.7,34.1,
37.8, 24.8, 0.28669
7055951 , 138633, 138633, 8.7, 8.3,10.5,32.0,40.5,
46.9, 25.8, 0.28130
7199084 , 137731, 137731, 8.7, 8.4,10.2,14.0,33.4,
40.9, 26.9, 0.27623
7338032 , 133201, 133201, 9.0, 8.4,10.9,34.0,39.4,
43.8, 27.9, 0.27116
7480439 , 137059, 137059, 8.8, 8.4,10.2,13.9,35.9,
39.5, 29.0, 0.26663
7647810 , 161209, 161209, 7.5, 7.8, 9.6,13.4,33.9,
77.9, 30.0, 0.26185
7898882 , 226498, 226498, 5.3, 5.2, 5.6,19.7, 108.5,
119.3, 31.1, 0.25629
8136305 , 223840, 223840, 5.4, 5.3, 5.6, 5.9,17.3,
23.2, 32.2, 0.24838
8372076 , 223790, 223790, 5.4, 5.3, 5.6, 6.0,15.2,
20.0, 33.2, 0.24095
8633758 , 232914, 232914, 5.1, 5.2, 5.6,17.5, 138.4,
182.0, 34.4, 0.23397
8869214 , 43, 43, 5.4, 5.3, 5.6, 6.0,15.2,
17.9, 35.4, 0.22717
9121652 , 223037, 223037, 5.4, 5.3, 5.6, 5.9,15.4,
18.8, 36.5, 0.22105
9360286 , 225070, 225070, 5.3, 5.3, 5.6,14.8,82.7,
92.1, 37.6, 0.21524
9609676 , 224089, 224089, 5.4, 5.3, 5.6, 5.8, 6.2,
14.3, 38.7, 0.20967
9848551 , 222123, 222123, 5.4, 5.3, 5.6, 5.9,24.2,
27.2, 39.8, 0.20440
1000 , 229511, 229511, 5.0, 5.2, 5.8,60.0,74.3,
132.0, 40.5, 0.19935
Results:
real op rate : 247211
adjusted op rate stderr : 0
key rate : 247211
latency mean : 5.4
latency median: 3.5
latency 95th percentile : 5.5
latency 99th percentile : 6.1
latency 99.9th percentile : 83.4
latency max
[jira] [Commented] (CASSANDRA-7546) AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory
[ https://issues.apache.org/jira/browse/CASSANDRA-7546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069896#comment-14069896 ] Benedict commented on CASSANDRA-7546: - bq. Alternatively if you are saying, let each thread keep working while they still believe they can win, This was my original rationale for the patch I posted, however now I am much more in favour of bq. a one way switch per Atomic*Columns instance that flips after a number waster "operations"? However whether it is one-way or not is somewhat unimportant for me. This flip would only last the lifetime of a memtable, which is not super lengthy (under heavily load probably only a few minutes), and would not have dramatically negative consequences if it got it slightly wrong However^2 I'm still having a hard time believing rebalancing costs in snap tree can be that high, and further if that really is the problem it should not be an issue in 2.1, as the b-tree rebalances with O(lg(N)) allocations. I'd be a little surprised if the snap tree didn't do the same, as if there were more than O(lg(N)) allocations, the algorithmic complexity would be > O(lg(N)) also. It's possible somehow that it manages to inter-refererence with on-going copies, so that we get a highly complex graph that retains exponentially more garbage the more competing updates there are, but again I would be very surprised if this were the case. However outside of either of these I would expect the garbage generated to all be immediately collectible, so it would have to be the sheer volume alone that overwhelmed the GC, which is certainly possible but this would entail a _lot_ of hinting, and I'd be surprised if a node could be receiving a large enough quantity. On the other hand the arena allocations in 2.0 are definitely incapable of being collected and could be allocated almost as rapidly. bq. I'm not sure which changes you are talking about back-porting and whether the "at most twice" refers to looping once then locking In this instance I'm referring to copying the source ColumnFamily locally in the variable once after failing the cas, so that we do not keep allocating arena space. Alternatively, we could just do it upfront in the method, as the only extra cost is an array allocation proportional in size to the input data, which is fairly cheap. All of this said, I think the behaviour of locking after wasting an excessive number of cycles is still a good one, so I'm comfortable introducing it either way, and it would certainly help with all of the above causes. > AtomicSortedColumns.addAllWithSizeDelta has a spin loop that allocates memory > - > > Key: CASSANDRA-7546 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7546 > Project: Cassandra > Issue Type: Bug > Components: Core >Reporter: graham sanderson >Assignee: graham sanderson > Attachments: 7546.20.txt, 7546.20_2.txt, 7546.20_alt.txt, > suggestion1.txt, suggestion1_21.txt > > > In order to preserve atomicity, this code attempts to read, clone/update, > then CAS the state of the partition. > Under heavy contention for updating a single partition this can cause some > fairly staggering memory growth (the more cores on your machine the worst it > gets). > Whilst many usage patterns don't do highly concurrent updates to the same > partition, hinting today, does, and in this case wild (order(s) of magnitude > more than expected) memory allocation rates can be seen (especially when the > updates being hinted are small updates to different partitions which can > happen very fast on their own) - see CASSANDRA-7545 > It would be best to eliminate/reduce/limit the spinning memory allocation > whilst not slowing down the very common un-contended case. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (CASSANDRA-7507) OOM creates unreliable state - die instantly better
[
https://issues.apache.org/jira/browse/CASSANDRA-7507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069906#comment-14069906
]
Jonathan Ellis commented on CASSANDRA-7507:
---
I'm not pushing for a Grand Exception Rewrite here. For 2.1.1 we should just
make the uncaught exception handler kill the server uncleanly on OOM. If there
are places that may potentially swallow OOM (this includes {{catch
(Throwable)}} but not {{catch (Exception)}}) then let's make sure those rethrow
after whatever custom logic they're doing.
> OOM creates unreliable state - die instantly better
> ---
>
> Key: CASSANDRA-7507
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7507
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Karl Mueller
>Assignee: Joshua McKenzie
>Priority: Minor
> Fix For: 2.1.1
>
> Attachments: 7507_v1.txt, 7507_v2.txt, exceptionHandlingResults.txt,
> findSwallowedExceptions.py
>
>
> I had a cassandra node run OOM. My heap had enough headroom, there was just
> something which either was a bug or some unfortunate amount of short-term
> memory utilization. This resulted in the following error:
> WARN [StorageServiceShutdownHook] 2014-06-30 09:38:38,251 StorageProxy.java
> (line 1713) Some hints were not written before shutdown. This is not
> supposed to happen. You should (a) run repair, and (b) file a bug report
> There are no other messages of relevance besides the OOM error about 90
> minutes earlier.
> My (limited) understanding of the JVM and Cassandra says that when it goes
> OOM, it will attempt to signal cassandra to shut down "cleanly". The problem,
> in my view, is that with an OOM situation, nothing is guaranteed anymore. I
> believe it's impossible to reliably "cleanly shut down" at this point, and
> therefore it's wrong to even try.
> Yes, ideally things could be written out, flushed to disk, memory messages
> written, other nodes notified, etc. but why is there any reason to believe
> any of those steps could happen? Would happen? Couldn't bad data be written
> at this point to disk rather than good data? Some network messages delivered,
> but not others?
> I think Cassandra should have the option to (and possibly default) to kill
> itself immediately upon the OOM condition happening in a hard way, and not
> rely on the java-based clean shutdown process. Cassandra already handles
> recovery from unclean shutdown, and it's not a big deal. My node, for
> example, kept in a sort-of alive state for 90 minutes where who knows what it
> was doing or not doing.
> I don't know enough about the JVM and options for it to know the best exact
> implementation of "die instantly on OOM", but it should be something that's
> possible either with some flags or a C library (which doesn't rely on java
> memory to do something which it may not be able to get!)
> Short version: a kill -9 of all C* processes in that instance without needing
> more java memory, when OOM is raised
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (CASSANDRA-6454) Pig support for hadoop CqlInputFormat
[ https://issues.apache.org/jira/browse/CASSANDRA-6454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14069908#comment-14069908 ] Jonathan Ellis commented on CASSANDRA-6454: --- I'm -0 on adding extra yaml files, fwiw. > Pig support for hadoop CqlInputFormat > - > > Key: CASSANDRA-6454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6454 > Project: Cassandra > Issue Type: Bug > Components: Hadoop >Reporter: Alex Liu >Assignee: Alex Liu > Fix For: 2.0.10 > > Attachments: 6454-2.0-branch.txt, 6454-v2-2.0-branch.txt, > 6454-v3-2.0-branch.txt, 6454-v3-2.1-branch.txt > > > CASSANDRA-6311 adds new CqlInputFormat, we need add the Pig support for it -- This message was sent by Atlassian JIRA (v6.2#6252)
