[jira] [Commented] (CASSANDRA-13285) Lower bound [INCL_START_BOUND(event_source) ]is bigger than first returned value
[
https://issues.apache.org/jira/browse/CASSANDRA-13285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891745#comment-15891745
]
JianwenSun commented on CASSANDRA-13285:
I lookup 3.9 source code and do the remote debug.
assert comparator().compare(lowerBound, ret.clustering()) <= 0
: String.format("Lower bound [%s ]is bigger than first returned
value [%s] for sstable %s",
lowerBound.toString(sstable.metadata),
ret.toString(sstable.metadata),
sstable.getFilename());
where lowerBound = "event_source" and ret.clustering() = "action" .
so why does the lowerBound = "event_source"?
'lowerBound' was made by getMetadataLowerBound function in which use
minClusteringValues and maxClusteringValues of the sstablemetadata to
assignment the value.
"minClusteringValues" is an array with only one item "source_node"
"maxClusteringValues" is an array with only one item "event_source"
notice: columnfamily(events) only contains 47 rows. some row contains
'source_node' column while some not.
so i didn't understand why does the 'maxClusteringValues' = "event_source" and
'minClusteringValues' = 'source_node'.
plz help.
> Lower bound [INCL_START_BOUND(event_source) ]is bigger than first returned
> value
>
>
> Key: CASSANDRA-13285
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13285
> Project: Cassandra
> Issue Type: Bug
> Components: Core
>Reporter: JianwenSun
> Attachments: schema.txt, using cql.png, using thrift.png
>
>
> We do the test of upgrade 1.2.4->3.9 , everything seems ok. but when we query
> some data (using thrift) server throw this exception:
> .. 1:07:40 PM
> java.lang.AssertionError: Lower bound [INCL_START_BOUND(event_source) ]is
> bigger than first returned value [Row: column1=action | value=] for sstable
> /opt/cassandra/upgrade/data/OpsCenter/events/mc-10-big-Data.db
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:122)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:46)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:500)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:360)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:70)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:666)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:615)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:492)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:397)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1801)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2486)
> ~[apache-cassandra-3.9.jar:3.9]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_121]
> at
> org.apache.cassandra.concurrent.AbstractLocalA
[jira] [Created] (CASSANDRA-13285) Lower bound [INCL_START_BOUND(event_source) ]is bigger than first returned value
JianwenSun created CASSANDRA-13285: -- Summary: Lower bound [INCL_START_BOUND(event_source) ]is bigger than first returned value Key: CASSANDRA-13285 URL: https://issues.apache.org/jira/browse/CASSANDRA-13285 Project: Cassandra Issue Type: Bug Components: Core Reporter: JianwenSun Attachments: schema.txt, using cql.png, using thrift.png We do the test of upgrade 1.2.4->3.9 , everything seems ok. but when we query some data (using thrift) server throw this exception: .. 1:07:40 PM java.lang.AssertionError: Lower bound [INCL_START_BOUND(event_source) ]is bigger than first returned value [Row: column1=action | value=] for sstable /opt/cassandra/upgrade/data/OpsCenter/events/mc-10-big-Data.db at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:122) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:46) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:374) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:186) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:155) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:500) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:360) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:70) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.SinglePartitionReadCommand.withSSTablesIterated(SinglePartitionReadCommand.java:666) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:615) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:492) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:397) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1801) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2486) ~[apache-cassandra-3.9.jar:3.9] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_121] at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) ~[apache-cassandra-3.9.jar:3.9] at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [apache-cassandra-3.9.jar:3.9] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_121] There was no problems when I use cassandra-cli under 1.2.4 cluster and cqlsh under 3.9 clusters to do the query, it occurred only if when I use thrift api to search it. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[ https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891641#comment-15891641 ] Ariel Weisberg commented on CASSANDRA-13265: I didn't notice that it already bails out and doesn't iterate the entire list once it encounters the first message that isn't timed out. In that case it's fine to attempt to iterate the queue frequently because the cost to check is low since it will fail on the first element. I am actually surprised this a performance issue. I guess that there is some pathological behavior once you start trying to have multiple threads iterate and remove at the same time and they actually end up needing to remove elements. I'll kick off utests and dtests for this tomorrow. > Epxiration in OutboundTcpConnection can block the reader Thread > --- > > Key: CASSANDRA-13265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13265 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 3.0.9 > Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version > 1.8.0_112-b15) > Linux 3.16 >Reporter: Christian Esken >Assignee: Christian Esken > Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz, > cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz > > > I observed that sometimes a single node in a Cassandra cluster fails to > communicate to the other nodes. This can happen at any time, during peak load > or low load. Restarting that single node from the cluster fixes the issue. > Before going in to details, I want to state that I have analyzed the > situation and am already developing a possible fix. Here is the analysis so > far: > - A Threaddump in this situation showed 324 Threads in the > OutboundTcpConnection class that want to lock the backlog queue for doing > expiration. > - A class histogram shows 262508 instances of > OutboundTcpConnection$QueuedMessage. > What is the effect of it? As soon as the Cassandra node has reached a certain > amount of queued messages, it starts thrashing itself to death. Each of the > Thread fully locks the Queue for reading and writing by calling > iterator.next(), making the situation worse and worse. > - Writing: Only after 262508 locking operation it can progress with actually > writing to the Queue. > - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and > fully lock the Queue > This means: Writing blocks the Queue for reading, and readers might even be > starved which makes the situation even worse. > - > The setup is: > - 3-node cluster > - replication factor 2 > - Consistency LOCAL_ONE > - No remote DC's > - high write throughput (10 INSERT statements per second and more during > peak times). > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-12661) Make gc_log and gc_warn settable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891624#comment-15891624 ] Jeff Jirsa commented on CASSANDRA-12661: github indicates it still applies cleanly - doesn't look like it's been committed to me. > Make gc_log and gc_warn settable at runtime > --- > > Key: CASSANDRA-12661 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12661 > Project: Cassandra > Issue Type: New Feature >Reporter: Edward Capriolo >Assignee: Edward Capriolo >Priority: Minor > > Changes: > * Move gc_log_threshold_in_ms and gc_warn_threshold_in_ms close together in > the config > * rename variables to match properties > * add unit tests to ensure hybration > * add unit tests to ensure variables are set propertly > * minor perf (do not consturct string from buffer f not logging) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (CASSANDRA-13284) break CQL specification & reference parser into separate repo
Jon Haddad created CASSANDRA-13284: -- Summary: break CQL specification & reference parser into separate repo Key: CASSANDRA-13284 URL: https://issues.apache.org/jira/browse/CASSANDRA-13284 Project: Cassandra Issue Type: Improvement Reporter: Jon Haddad It would be pretty awesome to be able to use the CQL parser in other projects, such as a driver, developer studio, etc. I propose we break out the CQL spec and parser into a sub-project. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-12661) Make gc_log and gc_warn settable at runtime
[
https://issues.apache.org/jira/browse/CASSANDRA-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891568#comment-15891568
]
Jon Haddad commented on CASSANDRA-12661:
Was this merged by someone else? I tried to take a look at this but git
errored out saying it's already been applied:
{code}
(cassandra) jhaddad@rustyrazorblade ~/dev/cassandra$ git am -3 12661.patch
Applying: CASSANDRA-12661 gc_log and gc_warn runtime setabble and minor cleanup
Using index info to reconstruct a base tree...
M conf/cassandra.yaml
M src/java/org/apache/cassandra/config/Config.java
M src/java/org/apache/cassandra/service/GCInspector.java
M src/java/org/apache/cassandra/service/GCInspectorMXBean.java
.git/rebase-apply/patch:96: trailing whitespace.
.git/rebase-apply/patch:111: trailing whitespace.
catch (RuntimeException | InstanceAlreadyExistsException |
MBeanRegistrationException |
.git/rebase-apply/patch:126: trailing whitespace.
.git/rebase-apply/patch:151: trailing whitespace.
throw new IllegalArgumentException("Threashold must be greater than
gcLogTreasholdInMs which is currently "
.git/rebase-apply/patch:168: trailing whitespace.
throw new IllegalArgumentException("Threashold must be less than
gcWarnTreasholdInMs which is currently "
warning: squelched 11 whitespace errors
warning: 16 lines add whitespace errors.
Falling back to patching base and 3-way merge...
No changes -- Patch already applied.
{code}
> Make gc_log and gc_warn settable at runtime
> ---
>
> Key: CASSANDRA-12661
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12661
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Edward Capriolo
>Assignee: Edward Capriolo
>Priority: Minor
>
> Changes:
> * Move gc_log_threshold_in_ms and gc_warn_threshold_in_ms close together in
> the config
> * rename variables to match properties
> * add unit tests to ensure hybration
> * add unit tests to ensure variables are set propertly
> * minor perf (do not consturct string from buffer f not logging)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13253) include a search on the doc home
[ https://issues.apache.org/jira/browse/CASSANDRA-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jon Haddad updated CASSANDRA-13253: --- Attachment: screenshot-1.png > include a search on the doc home > > > Key: CASSANDRA-13253 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13253 > Project: Cassandra > Issue Type: Improvement > Components: Documentation and Website >Reporter: Jon Haddad >Assignee: Jon Haddad > Attachments: screenshot-1.png > > > Current doc homepage doesn't have a search. > http://cassandra.apache.org/doc/latest/ -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13253) include a search on the doc home
[
https://issues.apache.org/jira/browse/CASSANDRA-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891551#comment-15891551
]
Jon Haddad commented on CASSANDRA-13253:
It was broken for me for a while, and now it's working again. I don't know why.
{code}
pip install -U sphinx sphinx_rtd_theme
ant gen-doc
cd doc/build/html
python -m SimpleHTTPServer 8000
{code}
See attached screenshot of the search results.
> include a search on the doc home
>
>
> Key: CASSANDRA-13253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13253
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Attachments: screenshot-1.png
>
>
> Current doc homepage doesn't have a search.
> http://cassandra.apache.org/doc/latest/
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13253) include a search on the doc home
[
https://issues.apache.org/jira/browse/CASSANDRA-13253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891551#comment-15891551
]
Jon Haddad edited comment on CASSANDRA-13253 at 3/2/17 3:19 AM:
It was broken for me for a while, and now it's working again. I don't know why.
{code}
pip install -U sphinx sphinx_rtd_theme
ant gen-doc
cd doc/build/html
python -m SimpleHTTPServer 8000
{code}
See attached screenshot of the search results.
The patch still applies cleanly on trunk.
was (Author: rustyrazorblade):
It was broken for me for a while, and now it's working again. I don't know why.
{code}
pip install -U sphinx sphinx_rtd_theme
ant gen-doc
cd doc/build/html
python -m SimpleHTTPServer 8000
{code}
See attached screenshot of the search results.
> include a search on the doc home
>
>
> Key: CASSANDRA-13253
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13253
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation and Website
>Reporter: Jon Haddad
>Assignee: Jon Haddad
> Attachments: screenshot-1.png
>
>
> Current doc homepage doesn't have a search.
> http://cassandra.apache.org/doc/latest/
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-8494) incremental bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-8494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891516#comment-15891516 ] Jon Haddad edited comment on CASSANDRA-8494 at 3/2/17 2:42 AM: --- It looks like the original thing won't be done because of the insanity due to shuffle. Was anything actually done related this this ticket? I guess it can be closed. Any objections? was (Author: rustyrazorblade): It looks like the original thing won't be done because of the insanity due to shuffle - so I'm renaming this & marking as done. > incremental bootstrap > - > > Key: CASSANDRA-8494 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8494 > Project: Cassandra > Issue Type: New Feature >Reporter: Jon Haddad >Assignee: Yuki Morishita >Priority: Minor > Labels: dense-storage > Fix For: 3.11.x > > > Current bootstrapping involves (to my knowledge) picking tokens and streaming > data before the node is available for requests. This can be problematic with > "fat nodes", since it may require 20TB of data to be streamed over before the > machine can be useful. This can result in a massive window of time before > the machine can do anything useful. > As a potential approach to mitigate the huge window of time before a node is > available, I suggest modifying the bootstrap process to only acquire a single > initial token before being marked UP. This would likely be a configuration > parameter "incremental_bootstrap" or something similar. > After the node is bootstrapped with this one token, it could go into UP > state, and could then acquire additional tokens (one or a handful at a time), > which would be streamed over while the node is active and serving requests. > The benefit here is that with the default 256 tokens a node could become an > active part of the cluster with less than 1% of it's final data streamed over. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-8494) incremental bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-8494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891516#comment-15891516 ] Jon Haddad commented on CASSANDRA-8494: --- It looks like the original thing won't be done because of the insanity due to shuffle - so I'm renaming this & marking as done. > incremental bootstrap > - > > Key: CASSANDRA-8494 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8494 > Project: Cassandra > Issue Type: New Feature >Reporter: Jon Haddad >Assignee: Yuki Morishita >Priority: Minor > Labels: dense-storage > Fix For: 3.11.x > > > Current bootstrapping involves (to my knowledge) picking tokens and streaming > data before the node is available for requests. This can be problematic with > "fat nodes", since it may require 20TB of data to be streamed over before the > machine can be useful. This can result in a massive window of time before > the machine can do anything useful. > As a potential approach to mitigate the huge window of time before a node is > available, I suggest modifying the bootstrap process to only acquire a single > initial token before being marked UP. This would likely be a configuration > parameter "incremental_bootstrap" or something similar. > After the node is bootstrapped with this one token, it could go into UP > state, and could then acquire additional tokens (one or a handful at a time), > which would be streamed over while the node is active and serving requests. > The benefit here is that with the default 256 tokens a node could become an > active part of the cluster with less than 1% of it's final data streamed over. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13071) cqlsh copy-from should error out when csv contains invalid data for collections
[
https://issues.apache.org/jira/browse/CASSANDRA-13071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania updated CASSANDRA-13071:
-
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 3.0.x)
3.11.0
3.0.12
Status: Resolved (was: Ready to Commit)
Thanks! Committed to 3.0 as 496cfa8f5b160d26794b346ce360b513069aa300 and merged
upwards.
> cqlsh copy-from should error out when csv contains invalid data for
> collections
> ---
>
> Key: CASSANDRA-13071
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13071
> Project: Cassandra
> Issue Type: Bug
> Components: Tools
>Reporter: Stefania
>Assignee: Stefania
>Priority: Minor
> Fix For: 3.0.12, 3.11.0
>
>
> If the csv file contains invalid data for collection types, at the moment the
> data is imported incorrectly, an error would be a better behavior.
> For example this table:
> {code}
> CREATE TABLE test.test (key text, value frozen>, PRIMARY KEY
> (key));
> {code}
> with this data:
> {code}
> "key1","{'test1', 'test2'}"
> "Key2","not_a_set"
> {code}
> will be imported by {{COPY test.test FROM 'test.csv';}} without errors but
> will result in the following data:
> {code}
> cqlsh> select * from test.test;
> key | value
> --+
> key1 | {'test1', 'test2'}
> Key2 |{'ot_a_se'}
> (2 rows)
> {code}
> The second row should have been rejected. The reason is that the [{{split}}
> function|https://github.com/stef1927/cassandra/blob/trunk/pylib/cqlshlib/copyutil.py#L1898]
> assumes that the first and last characters of the string passed in are
> parentheses, without actually checking it.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/943fb02f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/943fb02f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/943fb02f Branch: refs/heads/cassandra-3.11 Commit: 943fb02ffda7b7c65ec722d18a5c184fa480f525 Parents: 093b2e2 496cfa8 Author: Stefania Alborghetti Authored: Thu Mar 2 10:00:20 2017 +0800 Committer: Stefania Alborghetti Committed: Thu Mar 2 10:00:20 2017 +0800 -- CHANGES.txt| 1 + pylib/cqlshlib/copyutil.py | 29 - 2 files changed, 25 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/943fb02f/CHANGES.txt -- diff --cc CHANGES.txt index cf15d3a,1c3869f..eeb2215 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,15 -1,6 +1,16 @@@ -3.0.12 +3.11.0 + * Fix equality comparisons of columns using the duration type (CASSANDRA-13174) + * Obfuscate password in stress-graphs (CASSANDRA-12233) + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034) + * nodetool stopdaemon errors out (CASSANDRA-13030) + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954) + * Fix primary index calculation for SASI (CASSANDRA-12910) + * More fixes to the TokenAllocator (CASSANDRA-12990) + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983) +Merged from 3.0: + * Cqlsh copy-from should error out when csv contains invalid data for collections (CASSANDRA-13071) - * Update c.yaml doc for offheap memtables (CASSANDRA-13179) + * Fix "multiple versions of ant detected..." when running ant test (CASSANDRA-13232) + * Coalescing strategy sleeps too much (CASSANDRA-13090) * Faster StreamingHistogram (CASSANDRA-13038) * Legacy deserializer can create unexpected boundary range tombstones (CASSANDRA-13237) * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070) http://git-wip-us.apache.org/repos/asf/cassandra/blob/943fb02f/pylib/cqlshlib/copyutil.py --
[2/6] cassandra git commit: Cqlsh copy-from should error out when csv contains invalid data for collections
Cqlsh copy-from should error out when csv contains invalid data for collections
patch by Stefania Alborghetti; reviewed by Paulo Motta for CASSANDRA-13071
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/496cfa8f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/496cfa8f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/496cfa8f
Branch: refs/heads/cassandra-3.11
Commit: 496cfa8f5b160d26794b346ce360b513069aa300
Parents: 45b6b5b
Author: Stefania Alborghetti
Authored: Wed Feb 22 17:35:57 2017 +
Committer: Stefania Alborghetti
Committed: Thu Mar 2 09:58:58 2017 +0800
--
CHANGES.txt| 1 +
pylib/cqlshlib/copyutil.py | 29 -
2 files changed, 25 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/496cfa8f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c3ac39..1c3869f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.12
+ * Cqlsh copy-from should error out when csv contains invalid data for
collections (CASSANDRA-13071)
* Update c.yaml doc for offheap memtables (CASSANDRA-13179)
* Faster StreamingHistogram (CASSANDRA-13038)
* Legacy deserializer can create unexpected boundary range tombstones
(CASSANDRA-13237)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/496cfa8f/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 710a640..226fad5 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1881,11 +1881,30 @@ class ImportConversion(object):
def split(val, sep=','):
"""
-Split into a list of values whenever we encounter a separator but
+Split "val" into a list of values whenever the separator "sep" is
found, but
ignore separators inside parentheses or single quotes, except for
the two
-outermost parentheses, which will be ignored. We expect val to be
at least
-2 characters long (the two outer parentheses).
+outermost parentheses, which will be ignored. This method is
called when parsing composite
+types, "val" should be at least 2 characters long, the first char
should be an
+open parenthesis and the last char should be a matching closing
parenthesis. We could also
+check exactly which parenthesis type depending on the caller, but
I don't want to enforce
+too many checks that don't necessarily provide any additional
benefits, and risk breaking
+data that could previously be imported, even if strictly speaking
it is incorrect CQL.
+For example, right now we accept sets that start with '[' and ']',
I don't want to break this
+by enforcing '{' and '}' in a minor release.
"""
+def is_open_paren(cc):
+return cc == '{' or cc == '[' or cc == '('
+
+def is_close_paren(cc):
+return cc == '}' or cc == ']' or cc == ')'
+
+def paren_match(c1, c2):
+return (c1 == '{' and c2 == '}') or (c1 == '[' and c2 == ']')
or (c1 == '(' and c2 == ')')
+
+if len(val) < 2 or not paren_match(val[0], val[-1]):
+raise ParseError('Invalid composite string, it should start
and end with matching parentheses: {}'
+ .format(val))
+
ret = []
last = 1
level = 0
@@ -1894,9 +1913,9 @@ class ImportConversion(object):
if c == '\'':
quote = not quote
elif not quote:
-if c == '{' or c == '[' or c == '(':
+if is_open_paren(c):
level += 1
-elif c == '}' or c == ']' or c == ')':
+elif is_close_paren(c):
level -= 1
elif c == sep and level == 1:
ret.append(val[last:i])
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f08cf5af Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f08cf5af Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f08cf5af Branch: refs/heads/trunk Commit: f08cf5af0da0ca44944dfc30f22017f1062b1eff Parents: d91b400 943fb02 Author: Stefania Alborghetti Authored: Thu Mar 2 10:00:40 2017 +0800 Committer: Stefania Alborghetti Committed: Thu Mar 2 10:00:40 2017 +0800 -- CHANGES.txt| 1 + pylib/cqlshlib/copyutil.py | 29 - 2 files changed, 25 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f08cf5af/CHANGES.txt --
[3/6] cassandra git commit: Cqlsh copy-from should error out when csv contains invalid data for collections
Cqlsh copy-from should error out when csv contains invalid data for collections
patch by Stefania Alborghetti; reviewed by Paulo Motta for CASSANDRA-13071
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/496cfa8f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/496cfa8f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/496cfa8f
Branch: refs/heads/trunk
Commit: 496cfa8f5b160d26794b346ce360b513069aa300
Parents: 45b6b5b
Author: Stefania Alborghetti
Authored: Wed Feb 22 17:35:57 2017 +
Committer: Stefania Alborghetti
Committed: Thu Mar 2 09:58:58 2017 +0800
--
CHANGES.txt| 1 +
pylib/cqlshlib/copyutil.py | 29 -
2 files changed, 25 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/496cfa8f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c3ac39..1c3869f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.12
+ * Cqlsh copy-from should error out when csv contains invalid data for
collections (CASSANDRA-13071)
* Update c.yaml doc for offheap memtables (CASSANDRA-13179)
* Faster StreamingHistogram (CASSANDRA-13038)
* Legacy deserializer can create unexpected boundary range tombstones
(CASSANDRA-13237)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/496cfa8f/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 710a640..226fad5 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1881,11 +1881,30 @@ class ImportConversion(object):
def split(val, sep=','):
"""
-Split into a list of values whenever we encounter a separator but
+Split "val" into a list of values whenever the separator "sep" is
found, but
ignore separators inside parentheses or single quotes, except for
the two
-outermost parentheses, which will be ignored. We expect val to be
at least
-2 characters long (the two outer parentheses).
+outermost parentheses, which will be ignored. This method is
called when parsing composite
+types, "val" should be at least 2 characters long, the first char
should be an
+open parenthesis and the last char should be a matching closing
parenthesis. We could also
+check exactly which parenthesis type depending on the caller, but
I don't want to enforce
+too many checks that don't necessarily provide any additional
benefits, and risk breaking
+data that could previously be imported, even if strictly speaking
it is incorrect CQL.
+For example, right now we accept sets that start with '[' and ']',
I don't want to break this
+by enforcing '{' and '}' in a minor release.
"""
+def is_open_paren(cc):
+return cc == '{' or cc == '[' or cc == '('
+
+def is_close_paren(cc):
+return cc == '}' or cc == ']' or cc == ')'
+
+def paren_match(c1, c2):
+return (c1 == '{' and c2 == '}') or (c1 == '[' and c2 == ']')
or (c1 == '(' and c2 == ')')
+
+if len(val) < 2 or not paren_match(val[0], val[-1]):
+raise ParseError('Invalid composite string, it should start
and end with matching parentheses: {}'
+ .format(val))
+
ret = []
last = 1
level = 0
@@ -1894,9 +1913,9 @@ class ImportConversion(object):
if c == '\'':
quote = not quote
elif not quote:
-if c == '{' or c == '[' or c == '(':
+if is_open_paren(c):
level += 1
-elif c == '}' or c == ']' or c == ')':
+elif is_close_paren(c):
level -= 1
elif c == sep and level == 1:
ret.append(val[last:i])
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/943fb02f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/943fb02f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/943fb02f Branch: refs/heads/trunk Commit: 943fb02ffda7b7c65ec722d18a5c184fa480f525 Parents: 093b2e2 496cfa8 Author: Stefania Alborghetti Authored: Thu Mar 2 10:00:20 2017 +0800 Committer: Stefania Alborghetti Committed: Thu Mar 2 10:00:20 2017 +0800 -- CHANGES.txt| 1 + pylib/cqlshlib/copyutil.py | 29 - 2 files changed, 25 insertions(+), 5 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/943fb02f/CHANGES.txt -- diff --cc CHANGES.txt index cf15d3a,1c3869f..eeb2215 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,15 -1,6 +1,16 @@@ -3.0.12 +3.11.0 + * Fix equality comparisons of columns using the duration type (CASSANDRA-13174) + * Obfuscate password in stress-graphs (CASSANDRA-12233) + * Move to FastThreadLocalThread and FastThreadLocal (CASSANDRA-13034) + * nodetool stopdaemon errors out (CASSANDRA-13030) + * Tables in system_distributed should not use gcgs of 0 (CASSANDRA-12954) + * Fix primary index calculation for SASI (CASSANDRA-12910) + * More fixes to the TokenAllocator (CASSANDRA-12990) + * NoReplicationTokenAllocator should work with zero replication factor (CASSANDRA-12983) +Merged from 3.0: + * Cqlsh copy-from should error out when csv contains invalid data for collections (CASSANDRA-13071) - * Update c.yaml doc for offheap memtables (CASSANDRA-13179) + * Fix "multiple versions of ant detected..." when running ant test (CASSANDRA-13232) + * Coalescing strategy sleeps too much (CASSANDRA-13090) * Faster StreamingHistogram (CASSANDRA-13038) * Legacy deserializer can create unexpected boundary range tombstones (CASSANDRA-13237) * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070) http://git-wip-us.apache.org/repos/asf/cassandra/blob/943fb02f/pylib/cqlshlib/copyutil.py --
[1/6] cassandra git commit: Cqlsh copy-from should error out when csv contains invalid data for collections
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 45b6b5b1c -> 496cfa8f5
refs/heads/cassandra-3.11 093b2e294 -> 943fb02ff
refs/heads/trunk d91b40039 -> f08cf5af0
Cqlsh copy-from should error out when csv contains invalid data for collections
patch by Stefania Alborghetti; reviewed by Paulo Motta for CASSANDRA-13071
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/496cfa8f
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/496cfa8f
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/496cfa8f
Branch: refs/heads/cassandra-3.0
Commit: 496cfa8f5b160d26794b346ce360b513069aa300
Parents: 45b6b5b
Author: Stefania Alborghetti
Authored: Wed Feb 22 17:35:57 2017 +
Committer: Stefania Alborghetti
Committed: Thu Mar 2 09:58:58 2017 +0800
--
CHANGES.txt| 1 +
pylib/cqlshlib/copyutil.py | 29 -
2 files changed, 25 insertions(+), 5 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/496cfa8f/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 2c3ac39..1c3869f 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.12
+ * Cqlsh copy-from should error out when csv contains invalid data for
collections (CASSANDRA-13071)
* Update c.yaml doc for offheap memtables (CASSANDRA-13179)
* Faster StreamingHistogram (CASSANDRA-13038)
* Legacy deserializer can create unexpected boundary range tombstones
(CASSANDRA-13237)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/496cfa8f/pylib/cqlshlib/copyutil.py
--
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 710a640..226fad5 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -1881,11 +1881,30 @@ class ImportConversion(object):
def split(val, sep=','):
"""
-Split into a list of values whenever we encounter a separator but
+Split "val" into a list of values whenever the separator "sep" is
found, but
ignore separators inside parentheses or single quotes, except for
the two
-outermost parentheses, which will be ignored. We expect val to be
at least
-2 characters long (the two outer parentheses).
+outermost parentheses, which will be ignored. This method is
called when parsing composite
+types, "val" should be at least 2 characters long, the first char
should be an
+open parenthesis and the last char should be a matching closing
parenthesis. We could also
+check exactly which parenthesis type depending on the caller, but
I don't want to enforce
+too many checks that don't necessarily provide any additional
benefits, and risk breaking
+data that could previously be imported, even if strictly speaking
it is incorrect CQL.
+For example, right now we accept sets that start with '[' and ']',
I don't want to break this
+by enforcing '{' and '}' in a minor release.
"""
+def is_open_paren(cc):
+return cc == '{' or cc == '[' or cc == '('
+
+def is_close_paren(cc):
+return cc == '}' or cc == ']' or cc == ')'
+
+def paren_match(c1, c2):
+return (c1 == '{' and c2 == '}') or (c1 == '[' and c2 == ']')
or (c1 == '(' and c2 == ')')
+
+if len(val) < 2 or not paren_match(val[0], val[-1]):
+raise ParseError('Invalid composite string, it should start
and end with matching parentheses: {}'
+ .format(val))
+
ret = []
last = 1
level = 0
@@ -1894,9 +1913,9 @@ class ImportConversion(object):
if c == '\'':
quote = not quote
elif not quote:
-if c == '{' or c == '[' or c == '(':
+if is_open_paren(c):
level += 1
-elif c == '}' or c == ']' or c == ')':
+elif is_close_paren(c):
level -= 1
elif c == sep and level == 1:
ret.append(val[last:i])
[jira] [Updated] (CASSANDRA-12886) Streaming failed due to SSL Socket connection reset
[
https://issues.apache.org/jira/browse/CASSANDRA-12886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-12886:
Resolution: Fixed
Fix Version/s: 3.0.12
2.2.10
Status: Resolved (was: Ready to Commit)
Committed to 2.2 as 06feaefba50301734c490521d720c8a482f638e4 and merged to 3.0.
Thanks!
> Streaming failed due to SSL Socket connection reset
> ---
>
> Key: CASSANDRA-12886
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12886
> Project: Cassandra
> Issue Type: Bug
>Reporter: Bing Wu
>Assignee: Paulo Motta
> Fix For: 2.2.10, 3.0.12
>
> Attachments: debug.log.2016-11-10_2319.gz
>
>
> While running "nodetool repair", I see many instances of
> "javax.net.ssl.SSLException: java.net.SocketException: Connection reset" in
> system.logs on some nodes in the cluster. Timestamps correspond to streaming
> source/initiator's error messages of "sync failed between ..."
> Setup:
> - Cassandra 3.7.01
> - CentOS 6.7 in AWS (multi-region)
> - JDK version: {noformat}
> java version "1.8.0_102"
> Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
> Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)
> {noformat}
> - cassandra.yaml:
> {noformat}
> server_encryption_options:
> internode_encryption: all
> keystore: [path]
> keystore_password: [password]
> truststore: [path]
> truststore_password: [password]
> # More advanced defaults below:
> # protocol: TLS
> # algorithm: SunX509
> # store_type: JKS
> # cipher_suites:
> [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]
> require_client_auth: false
> {noformat}
> Error messages in system.log on the target host:
> {noformat}
> ERROR [STREAM-OUT-/54.247.111.232:7001] 2016-11-07 07:30:56,475
> StreamSession.java:529 - [Stream #e14abcb0-a4bb-11e6-9758-55b9ac38b78e]
> Streaming error occurred on session with peer 54.247.111.232
> javax.net.ssl.SSLException: Connection has been shutdown:
> javax.net.ssl.SSLException: java.net.SocketException: Connection reset
> at sun.security.ssl.SSLSocketImpl.checkEOF(SSLSocketImpl.java:1541)
> ~[na:1.8.0_102]
> at sun.security.ssl.SSLSocketImpl.checkWrite(SSLSocketImpl.java:1553)
> ~[na:1.8.0_102]
> at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:71)
> ~[na:1.8.0_102]
> at
> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
> ~[na:1.8.0_102]
> at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
> ~[na:1.8.0_102]
> at
> org.apache.cassandra.io.util.WrappedDataOutputStreamPlus.flush(WrappedDataOutputStreamPlus.java:66)
> ~[apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:371)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at
> org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:342)
> [apache-cassandra-3.7.0.jar:3.7.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
> Caused by: javax.net.ssl.SSLException: java.net.SocketException: Connection
> reset
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[6/6] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d91b4003 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d91b4003 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d91b4003 Branch: refs/heads/trunk Commit: d91b40039e7d8cd1c997acbbd06124e3bfab69b6 Parents: d74cc08 093b2e2 Author: Paulo Motta Authored: Wed Mar 1 22:13:04 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 22:13:04 2017 -0300 -- --
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/093b2e29 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/093b2e29 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/093b2e29 Branch: refs/heads/trunk Commit: 093b2e294c2de4ee9105c9283462a9c8a254af3c Parents: 4599da2 45b6b5b Author: Paulo Motta Authored: Wed Mar 1 22:12:36 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 22:12:36 2017 -0300 -- --
[3/6] cassandra git commit: ninja: fix CHANGES.txt
ninja: fix CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45b6b5b1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45b6b5b1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45b6b5b1 Branch: refs/heads/trunk Commit: 45b6b5b1c693b020361b4341d187725d016c0e17 Parents: 143c503 Author: Paulo Motta Authored: Wed Mar 1 22:11:19 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 22:11:19 2017 -0300 -- CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/45b6b5b1/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ec3d826..2c3ac39 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,11 +1,11 @@ 3.0.12 - * Avoid race on receiver by starting streaming sender thread after sending init message (CASSANDRA-12886) * Update c.yaml doc for offheap memtables (CASSANDRA-13179) * Faster StreamingHistogram (CASSANDRA-13038) * Legacy deserializer can create unexpected boundary range tombstones (CASSANDRA-13237) * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070) * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185) Merged from 2.2 + * Avoid race on receiver by starting streaming sender thread after sending init message (CASSANDRA-12886) * Fix "multiple versions of ant detected..." when running ant test (CASSANDRA-13232) * Coalescing strategy sleeps too much (CASSANDRA-13090) * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
[2/6] cassandra git commit: ninja: fix CHANGES.txt
ninja: fix CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45b6b5b1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45b6b5b1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45b6b5b1 Branch: refs/heads/cassandra-3.11 Commit: 45b6b5b1c693b020361b4341d187725d016c0e17 Parents: 143c503 Author: Paulo Motta Authored: Wed Mar 1 22:11:19 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 22:11:19 2017 -0300 -- CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/45b6b5b1/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ec3d826..2c3ac39 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,11 +1,11 @@ 3.0.12 - * Avoid race on receiver by starting streaming sender thread after sending init message (CASSANDRA-12886) * Update c.yaml doc for offheap memtables (CASSANDRA-13179) * Faster StreamingHistogram (CASSANDRA-13038) * Legacy deserializer can create unexpected boundary range tombstones (CASSANDRA-13237) * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070) * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185) Merged from 2.2 + * Avoid race on receiver by starting streaming sender thread after sending init message (CASSANDRA-12886) * Fix "multiple versions of ant detected..." when running ant test (CASSANDRA-13232) * Coalescing strategy sleeps too much (CASSANDRA-13090) * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/093b2e29 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/093b2e29 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/093b2e29 Branch: refs/heads/cassandra-3.11 Commit: 093b2e294c2de4ee9105c9283462a9c8a254af3c Parents: 4599da2 45b6b5b Author: Paulo Motta Authored: Wed Mar 1 22:12:36 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 22:12:36 2017 -0300 -- --
[1/6] cassandra git commit: ninja: fix CHANGES.txt
Repository: cassandra Updated Branches: refs/heads/cassandra-3.0 143c50339 -> 45b6b5b1c refs/heads/cassandra-3.11 4599da218 -> 093b2e294 refs/heads/trunk d74cc0802 -> d91b40039 ninja: fix CHANGES.txt Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/45b6b5b1 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/45b6b5b1 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/45b6b5b1 Branch: refs/heads/cassandra-3.0 Commit: 45b6b5b1c693b020361b4341d187725d016c0e17 Parents: 143c503 Author: Paulo Motta Authored: Wed Mar 1 22:11:19 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 22:11:19 2017 -0300 -- CHANGES.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/45b6b5b1/CHANGES.txt -- diff --git a/CHANGES.txt b/CHANGES.txt index ec3d826..2c3ac39 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,11 +1,11 @@ 3.0.12 - * Avoid race on receiver by starting streaming sender thread after sending init message (CASSANDRA-12886) * Update c.yaml doc for offheap memtables (CASSANDRA-13179) * Faster StreamingHistogram (CASSANDRA-13038) * Legacy deserializer can create unexpected boundary range tombstones (CASSANDRA-13237) * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070) * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185) Merged from 2.2 + * Avoid race on receiver by starting streaming sender thread after sending init message (CASSANDRA-12886) * Fix "multiple versions of ant detected..." when running ant test (CASSANDRA-13232) * Coalescing strategy sleeps too much (CASSANDRA-13090) * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4599da21 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4599da21 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4599da21 Branch: refs/heads/cassandra-3.11 Commit: 4599da2182722d3df9cd7b7000d600e316d31893 Parents: e345d4e 143c503 Author: Michael Shuler Authored: Wed Mar 1 18:47:42 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:42 2017 -0600 -- --
[04/10] cassandra git commit: Fix 2.2 build on byteman-install dependency
Fix 2.2 build on byteman-install dependency Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f99c4e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f99c4e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f99c4e Branch: refs/heads/trunk Commit: c0f99c4e68da8e4e7f3d430b2cb45a762c1ff9f2 Parents: 06feaef Author: Michael Shuler Authored: Wed Mar 1 18:47:07 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:07 2017 -0600 -- build.xml | 2 ++ 1 file changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f99c4e/build.xml -- diff --git a/build.xml b/build.xml index 6dbdb9e..867afa4 100644 --- a/build.xml +++ b/build.xml @@ -393,6 +393,7 @@ + @@ -528,6 +529,7 @@ +
[02/10] cassandra git commit: Fix 2.2 build on byteman-install dependency
Fix 2.2 build on byteman-install dependency Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f99c4e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f99c4e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f99c4e Branch: refs/heads/cassandra-3.0 Commit: c0f99c4e68da8e4e7f3d430b2cb45a762c1ff9f2 Parents: 06feaef Author: Michael Shuler Authored: Wed Mar 1 18:47:07 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:07 2017 -0600 -- build.xml | 2 ++ 1 file changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f99c4e/build.xml -- diff --git a/build.xml b/build.xml index 6dbdb9e..867afa4 100644 --- a/build.xml +++ b/build.xml @@ -393,6 +393,7 @@ + @@ -528,6 +529,7 @@ +
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4599da21 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4599da21 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4599da21 Branch: refs/heads/trunk Commit: 4599da2182722d3df9cd7b7000d600e316d31893 Parents: e345d4e 143c503 Author: Michael Shuler Authored: Wed Mar 1 18:47:42 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:42 2017 -0600 -- --
[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d74cc080 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d74cc080 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d74cc080 Branch: refs/heads/trunk Commit: d74cc0802c976713415aa39838bcebe333753e12 Parents: 9ba3185 4599da2 Author: Michael Shuler Authored: Wed Mar 1 18:48:02 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:48:02 2017 -0600 -- --
[01/10] cassandra git commit: Fix 2.2 build on byteman-install dependency
Repository: cassandra Updated Branches: refs/heads/cassandra-2.2 06feaefba -> c0f99c4e6 refs/heads/cassandra-3.0 815508f1c -> 143c50339 refs/heads/cassandra-3.11 e345d4e7b -> 4599da218 refs/heads/trunk 9ba3185b6 -> d74cc0802 Fix 2.2 build on byteman-install dependency Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f99c4e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f99c4e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f99c4e Branch: refs/heads/cassandra-2.2 Commit: c0f99c4e68da8e4e7f3d430b2cb45a762c1ff9f2 Parents: 06feaef Author: Michael Shuler Authored: Wed Mar 1 18:47:07 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:07 2017 -0600 -- build.xml | 2 ++ 1 file changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f99c4e/build.xml -- diff --git a/build.xml b/build.xml index 6dbdb9e..867afa4 100644 --- a/build.xml +++ b/build.xml @@ -393,6 +393,7 @@ + @@ -528,6 +529,7 @@ +
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/143c5033 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/143c5033 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/143c5033 Branch: refs/heads/cassandra-3.11 Commit: 143c503395418ba87db015b07e6a9e98fa120803 Parents: 815508f c0f99c4 Author: Michael Shuler Authored: Wed Mar 1 18:47:23 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:23 2017 -0600 -- --
[03/10] cassandra git commit: Fix 2.2 build on byteman-install dependency
Fix 2.2 build on byteman-install dependency Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/c0f99c4e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/c0f99c4e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/c0f99c4e Branch: refs/heads/cassandra-3.11 Commit: c0f99c4e68da8e4e7f3d430b2cb45a762c1ff9f2 Parents: 06feaef Author: Michael Shuler Authored: Wed Mar 1 18:47:07 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:07 2017 -0600 -- build.xml | 2 ++ 1 file changed, 2 insertions(+) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/c0f99c4e/build.xml -- diff --git a/build.xml b/build.xml index 6dbdb9e..867afa4 100644 --- a/build.xml +++ b/build.xml @@ -393,6 +393,7 @@ + @@ -528,6 +529,7 @@ +
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/143c5033 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/143c5033 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/143c5033 Branch: refs/heads/cassandra-3.0 Commit: 143c503395418ba87db015b07e6a9e98fa120803 Parents: 815508f c0f99c4 Author: Michael Shuler Authored: Wed Mar 1 18:47:23 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:23 2017 -0600 -- --
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/143c5033 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/143c5033 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/143c5033 Branch: refs/heads/trunk Commit: 143c503395418ba87db015b07e6a9e98fa120803 Parents: 815508f c0f99c4 Author: Michael Shuler Authored: Wed Mar 1 18:47:23 2017 -0600 Committer: Michael Shuler Committed: Wed Mar 1 18:47:23 2017 -0600 -- --
[jira] [Comment Edited] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to
[
https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891358#comment-15891358
]
Jon Haddad edited comment on CASSANDRA-13010 at 3/2/17 12:26 AM:
-
I set up 6 data directories and dropped my memtable_heap_space_in_mb to 32 MB
to force a ton of flushes and lots of compactions. I do see directories in the
compactionstats, indicating that it's working, but i haven't verified the right
directories are listed.
The output is a little hard to read, since the directories make the output look
like this:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table target
directory
completed totalunit progress
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
17696623 38340460 bytes 46.16%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
3651789 33841385 bytes 10.79%
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
4698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
>From a user interface perspective, It would be great if the tasks could be
>separated by directory, rather than inlining the directory in the table. So
>in my example something more like:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table completed
totalunit progress
/Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
17696623 38340460 bytes 46.16%
/Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
3651789 33841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
34012548 37166390 bytes 91.51%
/Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
4698 36996530 bytes 90.10%
/Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
If there are multiple compactions in a single directory, you would see this:
{code}
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
4698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
22993893 37875910 bytes 60.71%
{code}
I'm also not sure if there's value in including the keyspace & table directory,
since that information is duplicated. If we limit the result to the data
directory we would end up with at most N sections, 1 per data directory
specified in the yaml. I think that would be the *most* useful form of this
output (note the line between the directories):
{code}
/Users/jhaddad/var/lib/cassandra/data4
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
3651789 33841385 bytes 10.79%
03c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard2
3651789 13841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard3
34012548 57166390 bytes 91.51%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard5
3651789 43841385 bytes 10.79%
{code}
What do you think?
was (Author: rustyrazorblade):
I set up 6 data directories and dropped my memtabl
[jira] [Commented] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to
[
https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891358#comment-15891358
]
Jon Haddad commented on CASSANDRA-13010:
I set up 6 data directories and dropped my memtable_heap_space_in_mb to 32 MB
to force a ton of flushes and lots of compactions. I do see directories in the
compactionstats, indicating that it's working, but i haven't verified the right
directories are listed.
The output is a little hard to read, since the directories make the output look
like this:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table target
directory
completed totalunit progress
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
17696623 38340460 bytes 46.16%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
3651789 33841385 bytes 10.79%
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
4698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
/Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
>From a user interface perspective, It would be great if the tasks could be
>separated by directory, rather than inlining the directory in the table. So
>in my example something more like:
{code}
jhaddad@rustyrazorblade ~/dev/cassandra$ bin/nodetool compactionstats
pending tasks: 6
- keyspace1.standard1: 6
id compaction type keyspace table completed
totalunit progress
/Users/jhaddad/var/lib/cassandra/data1/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7f3e7a40-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
17696623 38340460 bytes 46.16%
/Users/jhaddad/var/lib/cassandra/data4/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
3651789 33841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
34012548 37166390 bytes 91.51%
/Users/jhaddad/var/lib/cassandra/data6/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
4698 36996530 bytes 90.10%
/Users/jhaddad/var/lib/cassandra/data2/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
22993893 37875910 bytes 60.71%
Active compaction remaining time : 0h00m04s
{code}
If there are multiple compactions in a single directory, you would see this:
{code}
/Users/jhaddad/var/lib/cassandra/data3/keyspace1/standard1-f867e470fedb11e6a2c121962153156a
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
34012548 37166390 bytes 91.51%
7a2e6560-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
4698 36996530 bytes 90.10%
7dabcc50-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
22993893 37875910 bytes 60.71%
{code}
I'm also not sure if there's value in including the keyspace & table directory,
since that information is duplicated. If we limit the result to the data
directory we would end up with at most N sections, 1 per data directory
specified in the yaml. I think that would be the *most* useful form of this
output. I believe this would be the most helpful output (note the line between
the directories):
{code}
/Users/jhaddad/var/lib/cassandra/data4
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard1
3651789 33841385 bytes 10.79%
03c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard2
3651789 13841385 bytes 10.79%
/Users/jhaddad/var/lib/cassandra/data3
7a1beed0-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard3
34012548 57166390 bytes 91.51%
83c14340-fedc-11e6-a477-b714107dace1 Compaction keyspace1 standard5
3651789 43841385 bytes 10.79%
{code}
What do you think?
> nodetool compactionstats should say which disk a compaction is writing to
> --
cassandra git commit: add ns timeunit marker to logger output
Repository: cassandra
Updated Branches:
refs/heads/trunk f32521808 -> 9ba3185b6
add ns timeunit marker to logger output
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9ba3185b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9ba3185b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9ba3185b
Branch: refs/heads/trunk
Commit: 9ba3185b60fa9cbd4fd8effe55b0935f1736fc2f
Parents: f325218
Author: Dave Brosius
Authored: Wed Mar 1 19:24:02 2017 -0500
Committer: Dave Brosius
Committed: Wed Mar 1 19:24:02 2017 -0500
--
src/java/org/apache/cassandra/gms/FailureDetector.java | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9ba3185b/src/java/org/apache/cassandra/gms/FailureDetector.java
--
diff --git a/src/java/org/apache/cassandra/gms/FailureDetector.java
b/src/java/org/apache/cassandra/gms/FailureDetector.java
index c73cc07..e5a29d6 100644
--- a/src/java/org/apache/cassandra/gms/FailureDetector.java
+++ b/src/java/org/apache/cassandra/gms/FailureDetector.java
@@ -270,7 +270,7 @@ public class FailureDetector implements IFailureDetector,
FailureDetectorMBean
}
if (logger.isTraceEnabled() && heartbeatWindow != null)
-logger.trace("Average for {} is {}", ep, heartbeatWindow.mean());
+logger.trace("Average for {} is {}ns", ep, heartbeatWindow.mean());
}
public void interpret(InetAddress ep)
@@ -285,7 +285,7 @@ public class FailureDetector implements IFailureDetector,
FailureDetectorMBean
lastInterpret = now;
if (diff > MAX_LOCAL_PAUSE_IN_NANOS)
{
-logger.warn("Not marking nodes down due to local pause of {} >
{}", diff, MAX_LOCAL_PAUSE_IN_NANOS);
+logger.warn("Not marking nodes down due to local pause of {}ns >
{}ns", diff, MAX_LOCAL_PAUSE_IN_NANOS);
lastPause = now;
return;
}
@@ -301,7 +301,7 @@ public class FailureDetector implements IFailureDetector,
FailureDetectorMBean
if (PHI_FACTOR * phi > getPhiConvictThreshold())
{
if (logger.isTraceEnabled())
-logger.trace("Node {} phi {} > {}; intervals: {} mean: {}",
new Object[]{ep, PHI_FACTOR * phi, getPhiConvictThreshold(), hbWnd,
hbWnd.mean()});
+logger.trace("Node {} phi {} > {}; intervals: {} mean: {}ns",
new Object[]{ep, PHI_FACTOR * phi, getPhiConvictThreshold(), hbWnd,
hbWnd.mean()});
for (IFailureDetectionEventListener listener : fdEvntListeners)
{
listener.convict(ep, phi);
@@ -314,7 +314,7 @@ public class FailureDetector implements IFailureDetector,
FailureDetectorMBean
else if (logger.isTraceEnabled())
{
logger.trace("PHI for {} : {}", ep, phi);
-logger.trace("mean for {} : {}", ep, hbWnd.mean());
+logger.trace("mean for {} : {}ns", ep, hbWnd.mean());
}
}
@@ -450,11 +450,11 @@ class ArrivalWindow
if (interArrivalTime <= MAX_INTERVAL_IN_NANO)
{
arrivalIntervals.add(interArrivalTime);
-logger.trace("Reporting interval time of {} for {}",
interArrivalTime, ep);
+logger.trace("Reporting interval time of {}ns for {}",
interArrivalTime, ep);
}
else
{
-logger.debug("Ignoring interval time of {} for {}",
interArrivalTime, ep);
+logger.debug("Ignoring interval time of {}ns for {}",
interArrivalTime, ep);
}
}
else
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[
https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891356#comment-15891356
]
Nate McCall commented on CASSANDRA-13265:
-
bq. How often does this issue occur?
Not very often, IME, but it does happen. You can see this if you monitor the
{{ConnectionMetrics#commandDroppedTasks}} gauge.
> Epxiration in OutboundTcpConnection can block the reader Thread
> ---
>
> Key: CASSANDRA-13265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13265
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 3.0.9
> Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version
> 1.8.0_112-b15)
> Linux 3.16
>Reporter: Christian Esken
>Assignee: Christian Esken
> Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz,
> cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz
>
>
> I observed that sometimes a single node in a Cassandra cluster fails to
> communicate to the other nodes. This can happen at any time, during peak load
> or low load. Restarting that single node from the cluster fixes the issue.
> Before going in to details, I want to state that I have analyzed the
> situation and am already developing a possible fix. Here is the analysis so
> far:
> - A Threaddump in this situation showed 324 Threads in the
> OutboundTcpConnection class that want to lock the backlog queue for doing
> expiration.
> - A class histogram shows 262508 instances of
> OutboundTcpConnection$QueuedMessage.
> What is the effect of it? As soon as the Cassandra node has reached a certain
> amount of queued messages, it starts thrashing itself to death. Each of the
> Thread fully locks the Queue for reading and writing by calling
> iterator.next(), making the situation worse and worse.
> - Writing: Only after 262508 locking operation it can progress with actually
> writing to the Queue.
> - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and
> fully lock the Queue
> This means: Writing blocks the Queue for reading, and readers might even be
> starved which makes the situation even worse.
> -
> The setup is:
> - 3-node cluster
> - replication factor 2
> - Consistency LOCAL_ONE
> - No remote DC's
> - high write throughput (10 INSERT statements per second and more during
> peak times).
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13060) NPE in StorageService.java while bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-13060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891330#comment-15891330
]
Jay Zhuang commented on CASSANDRA-13060:
[~alekiv] One workaround is set {{cassandra.ring_delay_ms}} to larger number
for example: 100 (seconds). The default value is 30 (seconds):
https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/service/StorageService.java#L122
The larger your cluster is, the longer it needs to wait.
> NPE in StorageService.java while bootstrapping
> --
>
> Key: CASSANDRA-13060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13060
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Trivial
> Labels: lhf
> Fix For: 3.0.x
>
> Attachments: 13060-3.0.txt
>
>
> Lots of NPE happens when bootstrapping new node:
> {code}
> WARN [SharedPool-Worker-1] 2016-12-19 23:09:09,034
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.service.StorageService.isRpcReady(StorageService.java:1829)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.service.StorageService.notifyUp(StorageService.java:1787)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.service.StorageService.onAlive(StorageService.java:2424)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at org.apache.cassandra.gms.Gossiper.realMarkAlive(Gossiper.java:999)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at org.apache.cassandra.gms.Gossiper$3.response(Gossiper.java:979)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:53)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
> [apache-cassandra-3.0.10.jar:3.0.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.0.10.jar:3.0.10]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[10/10] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f3252180 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f3252180 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f3252180 Branch: refs/heads/trunk Commit: f3252180863d0a2e78d1bfb5437506033a57f931 Parents: 21b4cae e345d4e Author: Paulo Motta Authored: Wed Mar 1 20:34:28 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 20:34:28 2017 -0300 -- --
[04/10] cassandra git commit: Avoid race on receiver by starting streaming sender thread after sending init message
Avoid race on receiver by starting streaming sender thread after sending init
message
Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-12886
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06feaefb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06feaefb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06feaefb
Branch: refs/heads/trunk
Commit: 06feaefba50301734c490521d720c8a482f638e4
Parents: 9bbb449
Author: Paulo Motta
Authored: Wed Mar 1 20:25:32 2017 -0300
Committer: Paulo Motta
Committed: Wed Mar 1 20:30:30 2017 -0300
--
CHANGES.txt | 1 +
.../cassandra/streaming/ConnectionHandler.java | 26 +++-
2 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 404440a..ca1aa27 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.10
+ * Avoid race on receiver by starting streaming sender thread after sending
init message (CASSANDRA-12886)
* Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
* Coalescing strategy sleeps too much (CASSANDRA-13090)
* Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index d3d8ed2..fe551a8 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -82,13 +82,11 @@ public class ConnectionHandler
{
logger.debug("[Stream #{}] Sending stream init for incoming stream",
session.planId());
Socket incomingSocket = session.createConnection();
-incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION);
-incoming.sendInitMessage(incomingSocket, true);
+incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION, true);
logger.debug("[Stream #{}] Sending stream init for outgoing stream",
session.planId());
Socket outgoingSocket = session.createConnection();
-outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION);
-outgoing.sendInitMessage(outgoingSocket, false);
+outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION, true);
}
/**
@@ -159,13 +157,15 @@ public class ConnectionHandler
protected int protocolVersion;
protected Socket socket;
+private final boolean isOutgoingHandler;
private final AtomicReference> closeFuture = new
AtomicReference<>();
private IncomingStreamingConnection incomingConnection;
-protected MessageHandler(StreamSession session)
+protected MessageHandler(StreamSession session, boolean
isOutgoingHandler)
{
this.session = session;
+this.isOutgoingHandler = isOutgoingHandler;
}
protected abstract String name();
@@ -187,14 +187,14 @@ public class ConnectionHandler
}
@SuppressWarnings("resource")
-public void sendInitMessage(Socket socket, boolean isForOutgoing)
throws IOException
+public void sendInitMessage() throws IOException
{
StreamInitMessage message = new StreamInitMessage(
FBUtilities.getBroadcastAddress(),
session.sessionIndex(),
session.planId(),
session.description(),
-isForOutgoing,
+!isOutgoingHandler,
session.keepSSTableLevel(),
session.isIncremental());
ByteBuffer messageBuf = message.createMessage(false,
protocolVersion);
@@ -203,16 +203,18 @@ public class ConnectionHandler
out.flush();
}
-public void start(IncomingStreamingConnection connection, int
protocolVersion)
+public void start(IncomingStreamingConnection connection, int
protocolVersion) throws IOException
{
this.incomingConnection = connection;
-start(connection.socket, protocolVersion);
+start(connection.socket, protocolVersion, false);
}
-public void start(Socket socket, int protocolVersion)
+public void start(Socket socket, int protocolVersion, boolean
initiator) throws IOException
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e345d4e7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e345d4e7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e345d4e7 Branch: refs/heads/trunk Commit: e345d4e7bc83e09e3c273981b5c95e5fb00d1df5 Parents: be0deb9 815508f Author: Paulo Motta Authored: Wed Mar 1 20:33:49 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 20:33:49 2017 -0300 -- --
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/815508f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/815508f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/815508f1
Branch: refs/heads/cassandra-3.11
Commit: 815508f1c240bbbac8246be56e15b1d09c1301b8
Parents: 218743d 06feaef
Author: Paulo Motta
Authored: Wed Mar 1 20:33:16 2017 -0300
Committer: Paulo Motta
Committed: Wed Mar 1 20:33:16 2017 -0300
--
CHANGES.txt | 1 +
.../cassandra/streaming/ConnectionHandler.java | 26 +++-
2 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815508f1/CHANGES.txt
--
diff --cc CHANGES.txt
index c4622ab,ca1aa27..ec3d826
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,12 -1,8 +1,13 @@@
-2.2.10
+3.0.12
+ * Avoid race on receiver by starting streaming sender thread after sending
init message (CASSANDRA-12886)
+ * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
+ * Faster StreamingHistogram (CASSANDRA-13038)
+ * Legacy deserializer can create unexpected boundary range tombstones
(CASSANDRA-13237)
+ * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
+Merged from 2.2
* Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
* Coalescing strategy sleeps too much (CASSANDRA-13090)
- * Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
* Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
* Fix failing COPY TO STDOUT (CASSANDRA-12497)
* Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815508f1/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --cc src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index b83c089,fe551a8..aa1c615
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@@ -215,8 -213,10 +215,10 @@@ public class ConnectionHandle
{
this.socket = socket;
this.protocolVersion = protocolVersion;
+ if (initiator)
+ sendInitMessage();
-new Thread(this, name() + "-" + session.peer).start();
+new Thread(NamedThreadFactory.threadLocalDeallocator(this),
name() + "-" + session.peer).start();
}
public ListenableFuture close()
[02/10] cassandra git commit: Avoid race on receiver by starting streaming sender thread after sending init message
Avoid race on receiver by starting streaming sender thread after sending init
message
Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-12886
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06feaefb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06feaefb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06feaefb
Branch: refs/heads/cassandra-3.0
Commit: 06feaefba50301734c490521d720c8a482f638e4
Parents: 9bbb449
Author: Paulo Motta
Authored: Wed Mar 1 20:25:32 2017 -0300
Committer: Paulo Motta
Committed: Wed Mar 1 20:30:30 2017 -0300
--
CHANGES.txt | 1 +
.../cassandra/streaming/ConnectionHandler.java | 26 +++-
2 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 404440a..ca1aa27 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.10
+ * Avoid race on receiver by starting streaming sender thread after sending
init message (CASSANDRA-12886)
* Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
* Coalescing strategy sleeps too much (CASSANDRA-13090)
* Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index d3d8ed2..fe551a8 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -82,13 +82,11 @@ public class ConnectionHandler
{
logger.debug("[Stream #{}] Sending stream init for incoming stream",
session.planId());
Socket incomingSocket = session.createConnection();
-incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION);
-incoming.sendInitMessage(incomingSocket, true);
+incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION, true);
logger.debug("[Stream #{}] Sending stream init for outgoing stream",
session.planId());
Socket outgoingSocket = session.createConnection();
-outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION);
-outgoing.sendInitMessage(outgoingSocket, false);
+outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION, true);
}
/**
@@ -159,13 +157,15 @@ public class ConnectionHandler
protected int protocolVersion;
protected Socket socket;
+private final boolean isOutgoingHandler;
private final AtomicReference> closeFuture = new
AtomicReference<>();
private IncomingStreamingConnection incomingConnection;
-protected MessageHandler(StreamSession session)
+protected MessageHandler(StreamSession session, boolean
isOutgoingHandler)
{
this.session = session;
+this.isOutgoingHandler = isOutgoingHandler;
}
protected abstract String name();
@@ -187,14 +187,14 @@ public class ConnectionHandler
}
@SuppressWarnings("resource")
-public void sendInitMessage(Socket socket, boolean isForOutgoing)
throws IOException
+public void sendInitMessage() throws IOException
{
StreamInitMessage message = new StreamInitMessage(
FBUtilities.getBroadcastAddress(),
session.sessionIndex(),
session.planId(),
session.description(),
-isForOutgoing,
+!isOutgoingHandler,
session.keepSSTableLevel(),
session.isIncremental());
ByteBuffer messageBuf = message.createMessage(false,
protocolVersion);
@@ -203,16 +203,18 @@ public class ConnectionHandler
out.flush();
}
-public void start(IncomingStreamingConnection connection, int
protocolVersion)
+public void start(IncomingStreamingConnection connection, int
protocolVersion) throws IOException
{
this.incomingConnection = connection;
-start(connection.socket, protocolVersion);
+start(connection.socket, protocolVersion, false);
}
-public void start(Socket socket, int protocolVersion)
+public void start(Socket socket, int protocolVersion, boolean
initiator) throws IOExcepti
[01/10] cassandra git commit: Avoid race on receiver by starting streaming sender thread after sending init message
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 9bbb44973 -> 06feaefba
refs/heads/cassandra-3.0 218743dc3 -> 815508f1c
refs/heads/cassandra-3.11 be0deb90e -> e345d4e7b
refs/heads/trunk 21b4cae61 -> f32521808
Avoid race on receiver by starting streaming sender thread after sending init
message
Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-12886
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06feaefb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06feaefb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06feaefb
Branch: refs/heads/cassandra-2.2
Commit: 06feaefba50301734c490521d720c8a482f638e4
Parents: 9bbb449
Author: Paulo Motta
Authored: Wed Mar 1 20:25:32 2017 -0300
Committer: Paulo Motta
Committed: Wed Mar 1 20:30:30 2017 -0300
--
CHANGES.txt | 1 +
.../cassandra/streaming/ConnectionHandler.java | 26 +++-
2 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 404440a..ca1aa27 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.10
+ * Avoid race on receiver by starting streaming sender thread after sending
init message (CASSANDRA-12886)
* Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
* Coalescing strategy sleeps too much (CASSANDRA-13090)
* Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index d3d8ed2..fe551a8 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -82,13 +82,11 @@ public class ConnectionHandler
{
logger.debug("[Stream #{}] Sending stream init for incoming stream",
session.planId());
Socket incomingSocket = session.createConnection();
-incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION);
-incoming.sendInitMessage(incomingSocket, true);
+incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION, true);
logger.debug("[Stream #{}] Sending stream init for outgoing stream",
session.planId());
Socket outgoingSocket = session.createConnection();
-outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION);
-outgoing.sendInitMessage(outgoingSocket, false);
+outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION, true);
}
/**
@@ -159,13 +157,15 @@ public class ConnectionHandler
protected int protocolVersion;
protected Socket socket;
+private final boolean isOutgoingHandler;
private final AtomicReference> closeFuture = new
AtomicReference<>();
private IncomingStreamingConnection incomingConnection;
-protected MessageHandler(StreamSession session)
+protected MessageHandler(StreamSession session, boolean
isOutgoingHandler)
{
this.session = session;
+this.isOutgoingHandler = isOutgoingHandler;
}
protected abstract String name();
@@ -187,14 +187,14 @@ public class ConnectionHandler
}
@SuppressWarnings("resource")
-public void sendInitMessage(Socket socket, boolean isForOutgoing)
throws IOException
+public void sendInitMessage() throws IOException
{
StreamInitMessage message = new StreamInitMessage(
FBUtilities.getBroadcastAddress(),
session.sessionIndex(),
session.planId(),
session.description(),
-isForOutgoing,
+!isOutgoingHandler,
session.keepSSTableLevel(),
session.isIncremental());
ByteBuffer messageBuf = message.createMessage(false,
protocolVersion);
@@ -203,16 +203,18 @@ public class ConnectionHandler
out.flush();
}
-public void start(IncomingStreamingConnection connection, int
protocolVersion)
+public void start(IncomingStreamingConnection connection, int
protocolVersion) throws IOException
{
this.incomingConnection = connection;
-start(connection.socket, protocolVersion);
+
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/e345d4e7 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/e345d4e7 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/e345d4e7 Branch: refs/heads/cassandra-3.11 Commit: e345d4e7bc83e09e3c273981b5c95e5fb00d1df5 Parents: be0deb9 815508f Author: Paulo Motta Authored: Wed Mar 1 20:33:49 2017 -0300 Committer: Paulo Motta Committed: Wed Mar 1 20:33:49 2017 -0300 -- --
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/815508f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/815508f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/815508f1
Branch: refs/heads/trunk
Commit: 815508f1c240bbbac8246be56e15b1d09c1301b8
Parents: 218743d 06feaef
Author: Paulo Motta
Authored: Wed Mar 1 20:33:16 2017 -0300
Committer: Paulo Motta
Committed: Wed Mar 1 20:33:16 2017 -0300
--
CHANGES.txt | 1 +
.../cassandra/streaming/ConnectionHandler.java | 26 +++-
2 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815508f1/CHANGES.txt
--
diff --cc CHANGES.txt
index c4622ab,ca1aa27..ec3d826
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,12 -1,8 +1,13 @@@
-2.2.10
+3.0.12
+ * Avoid race on receiver by starting streaming sender thread after sending
init message (CASSANDRA-12886)
+ * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
+ * Faster StreamingHistogram (CASSANDRA-13038)
+ * Legacy deserializer can create unexpected boundary range tombstones
(CASSANDRA-13237)
+ * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
+Merged from 2.2
* Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
* Coalescing strategy sleeps too much (CASSANDRA-13090)
- * Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
* Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
* Fix failing COPY TO STDOUT (CASSANDRA-12497)
* Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815508f1/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --cc src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index b83c089,fe551a8..aa1c615
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@@ -215,8 -213,10 +215,10 @@@ public class ConnectionHandle
{
this.socket = socket;
this.protocolVersion = protocolVersion;
+ if (initiator)
+ sendInitMessage();
-new Thread(this, name() + "-" + session.peer).start();
+new Thread(NamedThreadFactory.threadLocalDeallocator(this),
name() + "-" + session.peer).start();
}
public ListenableFuture close()
[03/10] cassandra git commit: Avoid race on receiver by starting streaming sender thread after sending init message
Avoid race on receiver by starting streaming sender thread after sending init
message
Patch by Paulo Motta; Reviewed by Yuki Morishita for CASSANDRA-12886
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/06feaefb
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/06feaefb
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/06feaefb
Branch: refs/heads/cassandra-3.11
Commit: 06feaefba50301734c490521d720c8a482f638e4
Parents: 9bbb449
Author: Paulo Motta
Authored: Wed Mar 1 20:25:32 2017 -0300
Committer: Paulo Motta
Committed: Wed Mar 1 20:30:30 2017 -0300
--
CHANGES.txt | 1 +
.../cassandra/streaming/ConnectionHandler.java | 26 +++-
2 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 404440a..ca1aa27 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.2.10
+ * Avoid race on receiver by starting streaming sender thread after sending
init message (CASSANDRA-12886)
* Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
* Coalescing strategy sleeps too much (CASSANDRA-13090)
* Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/06feaefb/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --git a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index d3d8ed2..fe551a8 100644
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@ -82,13 +82,11 @@ public class ConnectionHandler
{
logger.debug("[Stream #{}] Sending stream init for incoming stream",
session.planId());
Socket incomingSocket = session.createConnection();
-incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION);
-incoming.sendInitMessage(incomingSocket, true);
+incoming.start(incomingSocket, StreamMessage.CURRENT_VERSION, true);
logger.debug("[Stream #{}] Sending stream init for outgoing stream",
session.planId());
Socket outgoingSocket = session.createConnection();
-outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION);
-outgoing.sendInitMessage(outgoingSocket, false);
+outgoing.start(outgoingSocket, StreamMessage.CURRENT_VERSION, true);
}
/**
@@ -159,13 +157,15 @@ public class ConnectionHandler
protected int protocolVersion;
protected Socket socket;
+private final boolean isOutgoingHandler;
private final AtomicReference> closeFuture = new
AtomicReference<>();
private IncomingStreamingConnection incomingConnection;
-protected MessageHandler(StreamSession session)
+protected MessageHandler(StreamSession session, boolean
isOutgoingHandler)
{
this.session = session;
+this.isOutgoingHandler = isOutgoingHandler;
}
protected abstract String name();
@@ -187,14 +187,14 @@ public class ConnectionHandler
}
@SuppressWarnings("resource")
-public void sendInitMessage(Socket socket, boolean isForOutgoing)
throws IOException
+public void sendInitMessage() throws IOException
{
StreamInitMessage message = new StreamInitMessage(
FBUtilities.getBroadcastAddress(),
session.sessionIndex(),
session.planId(),
session.description(),
-isForOutgoing,
+!isOutgoingHandler,
session.keepSSTableLevel(),
session.isIncremental());
ByteBuffer messageBuf = message.createMessage(false,
protocolVersion);
@@ -203,16 +203,18 @@ public class ConnectionHandler
out.flush();
}
-public void start(IncomingStreamingConnection connection, int
protocolVersion)
+public void start(IncomingStreamingConnection connection, int
protocolVersion) throws IOException
{
this.incomingConnection = connection;
-start(connection.socket, protocolVersion);
+start(connection.socket, protocolVersion, false);
}
-public void start(Socket socket, int protocolVersion)
+public void start(Socket socket, int protocolVersion, boolean
initiator) throws IOExcept
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/815508f1
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/815508f1
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/815508f1
Branch: refs/heads/cassandra-3.0
Commit: 815508f1c240bbbac8246be56e15b1d09c1301b8
Parents: 218743d 06feaef
Author: Paulo Motta
Authored: Wed Mar 1 20:33:16 2017 -0300
Committer: Paulo Motta
Committed: Wed Mar 1 20:33:16 2017 -0300
--
CHANGES.txt | 1 +
.../cassandra/streaming/ConnectionHandler.java | 26 +++-
2 files changed, 15 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815508f1/CHANGES.txt
--
diff --cc CHANGES.txt
index c4622ab,ca1aa27..ec3d826
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,12 -1,8 +1,13 @@@
-2.2.10
+3.0.12
+ * Avoid race on receiver by starting streaming sender thread after sending
init message (CASSANDRA-12886)
+ * Update c.yaml doc for offheap memtables (CASSANDRA-13179)
+ * Faster StreamingHistogram (CASSANDRA-13038)
+ * Legacy deserializer can create unexpected boundary range tombstones
(CASSANDRA-13237)
+ * Remove unnecessary assertion from AntiCompactionTest (CASSANDRA-13070)
+ * Fix cqlsh COPY for dates before 1900 (CASSANDRA-13185)
+Merged from 2.2
* Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
* Coalescing strategy sleeps too much (CASSANDRA-13090)
- * Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
* Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
* Fix failing COPY TO STDOUT (CASSANDRA-12497)
* Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/815508f1/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
--
diff --cc src/java/org/apache/cassandra/streaming/ConnectionHandler.java
index b83c089,fe551a8..aa1c615
--- a/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
+++ b/src/java/org/apache/cassandra/streaming/ConnectionHandler.java
@@@ -215,8 -213,10 +215,10 @@@ public class ConnectionHandle
{
this.socket = socket;
this.protocolVersion = protocolVersion;
+ if (initiator)
+ sendInitMessage();
-new Thread(this, name() + "-" + session.peer).start();
+new Thread(NamedThreadFactory.threadLocalDeallocator(this),
name() + "-" + session.peer).start();
}
public ListenableFuture close()
[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891229#comment-15891229
]
Joel Knighton commented on CASSANDRA-13038:
---
Nit + 3.0 changes look good. If CI doesn't have any problems, +1.
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the
> fact the this table has a TWCS also gives us an additional hint that sampling
> deletion times would be fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891209#comment-15891209
]
Jeff Jirsa commented on CASSANDRA-13038:
Addressed your nit, also pushed related changes for 3.0 - care to glance at
those as well (CI should be starting in a second)
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the
> fact the this table has a TWCS also gives us an additional hint that sampling
> deletion times would be fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13060) NPE in StorageService.java while bootstrapping
[
https://issues.apache.org/jira/browse/CASSANDRA-13060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891200#comment-15891200
]
Aleksandr Ivanov commented on CASSANDRA-13060:
--
Any workaround exists for v3.0.10?
Currently it is blocking to bootstrap new nodes.
> NPE in StorageService.java while bootstrapping
> --
>
> Key: CASSANDRA-13060
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13060
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Jay Zhuang
>Assignee: Jay Zhuang
>Priority: Trivial
> Labels: lhf
> Fix For: 3.0.x
>
> Attachments: 13060-3.0.txt
>
>
> Lots of NPE happens when bootstrapping new node:
> {code}
> WARN [SharedPool-Worker-1] 2016-12-19 23:09:09,034
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]: {}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.service.StorageService.isRpcReady(StorageService.java:1829)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.service.StorageService.notifyUp(StorageService.java:1787)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.service.StorageService.onAlive(StorageService.java:2424)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at org.apache.cassandra.gms.Gossiper.realMarkAlive(Gossiper.java:999)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at org.apache.cassandra.gms.Gossiper$3.response(Gossiper.java:979)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.net.ResponseVerbHandler.doVerb(ResponseVerbHandler.java:53)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:67)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_111]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> ~[apache-cassandra-3.0.10.jar:3.0.10]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
> [apache-cassandra-3.0.10.jar:3.0.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.0.10.jar:3.0.10]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_111]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891175#comment-15891175
]
Joel Knighton commented on CASSANDRA-13038:
---
Thanks - on a first skim, both of those look good and fix the tests locally for
me. One minor nit - if removing the maxSpoolSize from {{equals}} on
{{StreamingHistogram}}, it seems we should remove it from {{hashCode}} as well
to respect the method contract.
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the
> fact the this table has a TWCS also gives us an additional hint that sampling
> deletion times would be fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13278) Update build.xml and build.properties.default maven repos
[
https://issues.apache.org/jira/browse/CASSANDRA-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-13278:
---
Resolution: Fixed
Fix Version/s: (was: 3.11.x)
(was: 4.x)
(was: 3.0.x)
(was: 2.2.x)
(was: 2.1.x)
4.0
3.11.0
3.0.12
2.2.10
2.1.18
Status: Resolved (was: Patch Available)
Thanks guys. Committing per conversation in IRC with [~mshuler] so I can
rebase other patches and get CI working again. Committed as
{{301f7c5b7bfbb73ecbfb65a6ac7b393a4d968e78}} and merged with patches provided
from 2.1 through trunk.
> Update build.xml and build.properties.default maven repos
> -
>
> Key: CASSANDRA-13278
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13278
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Michael Shuler
>Assignee: Robert Stupp
>Priority: Minor
> Labels: lhf
> Fix For: 2.1.18, 2.2.10, 3.0.12, 3.11.0, 4.0
>
>
> Only 2 of the 5 urls in build.properties.default are currently valid.
> java.net2, jclouds, and oauth urls all 404.
> {noformat}
> $ git grep remoteRepository
> build.properties.default:artifact.remoteRepository.central:
> http://repo1.maven.org/maven2
> build.properties.default:artifact.remoteRepository.java.net2:
> http://download.java.net/maven/2
> build.properties.default:artifact.remoteRepository.apache:
> https://repository.apache.org/content/repositories/releases
> build.properties.default:artifact.remoteRepository.jclouds:
> http://jclouds.googlecode.com/svn/repo
> build.properties.default:artifact.remoteRepository.oauth:
> http://oauth.googlecode.com/svn/code/maven
> build.xml: url="${artifact.remoteRepository.central}"/>
> build.xml: url="${artifact.remoteRepository.java.net2}"/>
> build.xml: url="${artifact.remoteRepository.apache}"/>
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[12/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/218743dc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/218743dc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/218743dc Branch: refs/heads/cassandra-3.0 Commit: 218743dc39a49bd31182c8e796d10781b6c236eb Parents: 873660c 9bbb449 Author: Jeff Jirsa Authored: Wed Mar 1 13:43:53 2017 -0800 Committer: Jeff Jirsa Committed: Wed Mar 1 13:44:55 2017 -0800 -- CHANGES.txt | 1 + build.properties.default | 3 --- build.xml| 6 ++ 3 files changed, 3 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/218743dc/CHANGES.txt -- diff --cc CHANGES.txt index 2289334,404440a..c4622ab --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -12,49 -7,10 +12,50 @@@ Merged from 2. * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222) * Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018) Merged from 2.1: + * Remove unused repositories (CASSANDRA-13278) * Log stacktrace of uncaught exceptions (CASSANDRA-13108) -2.2.9 +3.0.11 + * Use keyspace replication settings on system.size_estimates table (CASSANDRA-9639) + * Add vm.max_map_count StartupCheck (CASSANDRA-13008) + * Hint related logging should include the IP address of the destination in addition to + host ID (CASSANDRA-13205) + * Reloading logback.xml does not work (CASSANDRA-13173) + * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 (CASSANDRA-13109) + * Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 (CASSANDRA-13125) + * Fix UPDATE queries with empty IN restrictions (CASSANDRA-13152) + * Abort or retry on failed hints delivery (CASSANDRA-13124) + * Fix handling of partition with partition-level deletion plus + live rows in sstabledump (CASSANDRA-13177) + * Provide user workaround when system_schema.columns does not contain entries + for a table that's in system_schema.tables (CASSANDRA-13180) + * Dump threads when unit tests time out (CASSANDRA-13117) + * Better error when modifying function permissions without explicit keyspace (CASSANDRA-12925) + * Indexer is not correctly invoked when building indexes over sstables (CASSANDRA-13075) + * Read repair is not blocking repair to finish in foreground repair (CASSANDRA-13115) + * Stress daemon help is incorrect (CASSANDRA-12563) + * Remove ALTER TYPE support (CASSANDRA-12443) + * Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203) + * Set javac encoding to utf-8 (CASSANDRA-11077) + * Replace empty strings with null values if they cannot be converted (CASSANDRA-12794) + * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620) + * Add parent repair session id to anticompaction log message (CASSANDRA-12186) + * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905) + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829) + * Mark MVs as built after successful bootstrap (CASSANDRA-12984) + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME (CASSANDRA-13040) + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021) + * Thread local pools never cleaned up (CASSANDRA-13033) + * Set RPC_READY to false when draining or if a node is marked as shutdown (CASSANDRA-12781) + * Make sure sstables only get committed when it's safe to discard commit log records (CASSANDRA-12956) + * Reject default_time_to_live option when creating or altering MVs (CASSANDRA-12868) + * Nodetool should use a more sane max heap size (CASSANDRA-12739) + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651) + * AnticompactionRequestSerializer serializedSize is incorrect (CASSANDRA-12934) + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + * Reenable HeapPool (CASSANDRA-12900) +Merged from 2.2: * Fix negative mean latency metric (CASSANDRA-12876) * Use only one file pointer when creating commitlog segments (CASSANDRA-12539) * Fix speculative retry bugs (CASSANDRA-13009) http://git-wip-us.apache.org/repos/asf/cassandra/blob/218743dc/build.xml -- diff --cc build.xml index 69b6bdf,6dbdb9e..cffaa17 --- a/build.xml +++ b/build.xml @@@ -386,9 -387,15 +385,10 @@@ - - - ++ + + + @@@ -504,11 -5
[08/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9bbb4497
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9bbb4497
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9bbb4497
Branch: refs/heads/cassandra-3.0
Commit: 9bbb449736912562348f9f6565a17e6be1c1fba6
Parents: aa66c99 301f7c5
Author: Jeff Jirsa
Authored: Wed Mar 1 13:41:06 2017 -0800
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:43:37 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 5 -
3 files changed, 1 insertion(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/CHANGES.txt
--
diff --cc CHANGES.txt
index c27c2b1,111f3de..404440a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,46 -1,8 +1,47 @@@
-2.1.18
- * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+2.2.10
+ * Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
+ * Coalescing strategy sleeps too much (CASSANDRA-13090)
+ * Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
+ * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
+ * Fix failing COPY TO STDOUT (CASSANDRA-12497)
+ * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
+ * Exceptions encountered calling getSeeds() breaks OTC thread
(CASSANDRA-13018)
+Merged from 2.1:
+ * Remove unused repositories (CASSANDRA-13278)
+ * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
-2.1.17
+2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
+ * Use only one file pointer when creating commitlog segments
(CASSANDRA-12539)
+ * Fix speculative retry bugs (CASSANDRA-13009)
+ * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981)
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
+ * Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
+ * cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
+ * Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
+ * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
+ * Test bind parameters and unset parameters in InsertUpdateIfConditionTest
(CASSANDRA-12980)
+ * Do not specify local address on outgoing connection when
listen_on_broadcast_address is set (CASSANDRA-12673)
+ * Use saved tokens when setting local tokens on StorageService.joinRing
(CASSANDRA-12935)
+ * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
+ * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
+ * Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
+ * cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
+ * Fix Util.spinAssertEquals (CASSANDRA-12283)
+ * Fix potential NPE for compactionstats (CASSANDRA-12462)
+ * Prepare legacy authenticate statement if credentials table initialised
after node startup (CASSANDRA-12813)
+ * Change cassandra.wait_for_tracing_events_timeout_secs default to 0
(CASSANDRA-12754)
+ * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
+ * Limit colUpdateTimeDelta histogram updates to reasonable deltas
(CASSANDRA-7)
+ * Fix leak errors and execution rejected exceptions when draining
(CASSANDRA-12457)
+ * Fix merkle tree depth calculation (CASSANDRA-12580)
+ * Make Collections deserialization more robust (CASSANDRA-12618)
+ * Better handle invalid system roles table (CASSANDRA-12700)
+ * Split consistent range movement flag correction (CASSANDRA-12786)
+ * CompactionTasks now correctly drops sstables out of compaction when not
enough disk space is available (CASSANDRA-12979)
+Merged from 2.1:
* Use portable stderr for java error in startup (CASSANDRA-13211)
* Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
* Coalescing strategy can enter infinite loop (CASSANDRA-13159)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/build.xml
--
diff --cc build.xml
index d815ede,da42975..6dbdb9e
--- a/build.xml
+++ b/build.xml
@@@ -654,17 -571,12 +653,15 @@@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
-
+
+
+
-
-
[09/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9bbb4497
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9bbb4497
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9bbb4497
Branch: refs/heads/cassandra-3.11
Commit: 9bbb449736912562348f9f6565a17e6be1c1fba6
Parents: aa66c99 301f7c5
Author: Jeff Jirsa
Authored: Wed Mar 1 13:41:06 2017 -0800
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:43:37 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 5 -
3 files changed, 1 insertion(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/CHANGES.txt
--
diff --cc CHANGES.txt
index c27c2b1,111f3de..404440a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,46 -1,8 +1,47 @@@
-2.1.18
- * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+2.2.10
+ * Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
+ * Coalescing strategy sleeps too much (CASSANDRA-13090)
+ * Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
+ * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
+ * Fix failing COPY TO STDOUT (CASSANDRA-12497)
+ * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
+ * Exceptions encountered calling getSeeds() breaks OTC thread
(CASSANDRA-13018)
+Merged from 2.1:
+ * Remove unused repositories (CASSANDRA-13278)
+ * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
-2.1.17
+2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
+ * Use only one file pointer when creating commitlog segments
(CASSANDRA-12539)
+ * Fix speculative retry bugs (CASSANDRA-13009)
+ * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981)
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
+ * Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
+ * cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
+ * Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
+ * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
+ * Test bind parameters and unset parameters in InsertUpdateIfConditionTest
(CASSANDRA-12980)
+ * Do not specify local address on outgoing connection when
listen_on_broadcast_address is set (CASSANDRA-12673)
+ * Use saved tokens when setting local tokens on StorageService.joinRing
(CASSANDRA-12935)
+ * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
+ * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
+ * Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
+ * cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
+ * Fix Util.spinAssertEquals (CASSANDRA-12283)
+ * Fix potential NPE for compactionstats (CASSANDRA-12462)
+ * Prepare legacy authenticate statement if credentials table initialised
after node startup (CASSANDRA-12813)
+ * Change cassandra.wait_for_tracing_events_timeout_secs default to 0
(CASSANDRA-12754)
+ * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
+ * Limit colUpdateTimeDelta histogram updates to reasonable deltas
(CASSANDRA-7)
+ * Fix leak errors and execution rejected exceptions when draining
(CASSANDRA-12457)
+ * Fix merkle tree depth calculation (CASSANDRA-12580)
+ * Make Collections deserialization more robust (CASSANDRA-12618)
+ * Better handle invalid system roles table (CASSANDRA-12700)
+ * Split consistent range movement flag correction (CASSANDRA-12786)
+ * CompactionTasks now correctly drops sstables out of compaction when not
enough disk space is available (CASSANDRA-12979)
+Merged from 2.1:
* Use portable stderr for java error in startup (CASSANDRA-13211)
* Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
* Coalescing strategy can enter infinite loop (CASSANDRA-13159)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/build.xml
--
diff --cc build.xml
index d815ede,da42975..6dbdb9e
--- a/build.xml
+++ b/build.xml
@@@ -654,17 -571,12 +653,15 @@@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
-
+
+
+
-
-
[13/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be0deb90 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be0deb90 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be0deb90 Branch: refs/heads/cassandra-3.11 Commit: be0deb90e45d66a771fce4652a1548256ee5463e Parents: b26db7a 218743d Author: Jeff Jirsa Authored: Wed Mar 1 13:45:05 2017 -0800 Committer: Jeff Jirsa Committed: Wed Mar 1 13:45:59 2017 -0800 -- CHANGES.txt | 1 + build.properties.default | 3 --- build.xml| 6 ++ 3 files changed, 3 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/be0deb90/CHANGES.txt -- diff --cc CHANGES.txt index 497f5c4,c4622ab..cf15d3a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -26,136 -29,6 +26,137 @@@ Merged from 3.0 live rows in sstabledump (CASSANDRA-13177) * Provide user workaround when system_schema.columns does not contain entries for a table that's in system_schema.tables (CASSANDRA-13180) +Merged from 2.2: + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202) + * Fix failing COPY TO STDOUT (CASSANDRA-12497) + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222) + * Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018) + * Fix negative mean latency metric (CASSANDRA-12876) + * Use only one file pointer when creating commitlog segments (CASSANDRA-12539) +Merged from 2.1: ++ * Remove unused repositories (CASSANDRA-13278) + * Log stacktrace of uncaught exceptions (CASSANDRA-13108) + * Use portable stderr for java error in startup (CASSANDRA-13211) + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204) + * Coalescing strategy can enter infinite loop (CASSANDRA-13159) + +3.10 + * Fix secondary index queries regression (CASSANDRA-13013) + * Add duration type to the protocol V5 (CASSANDRA-12850) + * Fix duration type validation (CASSANDRA-13143) + * Fix flaky GcCompactionTest (CASSANDRA-12664) + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058) + * Fixed query monitoring for range queries (CASSANDRA-13050) + * Remove outboundBindAny configuration property (CASSANDRA-12673) + * Use correct bounds for all-data range when filtering (CASSANDRA-12666) + * Remove timing window in test case (CASSANDRA-12875) + * Resolve unit testing without JCE security libraries installed (CASSANDRA-12945) + * Fix inconsistencies in cassandra-stress load balancing policy (CASSANDRA-12919) + * Fix validation of non-frozen UDT cells (CASSANDRA-12916) + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903) + * Fix Murmur3PartitionerTest (CASSANDRA-12858) + * Move cqlsh syntax rules into separate module and allow easier customization (CASSANDRA-12897) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Fix cassandra-stress truncate option (CASSANDRA-12695) + * Fix crossNode value when receiving messages (CASSANDRA-12791) + * Don't load MX4J beans twice (CASSANDRA-12869) + * Extend native protocol request flags, add versions to SUPPORTED, and introduce ProtocolVersion enum (CASSANDRA-12838) + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836) + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845) + * Properly format IPv6 addresses when logging JMX service URL (CASSANDRA-12454) + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777) + * Use non-token restrictions for bounds when token restrictions are overridden (CASSANDRA-12419) + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803) + * Use different build directories for Eclipse and Ant (CASSANDRA-12466) + * Avoid potential AttributeError in cqlsh due to no table metadata (CASSANDRA-12815) + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster (CASSANDRA-12812) + * Upgrade commons-codec to 1.9 (CASSANDRA-12790) + * Make the fanout size for LeveledCompactionStrategy to be configurable (CASSANDRA-11550) + * Add duration data type (CASSANDRA-11873) + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784) + * Improve sum aggregate functions (CASSANDRA-12417) + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876 (CASSANDRA-12761) + * cqlsh fails to format collections when using aliases (CASSANDRA-11534) + * Check for hash conflicts in prepared statements (CASSANDRA-12733) + * Exit query parsing upon first error (CASSANDRA-12598) + * Fix cassandra-stress to use single seed in UUID generation (CASSANDRA-12729) + * CQLSSTableWriter does not allow Update s
[11/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/218743dc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/218743dc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/218743dc Branch: refs/heads/cassandra-3.11 Commit: 218743dc39a49bd31182c8e796d10781b6c236eb Parents: 873660c 9bbb449 Author: Jeff Jirsa Authored: Wed Mar 1 13:43:53 2017 -0800 Committer: Jeff Jirsa Committed: Wed Mar 1 13:44:55 2017 -0800 -- CHANGES.txt | 1 + build.properties.default | 3 --- build.xml| 6 ++ 3 files changed, 3 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/218743dc/CHANGES.txt -- diff --cc CHANGES.txt index 2289334,404440a..c4622ab --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -12,49 -7,10 +12,50 @@@ Merged from 2. * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222) * Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018) Merged from 2.1: + * Remove unused repositories (CASSANDRA-13278) * Log stacktrace of uncaught exceptions (CASSANDRA-13108) -2.2.9 +3.0.11 + * Use keyspace replication settings on system.size_estimates table (CASSANDRA-9639) + * Add vm.max_map_count StartupCheck (CASSANDRA-13008) + * Hint related logging should include the IP address of the destination in addition to + host ID (CASSANDRA-13205) + * Reloading logback.xml does not work (CASSANDRA-13173) + * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 (CASSANDRA-13109) + * Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 (CASSANDRA-13125) + * Fix UPDATE queries with empty IN restrictions (CASSANDRA-13152) + * Abort or retry on failed hints delivery (CASSANDRA-13124) + * Fix handling of partition with partition-level deletion plus + live rows in sstabledump (CASSANDRA-13177) + * Provide user workaround when system_schema.columns does not contain entries + for a table that's in system_schema.tables (CASSANDRA-13180) + * Dump threads when unit tests time out (CASSANDRA-13117) + * Better error when modifying function permissions without explicit keyspace (CASSANDRA-12925) + * Indexer is not correctly invoked when building indexes over sstables (CASSANDRA-13075) + * Read repair is not blocking repair to finish in foreground repair (CASSANDRA-13115) + * Stress daemon help is incorrect (CASSANDRA-12563) + * Remove ALTER TYPE support (CASSANDRA-12443) + * Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203) + * Set javac encoding to utf-8 (CASSANDRA-11077) + * Replace empty strings with null values if they cannot be converted (CASSANDRA-12794) + * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620) + * Add parent repair session id to anticompaction log message (CASSANDRA-12186) + * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905) + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829) + * Mark MVs as built after successful bootstrap (CASSANDRA-12984) + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME (CASSANDRA-13040) + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021) + * Thread local pools never cleaned up (CASSANDRA-13033) + * Set RPC_READY to false when draining or if a node is marked as shutdown (CASSANDRA-12781) + * Make sure sstables only get committed when it's safe to discard commit log records (CASSANDRA-12956) + * Reject default_time_to_live option when creating or altering MVs (CASSANDRA-12868) + * Nodetool should use a more sane max heap size (CASSANDRA-12739) + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651) + * AnticompactionRequestSerializer serializedSize is incorrect (CASSANDRA-12934) + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + * Reenable HeapPool (CASSANDRA-12900) +Merged from 2.2: * Fix negative mean latency metric (CASSANDRA-12876) * Use only one file pointer when creating commitlog segments (CASSANDRA-12539) * Fix speculative retry bugs (CASSANDRA-13009) http://git-wip-us.apache.org/repos/asf/cassandra/blob/218743dc/build.xml -- diff --cc build.xml index 69b6bdf,6dbdb9e..cffaa17 --- a/build.xml +++ b/build.xml @@@ -386,9 -387,15 +385,10 @@@ - - - ++ + + + @@@ -504,11 -
[14/15] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.11
Merge branch 'cassandra-3.0' into cassandra-3.11 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/be0deb90 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/be0deb90 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/be0deb90 Branch: refs/heads/trunk Commit: be0deb90e45d66a771fce4652a1548256ee5463e Parents: b26db7a 218743d Author: Jeff Jirsa Authored: Wed Mar 1 13:45:05 2017 -0800 Committer: Jeff Jirsa Committed: Wed Mar 1 13:45:59 2017 -0800 -- CHANGES.txt | 1 + build.properties.default | 3 --- build.xml| 6 ++ 3 files changed, 3 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/be0deb90/CHANGES.txt -- diff --cc CHANGES.txt index 497f5c4,c4622ab..cf15d3a --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -26,136 -29,6 +26,137 @@@ Merged from 3.0 live rows in sstabledump (CASSANDRA-13177) * Provide user workaround when system_schema.columns does not contain entries for a table that's in system_schema.tables (CASSANDRA-13180) +Merged from 2.2: + * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202) + * Fix failing COPY TO STDOUT (CASSANDRA-12497) + * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222) + * Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018) + * Fix negative mean latency metric (CASSANDRA-12876) + * Use only one file pointer when creating commitlog segments (CASSANDRA-12539) +Merged from 2.1: ++ * Remove unused repositories (CASSANDRA-13278) + * Log stacktrace of uncaught exceptions (CASSANDRA-13108) + * Use portable stderr for java error in startup (CASSANDRA-13211) + * Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204) + * Coalescing strategy can enter infinite loop (CASSANDRA-13159) + +3.10 + * Fix secondary index queries regression (CASSANDRA-13013) + * Add duration type to the protocol V5 (CASSANDRA-12850) + * Fix duration type validation (CASSANDRA-13143) + * Fix flaky GcCompactionTest (CASSANDRA-12664) + * Fix TestHintedHandoff.hintedhandoff_decom_test (CASSANDRA-13058) + * Fixed query monitoring for range queries (CASSANDRA-13050) + * Remove outboundBindAny configuration property (CASSANDRA-12673) + * Use correct bounds for all-data range when filtering (CASSANDRA-12666) + * Remove timing window in test case (CASSANDRA-12875) + * Resolve unit testing without JCE security libraries installed (CASSANDRA-12945) + * Fix inconsistencies in cassandra-stress load balancing policy (CASSANDRA-12919) + * Fix validation of non-frozen UDT cells (CASSANDRA-12916) + * Don't shut down socket input/output on StreamSession (CASSANDRA-12903) + * Fix Murmur3PartitionerTest (CASSANDRA-12858) + * Move cqlsh syntax rules into separate module and allow easier customization (CASSANDRA-12897) + * Fix CommitLogSegmentManagerTest (CASSANDRA-12283) + * Fix cassandra-stress truncate option (CASSANDRA-12695) + * Fix crossNode value when receiving messages (CASSANDRA-12791) + * Don't load MX4J beans twice (CASSANDRA-12869) + * Extend native protocol request flags, add versions to SUPPORTED, and introduce ProtocolVersion enum (CASSANDRA-12838) + * Set JOINING mode when running pre-join tasks (CASSANDRA-12836) + * remove net.mintern.primitive library due to license issue (CASSANDRA-12845) + * Properly format IPv6 addresses when logging JMX service URL (CASSANDRA-12454) + * Optimize the vnode allocation for single replica per DC (CASSANDRA-12777) + * Use non-token restrictions for bounds when token restrictions are overridden (CASSANDRA-12419) + * Fix CQLSH auto completion for PER PARTITION LIMIT (CASSANDRA-12803) + * Use different build directories for Eclipse and Ant (CASSANDRA-12466) + * Avoid potential AttributeError in cqlsh due to no table metadata (CASSANDRA-12815) + * Fix RandomReplicationAwareTokenAllocatorTest.testExistingCluster (CASSANDRA-12812) + * Upgrade commons-codec to 1.9 (CASSANDRA-12790) + * Make the fanout size for LeveledCompactionStrategy to be configurable (CASSANDRA-11550) + * Add duration data type (CASSANDRA-11873) + * Fix timeout in ReplicationAwareTokenAllocatorTest (CASSANDRA-12784) + * Improve sum aggregate functions (CASSANDRA-12417) + * Make cassandra.yaml docs for batch_size_*_threshold_in_kb reflect changes in CASSANDRA-10876 (CASSANDRA-12761) + * cqlsh fails to format collections when using aliases (CASSANDRA-11534) + * Check for hash conflicts in prepared statements (CASSANDRA-12733) + * Exit query parsing upon first error (CASSANDRA-12598) + * Fix cassandra-stress to use single seed in UUID generation (CASSANDRA-12729) + * CQLSSTableWriter does not allow Update statement
[03/15] cassandra git commit: Remove unused repositories
Remove unused repositories
Patch by Robert Stupp; Reviewed by Michael Shuler for CASSANDRA-13278
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/301f7c5b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/301f7c5b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/301f7c5b
Branch: refs/heads/trunk
Commit: 301f7c5b7bfbb73ecbfb65a6ac7b393a4d968e78
Parents: 3c2f876
Author: Robert Stupp
Authored: Wed Mar 1 13:24:22 2017 +0100
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:40:04 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 4
3 files changed, 1 insertion(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f3de538..111f3de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.1.18
* Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+ * Remove unused repositories (CASSANDRA-13278)
2.1.17
* Use portable stderr for java error in startup (CASSANDRA-13211)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.properties.default
--
diff --git a/build.properties.default b/build.properties.default
index ff6aea0..45d65d9 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -1,7 +1,4 @@
# Maven2 Repository Locations (you can override these in "build.properties" to
point to a local proxy, e.g. Nexus)
artifact.remoteRepository.central: http://repo1.maven.org/maven2
-artifact.remoteRepository.java.net2: http://download.java.net/maven/2
artifact.remoteRepository.apache:
https://repository.apache.org/content/repositories/releases
-artifact.remoteRepository.jclouds: http://jclouds.googlecode.com/svn/repo
-artifact.remoteRepository.oauth:
http://oauth.googlecode.com/svn/code/maven
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.xml
--
diff --git a/build.xml b/build.xml
index 94b1ed5..da42975 100644
--- a/build.xml
+++ b/build.xml
@@ -288,7 +288,6 @@
-
@@ -572,7 +571,6 @@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
@@ -597,7 +595,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
@@ -617,7 +614,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
[07/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9bbb4497
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9bbb4497
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9bbb4497
Branch: refs/heads/cassandra-2.2
Commit: 9bbb449736912562348f9f6565a17e6be1c1fba6
Parents: aa66c99 301f7c5
Author: Jeff Jirsa
Authored: Wed Mar 1 13:41:06 2017 -0800
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:43:37 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 5 -
3 files changed, 1 insertion(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/CHANGES.txt
--
diff --cc CHANGES.txt
index c27c2b1,111f3de..404440a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,46 -1,8 +1,47 @@@
-2.1.18
- * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+2.2.10
+ * Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
+ * Coalescing strategy sleeps too much (CASSANDRA-13090)
+ * Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
+ * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
+ * Fix failing COPY TO STDOUT (CASSANDRA-12497)
+ * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
+ * Exceptions encountered calling getSeeds() breaks OTC thread
(CASSANDRA-13018)
+Merged from 2.1:
+ * Remove unused repositories (CASSANDRA-13278)
+ * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
-2.1.17
+2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
+ * Use only one file pointer when creating commitlog segments
(CASSANDRA-12539)
+ * Fix speculative retry bugs (CASSANDRA-13009)
+ * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981)
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
+ * Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
+ * cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
+ * Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
+ * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
+ * Test bind parameters and unset parameters in InsertUpdateIfConditionTest
(CASSANDRA-12980)
+ * Do not specify local address on outgoing connection when
listen_on_broadcast_address is set (CASSANDRA-12673)
+ * Use saved tokens when setting local tokens on StorageService.joinRing
(CASSANDRA-12935)
+ * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
+ * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
+ * Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
+ * cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
+ * Fix Util.spinAssertEquals (CASSANDRA-12283)
+ * Fix potential NPE for compactionstats (CASSANDRA-12462)
+ * Prepare legacy authenticate statement if credentials table initialised
after node startup (CASSANDRA-12813)
+ * Change cassandra.wait_for_tracing_events_timeout_secs default to 0
(CASSANDRA-12754)
+ * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
+ * Limit colUpdateTimeDelta histogram updates to reasonable deltas
(CASSANDRA-7)
+ * Fix leak errors and execution rejected exceptions when draining
(CASSANDRA-12457)
+ * Fix merkle tree depth calculation (CASSANDRA-12580)
+ * Make Collections deserialization more robust (CASSANDRA-12618)
+ * Better handle invalid system roles table (CASSANDRA-12700)
+ * Split consistent range movement flag correction (CASSANDRA-12786)
+ * CompactionTasks now correctly drops sstables out of compaction when not
enough disk space is available (CASSANDRA-12979)
+Merged from 2.1:
* Use portable stderr for java error in startup (CASSANDRA-13211)
* Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
* Coalescing strategy can enter infinite loop (CASSANDRA-13159)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/build.xml
--
diff --cc build.xml
index d815ede,da42975..6dbdb9e
--- a/build.xml
+++ b/build.xml
@@@ -654,17 -571,12 +653,15 @@@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
-
+
+
+
-
-
[15/15] cassandra git commit: Merge branch 'cassandra-3.11' into trunk
Merge branch 'cassandra-3.11' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/21b4cae6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/21b4cae6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/21b4cae6 Branch: refs/heads/trunk Commit: 21b4cae615be53d16c5416b4202838658c3d72e2 Parents: ced54f8 be0deb9 Author: Jeff Jirsa Authored: Wed Mar 1 13:46:29 2017 -0800 Committer: Jeff Jirsa Committed: Wed Mar 1 13:46:58 2017 -0800 -- CHANGES.txt | 1 + build.properties.default | 3 --- build.xml| 6 ++ 3 files changed, 3 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/21b4cae6/CHANGES.txt -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/21b4cae6/build.xml --
[01/15] cassandra git commit: Remove unused repositories
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.1 3c2f87610 -> 301f7c5b7
refs/heads/cassandra-2.2 aa66c999a -> 9bbb44973
refs/heads/cassandra-3.0 873660cf5 -> 218743dc3
refs/heads/cassandra-3.11 b26db7a37 -> be0deb90e
refs/heads/trunk ced54f87b -> 21b4cae61
Remove unused repositories
Patch by Robert Stupp; Reviewed by Michael Shuler for CASSANDRA-13278
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/301f7c5b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/301f7c5b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/301f7c5b
Branch: refs/heads/cassandra-2.1
Commit: 301f7c5b7bfbb73ecbfb65a6ac7b393a4d968e78
Parents: 3c2f876
Author: Robert Stupp
Authored: Wed Mar 1 13:24:22 2017 +0100
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:40:04 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 4
3 files changed, 1 insertion(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f3de538..111f3de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.1.18
* Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+ * Remove unused repositories (CASSANDRA-13278)
2.1.17
* Use portable stderr for java error in startup (CASSANDRA-13211)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.properties.default
--
diff --git a/build.properties.default b/build.properties.default
index ff6aea0..45d65d9 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -1,7 +1,4 @@
# Maven2 Repository Locations (you can override these in "build.properties" to
point to a local proxy, e.g. Nexus)
artifact.remoteRepository.central: http://repo1.maven.org/maven2
-artifact.remoteRepository.java.net2: http://download.java.net/maven/2
artifact.remoteRepository.apache:
https://repository.apache.org/content/repositories/releases
-artifact.remoteRepository.jclouds: http://jclouds.googlecode.com/svn/repo
-artifact.remoteRepository.oauth:
http://oauth.googlecode.com/svn/code/maven
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.xml
--
diff --git a/build.xml b/build.xml
index 94b1ed5..da42975 100644
--- a/build.xml
+++ b/build.xml
@@ -288,7 +288,6 @@
-
@@ -572,7 +571,6 @@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
@@ -597,7 +595,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
@@ -617,7 +614,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
[05/15] cassandra git commit: Remove unused repositories
Remove unused repositories
Patch by Robert Stupp; Reviewed by Michael Shuler for CASSANDRA-13278
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/301f7c5b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/301f7c5b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/301f7c5b
Branch: refs/heads/cassandra-3.11
Commit: 301f7c5b7bfbb73ecbfb65a6ac7b393a4d968e78
Parents: 3c2f876
Author: Robert Stupp
Authored: Wed Mar 1 13:24:22 2017 +0100
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:40:04 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 4
3 files changed, 1 insertion(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f3de538..111f3de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.1.18
* Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+ * Remove unused repositories (CASSANDRA-13278)
2.1.17
* Use portable stderr for java error in startup (CASSANDRA-13211)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.properties.default
--
diff --git a/build.properties.default b/build.properties.default
index ff6aea0..45d65d9 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -1,7 +1,4 @@
# Maven2 Repository Locations (you can override these in "build.properties" to
point to a local proxy, e.g. Nexus)
artifact.remoteRepository.central: http://repo1.maven.org/maven2
-artifact.remoteRepository.java.net2: http://download.java.net/maven/2
artifact.remoteRepository.apache:
https://repository.apache.org/content/repositories/releases
-artifact.remoteRepository.jclouds: http://jclouds.googlecode.com/svn/repo
-artifact.remoteRepository.oauth:
http://oauth.googlecode.com/svn/code/maven
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.xml
--
diff --git a/build.xml b/build.xml
index 94b1ed5..da42975 100644
--- a/build.xml
+++ b/build.xml
@@ -288,7 +288,6 @@
-
@@ -572,7 +571,6 @@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
@@ -597,7 +595,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
@@ -617,7 +614,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
[04/15] cassandra git commit: Remove unused repositories
Remove unused repositories
Patch by Robert Stupp; Reviewed by Michael Shuler for CASSANDRA-13278
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/301f7c5b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/301f7c5b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/301f7c5b
Branch: refs/heads/cassandra-3.0
Commit: 301f7c5b7bfbb73ecbfb65a6ac7b393a4d968e78
Parents: 3c2f876
Author: Robert Stupp
Authored: Wed Mar 1 13:24:22 2017 +0100
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:40:04 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 4
3 files changed, 1 insertion(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f3de538..111f3de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.1.18
* Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+ * Remove unused repositories (CASSANDRA-13278)
2.1.17
* Use portable stderr for java error in startup (CASSANDRA-13211)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.properties.default
--
diff --git a/build.properties.default b/build.properties.default
index ff6aea0..45d65d9 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -1,7 +1,4 @@
# Maven2 Repository Locations (you can override these in "build.properties" to
point to a local proxy, e.g. Nexus)
artifact.remoteRepository.central: http://repo1.maven.org/maven2
-artifact.remoteRepository.java.net2: http://download.java.net/maven/2
artifact.remoteRepository.apache:
https://repository.apache.org/content/repositories/releases
-artifact.remoteRepository.jclouds: http://jclouds.googlecode.com/svn/repo
-artifact.remoteRepository.oauth:
http://oauth.googlecode.com/svn/code/maven
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.xml
--
diff --git a/build.xml b/build.xml
index 94b1ed5..da42975 100644
--- a/build.xml
+++ b/build.xml
@@ -288,7 +288,6 @@
-
@@ -572,7 +571,6 @@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
@@ -597,7 +595,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
@@ -617,7 +614,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
[10/15] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/218743dc Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/218743dc Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/218743dc Branch: refs/heads/trunk Commit: 218743dc39a49bd31182c8e796d10781b6c236eb Parents: 873660c 9bbb449 Author: Jeff Jirsa Authored: Wed Mar 1 13:43:53 2017 -0800 Committer: Jeff Jirsa Committed: Wed Mar 1 13:44:55 2017 -0800 -- CHANGES.txt | 1 + build.properties.default | 3 --- build.xml| 6 ++ 3 files changed, 3 insertions(+), 7 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/218743dc/CHANGES.txt -- diff --cc CHANGES.txt index 2289334,404440a..c4622ab --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -12,49 -7,10 +12,50 @@@ Merged from 2. * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222) * Exceptions encountered calling getSeeds() breaks OTC thread (CASSANDRA-13018) Merged from 2.1: + * Remove unused repositories (CASSANDRA-13278) * Log stacktrace of uncaught exceptions (CASSANDRA-13108) -2.2.9 +3.0.11 + * Use keyspace replication settings on system.size_estimates table (CASSANDRA-9639) + * Add vm.max_map_count StartupCheck (CASSANDRA-13008) + * Hint related logging should include the IP address of the destination in addition to + host ID (CASSANDRA-13205) + * Reloading logback.xml does not work (CASSANDRA-13173) + * Lightweight transactions temporarily fail after upgrade from 2.1 to 3.0 (CASSANDRA-13109) + * Duplicate rows after upgrading from 2.1.16 to 3.0.10/3.9 (CASSANDRA-13125) + * Fix UPDATE queries with empty IN restrictions (CASSANDRA-13152) + * Abort or retry on failed hints delivery (CASSANDRA-13124) + * Fix handling of partition with partition-level deletion plus + live rows in sstabledump (CASSANDRA-13177) + * Provide user workaround when system_schema.columns does not contain entries + for a table that's in system_schema.tables (CASSANDRA-13180) + * Dump threads when unit tests time out (CASSANDRA-13117) + * Better error when modifying function permissions without explicit keyspace (CASSANDRA-12925) + * Indexer is not correctly invoked when building indexes over sstables (CASSANDRA-13075) + * Read repair is not blocking repair to finish in foreground repair (CASSANDRA-13115) + * Stress daemon help is incorrect (CASSANDRA-12563) + * Remove ALTER TYPE support (CASSANDRA-12443) + * Fix assertion for certain legacy range tombstone pattern (CASSANDRA-12203) + * Set javac encoding to utf-8 (CASSANDRA-11077) + * Replace empty strings with null values if they cannot be converted (CASSANDRA-12794) + * Fixed flacky SSTableRewriterTest: check file counts before calling validateCFS (CASSANDRA-12348) + * Fix deserialization of 2.x DeletedCells (CASSANDRA-12620) + * Add parent repair session id to anticompaction log message (CASSANDRA-12186) + * Improve contention handling on failure to acquire MV lock for streaming and hints (CASSANDRA-12905) + * Fix DELETE and UPDATE queries with empty IN restrictions (CASSANDRA-12829) + * Mark MVs as built after successful bootstrap (CASSANDRA-12984) + * Estimated TS drop-time histogram updated with Cell.NO_DELETION_TIME (CASSANDRA-13040) + * Nodetool compactionstats fails with NullPointerException (CASSANDRA-13021) + * Thread local pools never cleaned up (CASSANDRA-13033) + * Set RPC_READY to false when draining or if a node is marked as shutdown (CASSANDRA-12781) + * Make sure sstables only get committed when it's safe to discard commit log records (CASSANDRA-12956) + * Reject default_time_to_live option when creating or altering MVs (CASSANDRA-12868) + * Nodetool should use a more sane max heap size (CASSANDRA-12739) + * LocalToken ensures token values are cloned on heap (CASSANDRA-12651) + * AnticompactionRequestSerializer serializedSize is incorrect (CASSANDRA-12934) + * Prevent reloading of logback.xml from UDF sandbox (CASSANDRA-12535) + * Reenable HeapPool (CASSANDRA-12900) +Merged from 2.2: * Fix negative mean latency metric (CASSANDRA-12876) * Use only one file pointer when creating commitlog segments (CASSANDRA-12539) * Fix speculative retry bugs (CASSANDRA-13009) http://git-wip-us.apache.org/repos/asf/cassandra/blob/218743dc/build.xml -- diff --cc build.xml index 69b6bdf,6dbdb9e..cffaa17 --- a/build.xml +++ b/build.xml @@@ -386,9 -387,15 +385,10 @@@ - - - ++ + + + @@@ -504,11 -525,12 +5
[06/15] cassandra git commit: Merge branch 'cassandra-2.1' into cassandra-2.2
Merge branch 'cassandra-2.1' into cassandra-2.2
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9bbb4497
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9bbb4497
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9bbb4497
Branch: refs/heads/trunk
Commit: 9bbb449736912562348f9f6565a17e6be1c1fba6
Parents: aa66c99 301f7c5
Author: Jeff Jirsa
Authored: Wed Mar 1 13:41:06 2017 -0800
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:43:37 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 5 -
3 files changed, 1 insertion(+), 8 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/CHANGES.txt
--
diff --cc CHANGES.txt
index c27c2b1,111f3de..404440a
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,46 -1,8 +1,47 @@@
-2.1.18
- * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+2.2.10
+ * Fix "multiple versions of ant detected..." when running ant test
(CASSANDRA-13232)
+ * Coalescing strategy sleeps too much (CASSANDRA-13090)
+ * Make sure compaction stats are updated when compaction is interrupted
(Backport from 3.0, CASSANDRA-12100)
+ * Fix flaky LongLeveledCompactionStrategyTest (CASSANDRA-12202)
+ * Fix failing COPY TO STDOUT (CASSANDRA-12497)
+ * Fix ColumnCounter::countAll behaviour for reverse queries (CASSANDRA-13222)
+ * Exceptions encountered calling getSeeds() breaks OTC thread
(CASSANDRA-13018)
+Merged from 2.1:
+ * Remove unused repositories (CASSANDRA-13278)
+ * Log stacktrace of uncaught exceptions (CASSANDRA-13108)
-2.1.17
+2.2.9
+ * Fix negative mean latency metric (CASSANDRA-12876)
+ * Use only one file pointer when creating commitlog segments
(CASSANDRA-12539)
+ * Fix speculative retry bugs (CASSANDRA-13009)
+ * Fix handling of nulls and unsets in IN conditions (CASSANDRA-12981)
+ * Remove support for non-JavaScript UDFs (CASSANDRA-12883)
+ * Fix DynamicEndpointSnitch noop in multi-datacenter situations
(CASSANDRA-13074)
+ * cqlsh copy-from: encode column names to avoid primary key parsing errors
(CASSANDRA-12909)
+ * Temporarily fix bug that creates commit log when running offline tools
(CASSANDRA-8616)
+ * Reduce granuality of OpOrder.Group during index build (CASSANDRA-12796)
+ * Test bind parameters and unset parameters in InsertUpdateIfConditionTest
(CASSANDRA-12980)
+ * Do not specify local address on outgoing connection when
listen_on_broadcast_address is set (CASSANDRA-12673)
+ * Use saved tokens when setting local tokens on StorageService.joinRing
(CASSANDRA-12935)
+ * cqlsh: fix DESC TYPES errors (CASSANDRA-12914)
+ * Fix leak on skipped SSTables in sstableupgrade (CASSANDRA-12899)
+ * Avoid blocking gossip during pending range calculation (CASSANDRA-12281)
+ * Fix purgeability of tombstones with max timestamp (CASSANDRA-12792)
+ * Fail repair if participant dies during sync or anticompaction
(CASSANDRA-12901)
+ * cqlsh COPY: unprotected pk values before converting them if not using
prepared statements (CASSANDRA-12863)
+ * Fix Util.spinAssertEquals (CASSANDRA-12283)
+ * Fix potential NPE for compactionstats (CASSANDRA-12462)
+ * Prepare legacy authenticate statement if credentials table initialised
after node startup (CASSANDRA-12813)
+ * Change cassandra.wait_for_tracing_events_timeout_secs default to 0
(CASSANDRA-12754)
+ * Clean up permissions when a UDA is dropped (CASSANDRA-12720)
+ * Limit colUpdateTimeDelta histogram updates to reasonable deltas
(CASSANDRA-7)
+ * Fix leak errors and execution rejected exceptions when draining
(CASSANDRA-12457)
+ * Fix merkle tree depth calculation (CASSANDRA-12580)
+ * Make Collections deserialization more robust (CASSANDRA-12618)
+ * Better handle invalid system roles table (CASSANDRA-12700)
+ * Split consistent range movement flag correction (CASSANDRA-12786)
+ * CompactionTasks now correctly drops sstables out of compaction when not
enough disk space is available (CASSANDRA-12979)
+Merged from 2.1:
* Use portable stderr for java error in startup (CASSANDRA-13211)
* Fix Thread Leak in OutboundTcpConnection (CASSANDRA-13204)
* Coalescing strategy can enter infinite loop (CASSANDRA-13159)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/9bbb4497/build.xml
--
diff --cc build.xml
index d815ede,da42975..6dbdb9e
--- a/build.xml
+++ b/build.xml
@@@ -654,17 -571,12 +653,15 @@@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
-
+
+
+
-
-
[02/15] cassandra git commit: Remove unused repositories
Remove unused repositories
Patch by Robert Stupp; Reviewed by Michael Shuler for CASSANDRA-13278
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/301f7c5b
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/301f7c5b
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/301f7c5b
Branch: refs/heads/cassandra-2.2
Commit: 301f7c5b7bfbb73ecbfb65a6ac7b393a4d968e78
Parents: 3c2f876
Author: Robert Stupp
Authored: Wed Mar 1 13:24:22 2017 +0100
Committer: Jeff Jirsa
Committed: Wed Mar 1 13:40:04 2017 -0800
--
CHANGES.txt | 1 +
build.properties.default | 3 ---
build.xml| 4
3 files changed, 1 insertion(+), 7 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index f3de538..111f3de 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,5 +1,6 @@
2.1.18
* Log stacktrace of uncaught exceptions (CASSANDRA-13108)
+ * Remove unused repositories (CASSANDRA-13278)
2.1.17
* Use portable stderr for java error in startup (CASSANDRA-13211)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.properties.default
--
diff --git a/build.properties.default b/build.properties.default
index ff6aea0..45d65d9 100644
--- a/build.properties.default
+++ b/build.properties.default
@@ -1,7 +1,4 @@
# Maven2 Repository Locations (you can override these in "build.properties" to
point to a local proxy, e.g. Nexus)
artifact.remoteRepository.central: http://repo1.maven.org/maven2
-artifact.remoteRepository.java.net2: http://download.java.net/maven/2
artifact.remoteRepository.apache:
https://repository.apache.org/content/repositories/releases
-artifact.remoteRepository.jclouds: http://jclouds.googlecode.com/svn/repo
-artifact.remoteRepository.oauth:
http://oauth.googlecode.com/svn/code/maven
http://git-wip-us.apache.org/repos/asf/cassandra/blob/301f7c5b/build.xml
--
diff --git a/build.xml b/build.xml
index 94b1ed5..da42975 100644
--- a/build.xml
+++ b/build.xml
@@ -288,7 +288,6 @@
-
@@ -572,7 +571,6 @@
dependencyRefsBuildFile="${build.dir}/build-dependencies.xml">
-
@@ -597,7 +595,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
@@ -617,7 +614,6 @@
dependencyRefsBuildFile="${build.dir}/test-dependencies.xml">
-
[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891018#comment-15891018
]
Jeff Jirsa commented on CASSANDRA-13038:
Sorry [~jkni] - I've repro'd and pushed a fix. Will restart CI.
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the
> fact the this table has a TWCS also gives us an additional hint that sampling
> deletion times would be fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891013#comment-15891013
]
Joel Knighton commented on CASSANDRA-13038:
---
Thanks! I can reproduce the {{CompactionsTest}} failure locally, so feel free
to ping me if I can help diagnose.
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the
> fact the this table has a TWCS also gives us an additional hint that sampling
> deletion times would be fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13065) Consistent range movements to not require MV updates to go through write paths
[ https://issues.apache.org/jira/browse/CASSANDRA-13065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paulo Motta updated CASSANDRA-13065: Reviewer: Paulo Motta > Consistent range movements to not require MV updates to go through write > paths > --- > > Key: CASSANDRA-13065 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13065 > Project: Cassandra > Issue Type: Improvement >Reporter: Benjamin Roth >Assignee: Benjamin Roth >Priority: Critical > Fix For: 4.0 > > > Booting or decommisioning nodes with MVs is unbearably slow as all streams go > through the regular write paths. This causes read-before-writes for every > mutation and during bootstrap it causes them to be sent to batchlog. > The makes it virtually impossible to boot a new node in an acceptable amount > of time. > Using the regular streaming behaviour for consistent range movements works > much better in this case and does not break the MV local consistency contract. > Already tested on own cluster. > Bootstrap case is super easy to handle, decommission case requires > CASSANDRA-13064 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890982#comment-15890982
]
Jeff Jirsa commented on CASSANDRA-13038:
Ack. I've pushed a fix for
{{org.apache.cassandra.io.sstable.metadata.MetadataSerializerTest.testSerialization}}
and kicked off CI again.
So far I havent been able to get
{{org.apache.cassandra.db.compaction.CompactionsTest.testSingleSSTableCompactionWithSizeTieredCompaction}}
to fail locally, but I do see it in the previous trunk testall, so I'll try to
get that by EOD.
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the
> fact the this table has a TWCS also gives us an additional hint that sampling
> deletion times would be fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13064) Add stream type or purpose to stream plan / stream
[
https://issues.apache.org/jira/browse/CASSANDRA-13064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-13064:
Reviewer: Paulo Motta
> Add stream type or purpose to stream plan / stream
> --
>
> Key: CASSANDRA-13064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13064
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benjamin Roth
>Assignee: Benjamin Roth
> Fix For: 4.0
>
>
> It would be very good to know the type or purpose of a certain stream on the
> receiver side. It should be both available in a stream request and a stream
> task.
> Why?
> It would be helpful to distinguish the purpose to allow different handling of
> streams and requests. Examples:
> - In stream request a global flush is done. This is not necessary for all
> types of streams. A repair stream(-plan) does not require a flush as this has
> been done shortly before in validation compaction and only the sstables that
> have been validated also have to be streamed.
> - In StreamReceiveTask streams for MVs go through the regular write path this
> is painfully slow especially on bootstrap and decomission. Both for bootstrap
> and decommission this is not necessary. Sstables can be directly streamed
> down in this case. Handling bootstrap is no problem as it relies on a local
> state but during decommission, the decom-state is bound to the sender and not
> the receiver, so the receiver has to know that it is safe to stream that
> sstable directly, not through the write-path. Thats why we have to know the
> purpose of the stream.
> I'd love to implement this on my own but I am not sure how not to break the
> streaming protocol for backwards compat or if it is ok to do so.
> Furthermore I'd love to get some feedback on that idea and some proposals
> what stream types to distinguish. I could imagine:
> - bootstrap
> - decommission
> - repair
> - replace node
> - remove node
> - range relocation
> Comments like this support my idea, knowing the purpose could avoid this.
> {quote}
> // TODO each call to transferRanges re-flushes, this is
> potentially a lot of waste
> streamPlan.transferRanges(newEndpoint, preferred,
> keyspaceName, ranges);
> {quote}
> And alternative to passing the purpose of the stream was to pass flags like:
> - requiresFlush
> - requiresWritePathForMaterializedView
> ...
> I guess passing the purpose will make the streaming protocol more robust for
> future changes and leaves decisions up to the receiver.
> But an additional "requiresFlush" would also avoid putting too much logic
> into the streaming code. The streaming code should not care about purposes,
> the caller or receiver should. So the decision if a stream requires as flush
> before stream should be up to the stream requester and the stream request
> receiver depending on the purpose of the stream.
> I'm excited about your feedback :)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890941#comment-15890941
]
Joel Knighton edited comment on CASSANDRA-13038 at 3/1/17 8:09 PM:
---
It looks like this ticket introduced a few test failures.
{{org.apache.cassandra.io.sstable.metadata.MetadataSerializerTest.testSerialization}}
is consistently failing on 3.11 and trunk after this commit, and
{{org.apache.cassandra.db.compaction.CompactionsTest.testSingleSSTableCompactionWithSizeTieredCompaction}}
is failing nearly 100% of the time after this commit on trunk.
In both cases, these tests are failing on the linked CI above and appear to
have no historical failures. I don't see any discussion of these CI failures
for the linked branches on the ticket - are they being resolved elsewhere?
EDIT: In addition, reverting this commit fixes these test failures.
was (Author: jkni):
It looks like this ticket introduced a few test failures.
{{org.apache.cassandra.io.sstable.metadata.MetadataSerializerTest.testSerialization}}
is consistently failing on 3.11 and trunk after this commit, and
{{org.apache.cassandra.db.compaction.CompactionsTest.testSingleSSTableCompactionWithSizeTieredCompaction}}
is failing nearly 100% of the time after this commit on trunk.
In both cases, these tests are failing on the linked CI above and appear to
have no historical failures. I don't see any discussion of these CI failures
for the linked branches on the ticket - are they being resolved elsewhere?
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely ca
[jira] [Reopened] (CASSANDRA-13038) 33% of compaction time spent in StreamingHistogram.update()
[
https://issues.apache.org/jira/browse/CASSANDRA-13038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joel Knighton reopened CASSANDRA-13038:
---
It looks like this ticket introduced a few test failures.
{{org.apache.cassandra.io.sstable.metadata.MetadataSerializerTest.testSerialization}}
is consistently failing on 3.11 and trunk after this commit, and
{{org.apache.cassandra.db.compaction.CompactionsTest.testSingleSSTableCompactionWithSizeTieredCompaction}}
is failing nearly 100% of the time after this commit on trunk.
In both cases, these tests are failing on the linked CI above and appear to
have no historical failures. I don't see any discussion of these CI failures
for the linked branches on the ticket - are they being resolved elsewhere?
> 33% of compaction time spent in StreamingHistogram.update()
> ---
>
> Key: CASSANDRA-13038
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13038
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Corentin Chary
>Assignee: Jeff Jirsa
> Fix For: 3.0.12, 3.11.0
>
> Attachments: compaction-speedup.patch,
> compaction-streaminghistrogram.png, profiler-snapshot.nps
>
>
> With the following table, that contains a *lot* of cells:
> {code}
> CREATE TABLE biggraphite.datapoints_11520p_60s (
> metric uuid,
> time_start_ms bigint,
> offset smallint,
> count int,
> value double,
> PRIMARY KEY ((metric, time_start_ms), offset)
> ) WITH CLUSTERING ORDER BY (offset DESC);
> AND compaction = {'class':
> 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy',
> 'compaction_window_size': '6', 'compaction_window_unit': 'HOURS',
> 'max_threshold': '32', 'min_threshold': '6'}
> Keyspace : biggraphite
> Read Count: 1822
> Read Latency: 1.8870054884742042 ms.
> Write Count: 2212271647
> Write Latency: 0.027705127678653473 ms.
> Pending Flushes: 0
> Table: datapoints_11520p_60s
> SSTable count: 47
> Space used (live): 300417555945
> Space used (total): 303147395017
> Space used by snapshots (total): 0
> Off heap memory used (total): 207453042
> SSTable Compression Ratio: 0.4955200053039823
> Number of keys (estimate): 16343723
> Memtable cell count: 220576
> Memtable data size: 17115128
> Memtable off heap memory used: 0
> Memtable switch count: 2872
> Local read count: 0
> Local read latency: NaN ms
> Local write count: 1103167888
> Local write latency: 0.025 ms
> Pending flushes: 0
> Percent repaired: 0.0
> Bloom filter false positives: 0
> Bloom filter false ratio: 0.0
> Bloom filter space used: 105118296
> Bloom filter off heap memory used: 106547192
> Index summary off heap memory used: 27730962
> Compression metadata off heap memory used: 73174888
> Compacted partition minimum bytes: 61
> Compacted partition maximum bytes: 51012
> Compacted partition mean bytes: 7899
> Average live cells per slice (last five minutes): NaN
> Maximum live cells per slice (last five minutes): 0
> Average tombstones per slice (last five minutes): NaN
> Maximum tombstones per slice (last five minutes): 0
> Dropped Mutations: 0
> {code}
> It looks like a good chunk of the compaction time is lost in
> StreamingHistogram.update() (which is used to store the estimated tombstone
> drop times).
> This could be caused by a huge number of different deletion times which would
> makes the bin huge but it this histogram should be capped to 100 keys. It's
> more likely caused by the huge number of cells.
> A simple solutions could be to only take into accounts part of the cells, the
> fact the this table has a TWCS also gives us an additional hint that sampling
> deletion times would be fine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-2848) Make the Client API support passing down timeouts
[ https://issues.apache.org/jira/browse/CASSANDRA-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890894#comment-15890894 ] sankalp kohli commented on CASSANDRA-2848: -- [~slebresne] Are you still reviewing this? > Make the Client API support passing down timeouts > - > > Key: CASSANDRA-2848 > URL: https://issues.apache.org/jira/browse/CASSANDRA-2848 > Project: Cassandra > Issue Type: Improvement >Reporter: Chris Goffinet >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 3.11.x > > Attachments: 2848-trunk.txt, 2848-trunk-v2.txt > > > Having a max server RPC timeout is good for worst case, but many applications > that have middleware in front of Cassandra, might have higher timeout > requirements. In a fail fast environment, if my application starting at say > the front-end, only has 20ms to process a request, and it must connect to X > services down the stack, by the time it hits Cassandra, we might only have > 10ms. I propose we provide the ability to specify the timeout on each call we > do optionally. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13271) Reduce lock contention on instance factories of ListType and SetType
[ https://issues.apache.org/jira/browse/CASSANDRA-13271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890871#comment-15890871 ] vincent royer commented on CASSANDRA-13271: --- Sorry, here is 3.11 trunk + patch, including DynamicCompositeType : https://github.com/strapdata/cassandra-3.11-13271 import java.util.* has been replaced by only-used imports (Eclipse organize imports). > Reduce lock contention on instance factories of ListType and SetType > > > Key: CASSANDRA-13271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13271 > Project: Cassandra > Issue Type: Improvement > Components: Core >Reporter: vincent royer >Priority: Minor > Labels: performance > Fix For: 4.x > > Attachments: 0001-CASSANDRA-13271-computeIfAbsent.patch, > 0001-CASSANDRA-13271-singleton-factory-concurrency-opimiz.patch > > > By doing some performance tests, i noticed that getInstance() in > org.apache.cassandra.db.marshal.ListType and SetType could suffer from lock > contention on the singleton factory getInstance(). Here is a proposal to > reduce lock contention by using a ConcurrentMap and the putIfAbsent method > rather than a synchronized method. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[ https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890865#comment-15890865 ] Ariel Weisberg commented on CASSANDRA-13265: bq. This happens in the existing code already. That doesn't make it not a bad behavior that blocks the reader thread while it goes to do some potentially large task that doesn't actually accomplish anything. Iterating a list with 262508 elements is not going to be very fast. The list can actually be longer now that threads aren't going to be slowed down appending to the list by the need to iterate for expiration. We have no way of knowing if this is actually fixed. How often does this issue occur? This expiration code is ancient, but it hasn't been reported until now. I don't think we should leave it to silently afflict people later on who may not know what's going on or how to address it. Expiration is based on time. There is no point in attempting expiration again immediately because almost nothing will have expired. It allows one bad connection to consume resources it shouldn't in the form of hijacking a thread to iterate a list. I don't see the downside of switching from a boolean to a long and CASing that instead. If we aren't confident in it we can set a small interval so that it still checks for expiration often although I think that just generates useless work. We can't make timeouts pass faster. > Epxiration in OutboundTcpConnection can block the reader Thread > --- > > Key: CASSANDRA-13265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13265 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 3.0.9 > Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version > 1.8.0_112-b15) > Linux 3.16 >Reporter: Christian Esken >Assignee: Christian Esken > Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz, > cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz > > > I observed that sometimes a single node in a Cassandra cluster fails to > communicate to the other nodes. This can happen at any time, during peak load > or low load. Restarting that single node from the cluster fixes the issue. > Before going in to details, I want to state that I have analyzed the > situation and am already developing a possible fix. Here is the analysis so > far: > - A Threaddump in this situation showed 324 Threads in the > OutboundTcpConnection class that want to lock the backlog queue for doing > expiration. > - A class histogram shows 262508 instances of > OutboundTcpConnection$QueuedMessage. > What is the effect of it? As soon as the Cassandra node has reached a certain > amount of queued messages, it starts thrashing itself to death. Each of the > Thread fully locks the Queue for reading and writing by calling > iterator.next(), making the situation worse and worse. > - Writing: Only after 262508 locking operation it can progress with actually > writing to the Queue. > - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and > fully lock the Queue > This means: Writing blocks the Queue for reading, and readers might even be > starved which makes the situation even worse. > - > The setup is: > - 3-node cluster > - replication factor 2 > - Consistency LOCAL_ONE > - No remote DC's > - high write throughput (10 INSERT statements per second and more during > peak times). > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[
https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890836#comment-15890836
]
Jason Brown commented on CASSANDRA-13265:
-
bq. This still allows back to back expirations to continue
This happens in the existing code already. We exit out of the loop when we find
the first message that is not timed out:
[code|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L563],
however, you could (and can) have multiple threads each iterating over the
LBQ. Where things get really nuts is in the iterator in LBQ
({{LinkedBlockingQueue.Itr}}), all of the methods need to acquire both the put
and take locks for the instance, combined with the OTC thread itself trying to
acquire the take lock. So it's pretty clear how you can get into lock
acquisition hell here if the messages are not being consumed fast enough.
[~cesken]'s patch relieves the callers of {{OutboundTcpConnection#enqueue()}}
from having to contend for the LBQ locks, and I think that's the biggest
benefit. I don't have a strong enough opinion about only expiring after some
length of time has elapsed. tbh, I'd prefer to defer that kind of extra
behavior until we can prove that the current patch doesn't satisfy what is
needed here.
> Epxiration in OutboundTcpConnection can block the reader Thread
> ---
>
> Key: CASSANDRA-13265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13265
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 3.0.9
> Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version
> 1.8.0_112-b15)
> Linux 3.16
>Reporter: Christian Esken
>Assignee: Christian Esken
> Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz,
> cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz
>
>
> I observed that sometimes a single node in a Cassandra cluster fails to
> communicate to the other nodes. This can happen at any time, during peak load
> or low load. Restarting that single node from the cluster fixes the issue.
> Before going in to details, I want to state that I have analyzed the
> situation and am already developing a possible fix. Here is the analysis so
> far:
> - A Threaddump in this situation showed 324 Threads in the
> OutboundTcpConnection class that want to lock the backlog queue for doing
> expiration.
> - A class histogram shows 262508 instances of
> OutboundTcpConnection$QueuedMessage.
> What is the effect of it? As soon as the Cassandra node has reached a certain
> amount of queued messages, it starts thrashing itself to death. Each of the
> Thread fully locks the Queue for reading and writing by calling
> iterator.next(), making the situation worse and worse.
> - Writing: Only after 262508 locking operation it can progress with actually
> writing to the Queue.
> - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and
> fully lock the Queue
> This means: Writing blocks the Queue for reading, and readers might even be
> starved which makes the situation even worse.
> -
> The setup is:
> - 3-node cluster
> - replication factor 2
> - Consistency LOCAL_ONE
> - No remote DC's
> - high write throughput (10 INSERT statements per second and more during
> peak times).
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13215) Cassandra nodes startup time 20x more after upgarding to 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-13215: Reproduced In: 3.10, 3.9, 3.7 (was: 3.7, 3.9, 3.10) Reviewer: Marcus Eriksson > Cassandra nodes startup time 20x more after upgarding to 3.x > > > Key: CASSANDRA-13215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13215 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Cluster setup: two datacenters (dc-main, dc-backup). > dc-main - 9 servers, no vnodes > dc-backup - 6 servers, vnodes >Reporter: Viktor Kuzmin > Attachments: simple-cache.patch > > > CompactionStrategyManage.getCompactionStrategyIndex is called on each sstable > at startup. And this function calls StorageService.getDiskBoundaries. And > getDiskBoundaries calls AbstractReplicationStrategy.getAddressRanges. > It appears that last function can be really slow. In our environment we have > 1545 tokens and with NetworkTopologyStrategy it can make 1545*1545 > computations in worst case (maybe I'm wrong, but it really takes lot's of > cpu). > Also this function can affect runtime later, cause it is called not only > during startup. > I've tried to implement simple cache for getDiskBoundaries results and now > startup time is about one minute instead of 25m, but I'm not sure if it's a > good solution. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (CASSANDRA-12839) CommitLogReplayException: Unexpected error deserializing mutation
[
https://issues.apache.org/jira/browse/CASSANDRA-12839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeff Jirsa updated CASSANDRA-12839:
---
Priority: Critical (was: Blocker)
> CommitLogReplayException: Unexpected error deserializing mutation
> -
>
> Key: CASSANDRA-12839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12839
> Project: Cassandra
> Issue Type: Bug
> Environment: Ubuntu, Cassandra 3.7, RF3
>Reporter: Malte Pickhan
>Priority: Critical
>
> We wanted to run a *nodetool repair* on the affected node. This was not
> possible due to CASSANDRA-12694.
> Therefore we wanted to shutdown the node in order to run scrubsstables. After
> this was done, bootstrapping the node was not possible anymore. It was not
> possible since it wasn't able to replay the commitlog. Unfortunately we
> didn't run a drain before.
> {quote}
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException:
> Unexpected error deserializing mutation; saved to
> /tmp/mutation4616409546670104469dat. This may be caused by replaying a
> mutation against a table with the same name but incompatible schema.
> Exception follows: java.io.IOError: java.io.IOException: Corrupt empty row
> found in unfiltered partition
> {quote}
> Two days before that, we actually did some modification on the table and
> dropped a column.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13278) Update build.xml and build.properties.default maven repos
[
https://issues.apache.org/jira/browse/CASSANDRA-13278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890669#comment-15890669
]
Michael Shuler commented on CASSANDRA-13278:
+1
Thanks!
> Update build.xml and build.properties.default maven repos
> -
>
> Key: CASSANDRA-13278
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13278
> Project: Cassandra
> Issue Type: Bug
> Components: Build
>Reporter: Michael Shuler
>Assignee: Robert Stupp
>Priority: Minor
> Labels: lhf
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> Only 2 of the 5 urls in build.properties.default are currently valid.
> java.net2, jclouds, and oauth urls all 404.
> {noformat}
> $ git grep remoteRepository
> build.properties.default:artifact.remoteRepository.central:
> http://repo1.maven.org/maven2
> build.properties.default:artifact.remoteRepository.java.net2:
> http://download.java.net/maven/2
> build.properties.default:artifact.remoteRepository.apache:
> https://repository.apache.org/content/repositories/releases
> build.properties.default:artifact.remoteRepository.jclouds:
> http://jclouds.googlecode.com/svn/repo
> build.properties.default:artifact.remoteRepository.oauth:
> http://oauth.googlecode.com/svn/code/maven
> build.xml: url="${artifact.remoteRepository.central}"/>
> build.xml: url="${artifact.remoteRepository.java.net2}"/>
> build.xml: url="${artifact.remoteRepository.apache}"/>
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> build.xml:
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (CASSANDRA-13215) Cassandra nodes startup time 20x more after upgarding to 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-13215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeff Jirsa updated CASSANDRA-13215: --- Reproduced In: 3.10, 3.9, 3.7 (was: 3.7, 3.9, 3.10) Status: Awaiting Feedback (was: Open) > Cassandra nodes startup time 20x more after upgarding to 3.x > > > Key: CASSANDRA-13215 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13215 > Project: Cassandra > Issue Type: Improvement > Components: Core > Environment: Cluster setup: two datacenters (dc-main, dc-backup). > dc-main - 9 servers, no vnodes > dc-backup - 6 servers, vnodes >Reporter: Viktor Kuzmin > Attachments: simple-cache.patch > > > CompactionStrategyManage.getCompactionStrategyIndex is called on each sstable > at startup. And this function calls StorageService.getDiskBoundaries. And > getDiskBoundaries calls AbstractReplicationStrategy.getAddressRanges. > It appears that last function can be really slow. In our environment we have > 1545 tokens and with NetworkTopologyStrategy it can make 1545*1545 > computations in worst case (maybe I'm wrong, but it really takes lot's of > cpu). > Also this function can affect runtime later, cause it is called not only > during startup. > I've tried to implement simple cache for getDiskBoundaries results and now > startup time is about one minute instead of 25m, but I'm not sure if it's a > good solution. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[ https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890471#comment-15890471 ] Ariel Weisberg edited comment on CASSANDRA-13265 at 3/1/17 4:21 PM: [~jjirsa] Seems like wires got crossed. This issue still exists. It's just the infinite loop in the comments that was already addressed. [~cesken] This still allows back to back expirations to continue. I think we only want to attempt expiration every 200 milliseconds or something. was (Author: aweisberg): This still allows back to back expirations to continue. I think we only want to attempt expiration every 200 milliseconds or something. > Epxiration in OutboundTcpConnection can block the reader Thread > --- > > Key: CASSANDRA-13265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13265 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 3.0.9 > Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version > 1.8.0_112-b15) > Linux 3.16 >Reporter: Christian Esken >Assignee: Christian Esken > Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz, > cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz > > > I observed that sometimes a single node in a Cassandra cluster fails to > communicate to the other nodes. This can happen at any time, during peak load > or low load. Restarting that single node from the cluster fixes the issue. > Before going in to details, I want to state that I have analyzed the > situation and am already developing a possible fix. Here is the analysis so > far: > - A Threaddump in this situation showed 324 Threads in the > OutboundTcpConnection class that want to lock the backlog queue for doing > expiration. > - A class histogram shows 262508 instances of > OutboundTcpConnection$QueuedMessage. > What is the effect of it? As soon as the Cassandra node has reached a certain > amount of queued messages, it starts thrashing itself to death. Each of the > Thread fully locks the Queue for reading and writing by calling > iterator.next(), making the situation worse and worse. > - Writing: Only after 262508 locking operation it can progress with actually > writing to the Queue. > - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and > fully lock the Queue > This means: Writing blocks the Queue for reading, and readers might even be > starved which makes the situation even worse. > - > The setup is: > - 3-node cluster > - replication factor 2 > - Consistency LOCAL_ONE > - No remote DC's > - high write throughput (10 INSERT statements per second and more during > peak times). > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[ https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890471#comment-15890471 ] Ariel Weisberg commented on CASSANDRA-13265: This still allows back to back expirations to continue. I think we only want to attempt expiration every 200 milliseconds or something. > Epxiration in OutboundTcpConnection can block the reader Thread > --- > > Key: CASSANDRA-13265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13265 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 3.0.9 > Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version > 1.8.0_112-b15) > Linux 3.16 >Reporter: Christian Esken >Assignee: Christian Esken > Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz, > cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz > > > I observed that sometimes a single node in a Cassandra cluster fails to > communicate to the other nodes. This can happen at any time, during peak load > or low load. Restarting that single node from the cluster fixes the issue. > Before going in to details, I want to state that I have analyzed the > situation and am already developing a possible fix. Here is the analysis so > far: > - A Threaddump in this situation showed 324 Threads in the > OutboundTcpConnection class that want to lock the backlog queue for doing > expiration. > - A class histogram shows 262508 instances of > OutboundTcpConnection$QueuedMessage. > What is the effect of it? As soon as the Cassandra node has reached a certain > amount of queued messages, it starts thrashing itself to death. Each of the > Thread fully locks the Queue for reading and writing by calling > iterator.next(), making the situation worse and worse. > - Writing: Only after 262508 locking operation it can progress with actually > writing to the Queue. > - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and > fully lock the Queue > This means: Writing blocks the Queue for reading, and readers might even be > starved which makes the situation even worse. > - > The setup is: > - 3-node cluster > - replication factor 2 > - Consistency LOCAL_ONE > - No remote DC's > - high write throughput (10 INSERT statements per second and more during > peak times). > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[
https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890439#comment-15890439
]
Jason Brown commented on CASSANDRA-13265:
-
- remove {{BACKLOG_EXPIRATION_DEBUG}} logging and constant.
- {{backlogExpireAt}} should be a class constant ({{private static final
BACKLOG_EXPIRE_AT}}. Also, maybe rename this to something like
{{BACKLOG_PURGE_SIZE}}.
> Epxiration in OutboundTcpConnection can block the reader Thread
> ---
>
> Key: CASSANDRA-13265
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13265
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 3.0.9
> Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version
> 1.8.0_112-b15)
> Linux 3.16
>Reporter: Christian Esken
>Assignee: Christian Esken
> Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz,
> cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz
>
>
> I observed that sometimes a single node in a Cassandra cluster fails to
> communicate to the other nodes. This can happen at any time, during peak load
> or low load. Restarting that single node from the cluster fixes the issue.
> Before going in to details, I want to state that I have analyzed the
> situation and am already developing a possible fix. Here is the analysis so
> far:
> - A Threaddump in this situation showed 324 Threads in the
> OutboundTcpConnection class that want to lock the backlog queue for doing
> expiration.
> - A class histogram shows 262508 instances of
> OutboundTcpConnection$QueuedMessage.
> What is the effect of it? As soon as the Cassandra node has reached a certain
> amount of queued messages, it starts thrashing itself to death. Each of the
> Thread fully locks the Queue for reading and writing by calling
> iterator.next(), making the situation worse and worse.
> - Writing: Only after 262508 locking operation it can progress with actually
> writing to the Queue.
> - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and
> fully lock the Queue
> This means: Writing blocks the Queue for reading, and readers might even be
> starved which makes the situation even worse.
> -
> The setup is:
> - 3-node cluster
> - replication factor 2
> - Consistency LOCAL_ONE
> - No remote DC's
> - high write throughput (10 INSERT statements per second and more during
> peak times).
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[ https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890429#comment-15890429 ] Jason Brown commented on CASSANDRA-13283: - If tests pass, I'm +1 > Make it possible to override MessagingService.Verb ids > -- > > Key: CASSANDRA-13283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13283 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > We need to add a few custom Verbs in MessagingService, it would be nice to be > able to give them id's that wont clash with any future Verbs -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to
[ https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890376#comment-15890376 ] Alex Lourie commented on CASSANDRA-13010: - Hi [~rustyrazorblade] In essence, I'd like to know if I'm on the right path. There are a lot of things in the codebase that I don't understand, but I hope that I'm at least going in right direction. Thanks. > nodetool compactionstats should say which disk a compaction is writing to > - > > Key: CASSANDRA-13010 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13010 > Project: Cassandra > Issue Type: New Feature >Reporter: Jon Haddad >Assignee: Alex Lourie > Labels: lhf > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[ https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890347#comment-15890347 ] Marcus Eriksson commented on CASSANDRA-13283: - [pushed|https://github.com/apache/cassandra/commit/2c88730b2b97791995ab4db1cad040da5ef78148] and retriggered tests > Make it possible to override MessagingService.Verb ids > -- > > Key: CASSANDRA-13283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13283 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > We need to add a few custom Verbs in MessagingService, it would be nice to be > able to give them id's that wont clash with any future Verbs -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[
https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890342#comment-15890342
]
Robert Stupp commented on CASSANDRA-13283:
--
Yea, fine with {{IntObjectOpenHashMap}}.
> Make it possible to override MessagingService.Verb ids
> --
>
> Key: CASSANDRA-13283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13283
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We need to add a few custom Verbs in MessagingService, it would be nice to be
> able to give them id's that wont clash with any future Verbs
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13010) nodetool compactionstats should say which disk a compaction is writing to
[ https://issues.apache.org/jira/browse/CASSANDRA-13010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890331#comment-15890331 ] Jon Haddad commented on CASSANDRA-13010: Hi [~alourie]! Thanks for the patch. I'm taking a look today. Schedule is a bit busy since I'm on the road but I'll try to get you some feedback tonight. Anything I should look out for? Any questions you have? > nodetool compactionstats should say which disk a compaction is writing to > - > > Key: CASSANDRA-13010 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13010 > Project: Cassandra > Issue Type: New Feature >Reporter: Jon Haddad >Assignee: Alex Lourie > Labels: lhf > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[ https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890316#comment-15890316 ] Christian Esken commented on CASSANDRA-13265: - I created a fresh branch to have a clean commit. The Pull request is opened: https://github.com/apache/cassandra/pull/95 > Epxiration in OutboundTcpConnection can block the reader Thread > --- > > Key: CASSANDRA-13265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13265 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 3.0.9 > Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version > 1.8.0_112-b15) > Linux 3.16 >Reporter: Christian Esken >Assignee: Christian Esken > Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz, > cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz > > > I observed that sometimes a single node in a Cassandra cluster fails to > communicate to the other nodes. This can happen at any time, during peak load > or low load. Restarting that single node from the cluster fixes the issue. > Before going in to details, I want to state that I have analyzed the > situation and am already developing a possible fix. Here is the analysis so > far: > - A Threaddump in this situation showed 324 Threads in the > OutboundTcpConnection class that want to lock the backlog queue for doing > expiration. > - A class histogram shows 262508 instances of > OutboundTcpConnection$QueuedMessage. > What is the effect of it? As soon as the Cassandra node has reached a certain > amount of queued messages, it starts thrashing itself to death. Each of the > Thread fully locks the Queue for reading and writing by calling > iterator.next(), making the situation worse and worse. > - Writing: Only after 262508 locking operation it can progress with actually > writing to the Queue. > - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and > fully lock the Queue > This means: Writing blocks the Queue for reading, and readers might even be > starved which makes the situation even worse. > - > The setup is: > - 3-node cluster > - replication factor 2 > - Consistency LOCAL_ONE > - No remote DC's > - high write throughput (10 INSERT statements per second and more during > peak times). > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13265) Epxiration in OutboundTcpConnection can block the reader Thread
[ https://issues.apache.org/jira/browse/CASSANDRA-13265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890315#comment-15890315 ] ASF GitHub Bot commented on CASSANDRA-13265: GitHub user christian-esken opened a pull request: https://github.com/apache/cassandra/pull/95 Expire messages by a single Thread When queue expiration is done, one single Thread is elected to do the work. Previously, all Threads would go in and do the same work, producing high lock contention. The Thread reading from the Queue could even be starved by not be able to acquire the read lock. CASSANDRA-13265 You can merge this pull request into a Git repository by running: $ git pull https://github.com/christian-esken/cassandra 13265b-3.0 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/cassandra/pull/95.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #95 commit 448d7a9c1430d9f23bd819895fc618bb227ca833 Author: Christian Esken Date: 2017-03-01T14:56:36Z Expire messages by a single Thread When queue expiration is done, one single Thread is elected to do the work. Previously, all Threads would go in and do the same work, producing high lock contention. The Thread reading from the Queue could even be starved by not be able to acquire the read lock. CASSANDRA-13265 > Epxiration in OutboundTcpConnection can block the reader Thread > --- > > Key: CASSANDRA-13265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13265 > Project: Cassandra > Issue Type: Bug > Environment: Cassandra 3.0.9 > Java HotSpot(TM) 64-Bit Server VM version 25.112-b15 (Java version > 1.8.0_112-b15) > Linux 3.16 >Reporter: Christian Esken >Assignee: Christian Esken > Attachments: cassandra.pb-cache4-dus.2017-02-17-19-36-26.chist.xz, > cassandra.pb-cache4-dus.2017-02-17-19-36-26.td.xz > > > I observed that sometimes a single node in a Cassandra cluster fails to > communicate to the other nodes. This can happen at any time, during peak load > or low load. Restarting that single node from the cluster fixes the issue. > Before going in to details, I want to state that I have analyzed the > situation and am already developing a possible fix. Here is the analysis so > far: > - A Threaddump in this situation showed 324 Threads in the > OutboundTcpConnection class that want to lock the backlog queue for doing > expiration. > - A class histogram shows 262508 instances of > OutboundTcpConnection$QueuedMessage. > What is the effect of it? As soon as the Cassandra node has reached a certain > amount of queued messages, it starts thrashing itself to death. Each of the > Thread fully locks the Queue for reading and writing by calling > iterator.next(), making the situation worse and worse. > - Writing: Only after 262508 locking operation it can progress with actually > writing to the Queue. > - Reading: Is also blocked, as 324 Threads try to do iterator.next(), and > fully lock the Queue > This means: Writing blocks the Queue for reading, and readers might even be > starved which makes the situation even worse. > - > The setup is: > - 3-node cluster > - replication factor 2 > - Consistency LOCAL_ONE > - No remote DC's > - high write throughput (10 INSERT statements per second and more during > peak times). > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[ https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890291#comment-15890291 ] Jason Brown commented on CASSANDRA-13283: - [~slebresne] good call. > Make it possible to override MessagingService.Verb ids > -- > > Key: CASSANDRA-13283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13283 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > We need to add a few custom Verbs in MessagingService, it would be nice to be > able to give them id's that wont clash with any future Verbs -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[ https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890253#comment-15890253 ] Sylvain Lebresne commented on CASSANDRA-13283: -- You could use a int->Object map from https://github.com/carrotsearch/hppc to avoid boxing though (we already depend on this (though not a very recent version if I recall) due to SASI). > Make it possible to override MessagingService.Verb ids > -- > > Key: CASSANDRA-13283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13283 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > We need to add a few custom Verbs in MessagingService, it would be nice to be > able to give them id's that wont clash with any future Verbs -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[
https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890243#comment-15890243
]
Jason Brown commented on CASSANDRA-13283:
-
I figured that map. What do you actually propose with an array, then? With an
array, if you use the {{Verb.id}} as a position into the array, you a) you
can't use negative {{id}} values, and b) need to allocate an array wide enough
to hold the highest {{id}} value.
> Make it possible to override MessagingService.Verb ids
> --
>
> Key: CASSANDRA-13283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13283
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We need to add a few custom Verbs in MessagingService, it would be nice to be
> able to give them id's that wont clash with any future Verbs
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[ https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890211#comment-15890211 ] Robert Stupp commented on CASSANDRA-13283: -- yes, an array instead of the map here: https://github.com/krummas/cassandra/commit/52b57ad48a4193922179d71ec75e11d83a638cb7#diff-af09288f448c37a525e831ee90ea49f9R259 > Make it possible to override MessagingService.Verb ids > -- > > Key: CASSANDRA-13283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13283 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 4.x > > > We need to add a few custom Verbs in MessagingService, it would be nice to be > able to give them id's that wont clash with any future Verbs -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (CASSANDRA-13283) Make it possible to override MessagingService.Verb ids
[
https://issues.apache.org/jira/browse/CASSANDRA-13283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890206#comment-15890206
]
Jason Brown commented on CASSANDRA-13283:
-
[~snazy] Are you assuming the {{Verb.id}} to be a position in the array you
propose? Or is there something else I'm not imagining? :)
> Make it possible to override MessagingService.Verb ids
> --
>
> Key: CASSANDRA-13283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13283
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 4.x
>
>
> We need to add a few custom Verbs in MessagingService, it would be nice to be
> able to give them id's that wont clash with any future Verbs
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
