[jira] [Created] (CASSANDRA-11410) Option to specify ProtocolVersion in cassandra-stress
mck created CASSANDRA-11410:
---
Summary: Option to specify ProtocolVersion in cassandra-stress
Key: CASSANDRA-11410
URL: https://issues.apache.org/jira/browse/CASSANDRA-11410
Project: Cassandra
Issue Type: Improvement
Components: Testing, Tools
Reporter: mck
Assignee: mck
Currently {{cassandra-stress}} is hardcoded to ProtocolVersion.NEWEST_SUPPORTED.
It is not always the true that the cassandra version being stressed is the same
as that cassandra-stress is being used from. An example is wanting to use the
graphing feature against Cassandra-2.1
This patch offers the option to specify the ProtocolVersion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11410) Option to specify ProtocolVersion in cassandra-stress
[
https://issues.apache.org/jira/browse/CASSANDRA-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-11410:
Attachment: 11410-trunk.txt
> Option to specify ProtocolVersion in cassandra-stress
> -
>
> Key: CASSANDRA-11410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11410
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: mck
>Assignee: mck
> Attachments: 11410-trunk.txt
>
>
> Currently {{cassandra-stress}} is hardcoded to
> ProtocolVersion.NEWEST_SUPPORTED.
> It is not always the true that the cassandra version being stressed is the
> same as that cassandra-stress is being used from. An example is wanting to
> use the graphing feature against Cassandra-2.1
> This patch offers the option to specify the ProtocolVersion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11410) Option to specify ProtocolVersion in cassandra-stress
[
https://issues.apache.org/jira/browse/CASSANDRA-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-11410:
Attachment: 11410-trunk.txt
> Option to specify ProtocolVersion in cassandra-stress
> -
>
> Key: CASSANDRA-11410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11410
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: mck
>Assignee: mck
> Attachments: 11410-trunk.txt
>
>
> Currently {{cassandra-stress}} is hardcoded to
> ProtocolVersion.NEWEST_SUPPORTED.
> It is not always the true that the cassandra version being stressed is the
> same as that cassandra-stress is being used from. An example is wanting to
> use the graphing feature against Cassandra-2.1
> This patch offers the option to specify the ProtocolVersion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11410) Option to specify ProtocolVersion in cassandra-stress
[
https://issues.apache.org/jira/browse/CASSANDRA-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-11410:
Attachment: (was: 11410-trunk.txt)
> Option to specify ProtocolVersion in cassandra-stress
> -
>
> Key: CASSANDRA-11410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11410
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: mck
>Assignee: mck
> Attachments: 11410-trunk.txt
>
>
> Currently {{cassandra-stress}} is hardcoded to
> ProtocolVersion.NEWEST_SUPPORTED.
> It is not always the true that the cassandra version being stressed is the
> same as that cassandra-stress is being used from. An example is wanting to
> use the graphing feature against Cassandra-2.1
> This patch offers the option to specify the ProtocolVersion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11410) Option to specify ProtocolVersion in cassandra-stress
[
https://issues.apache.org/jira/browse/CASSANDRA-11410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mck updated CASSANDRA-11410:
Status: Patch Available (was: Open)
> Option to specify ProtocolVersion in cassandra-stress
> -
>
> Key: CASSANDRA-11410
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11410
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing, Tools
>Reporter: mck
>Assignee: mck
> Attachments: 11410-trunk.txt
>
>
> Currently {{cassandra-stress}} is hardcoded to
> ProtocolVersion.NEWEST_SUPPORTED.
> It is not always the true that the cassandra version being stressed is the
> same as that cassandra-stress is being used from. An example is wanting to
> use the graphing feature against Cassandra-2.1
> This patch offers the option to specify the ProtocolVersion.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11390) Too big MerkleTrees allocated during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208022#comment-15208022 ] Marcus Eriksson commented on CASSANDRA-11390: - agreed, lets create a new ticket where we evaluate if we want to include the compaction gain factor when estimating the trees (and maybe increase the maxDepth) pushed a commit with log fixes (moved the logging to debug) and triggered new builds > Too big MerkleTrees allocated during repair > --- > > Key: CASSANDRA-11390 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11390 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.0.x, 3.x > > > Since CASSANDRA-5220 we create one merkle tree per range, but each of those > trees is allocated to hold all the keys on the node, taking up too much memory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11411) Reduce the amount of logging during repair
Marcus Eriksson created CASSANDRA-11411: --- Summary: Reduce the amount of logging during repair Key: CASSANDRA-11411 URL: https://issues.apache.org/jira/browse/CASSANDRA-11411 Project: Cassandra Issue Type: Bug Reporter: Marcus Eriksson Assignee: Marcus Eriksson Fix For: 2.2.x, 3.0.x, 3.x We should move some repair logging to trace - currently [this|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3418] generates 13MB of logs on a vnode cluster for a single table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11411) Reduce the amount of logging during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-11411: Issue Type: Improvement (was: Bug) > Reduce the amount of logging during repair > -- > > Key: CASSANDRA-11411 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11411 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.2.x, 3.0.x, 3.x > > > We should move some repair logging to trace - currently > [this|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3418] > generates 13MB of logs on a vnode cluster for a single table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11411) Reduce the amount of logging during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-11411: Priority: Minor (was: Major) > Reduce the amount of logging during repair > -- > > Key: CASSANDRA-11411 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11411 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.2.x, 3.0.x, 3.x > > > We should move some repair logging to trace - currently > [this|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3418] > generates 13MB of logs on a vnode cluster for a single table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11411) Reduce the amount of logging during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-11411: Status: Patch Available (was: Open) https://github.com/krummas/cassandra/commits/marcuse/11411 tests: http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11411-dtest/ http://cassci.datastax.com/view/Dev/view/krummas/job/krummas-marcuse-11411-testall/ > Reduce the amount of logging during repair > -- > > Key: CASSANDRA-11411 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11411 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.2.x, 3.0.x, 3.x > > > We should move some repair logging to trace - currently > [this|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3418] > generates 13MB of logs on a vnode cluster for a single table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11412) Many sstablescanners opened during repair
Marcus Eriksson created CASSANDRA-11412:
---
Summary: Many sstablescanners opened during repair
Key: CASSANDRA-11412
URL: https://issues.apache.org/jira/browse/CASSANDRA-11412
Project: Cassandra
Issue Type: Improvement
Reporter: Marcus Eriksson
Since CASSANDRA-5220 we open [one sstablescanner per range per
sstable|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java#L374].
If compaction gets behind and you are running vnodes with 256 tokens and RF3,
this could become a problem (ie, {{768 * number of sstables}} scanners)
We could probably refactor this similar to the way we handle scanners with LCS
- only open the scanner once we need it
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11412) Many sstablescanners opened during repair
[
https://issues.apache.org/jira/browse/CASSANDRA-11412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208065#comment-15208065
]
Marcus Eriksson commented on CASSANDRA-11412:
-
[this|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java#L381]
could also be a bug(?) since the scanners don't implement hashCode or equals
(cc [~molsson])
> Many sstablescanners opened during repair
> -
>
> Key: CASSANDRA-11412
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11412
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Marcus Eriksson
>
> Since CASSANDRA-5220 we open [one sstablescanner per range per
> sstable|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java#L374].
> If compaction gets behind and you are running vnodes with 256 tokens and
> RF3, this could become a problem (ie, {{768 * number of sstables}} scanners)
> We could probably refactor this similar to the way we handle scanners with
> LCS - only open the scanner once we need it
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11299) AssertionError when quering by secondary index
[
https://issues.apache.org/jira/browse/CASSANDRA-11299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208077#comment-15208077
]
Sam Tunnicliffe commented on CASSANDRA-11299:
-
Ideally, the sstables for the affected table would be really useful, but I can
appreciate it may not be possible to share those. It would also be useful to
identify exactly how many/which sstables contain the row being duplicated.
Unfortunately, the sstabledump tool that ships with 3.0 can't read the old
format tables so you'll need to use sstable2json from 2.2.
You can contact me directly via the email on my JIRA profile if you want to
discuss the contents of the sstables privately.
> AssertionError when quering by secondary index
> --
>
> Key: CASSANDRA-11299
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11299
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 3.3
>Reporter: Michał Matłoka
>
> Hi,
> Recently we have upgraded from Cassandra 2.2.4 to 3.3. I have issues with one
> table. When I try to query using any secondary index I get e.g. in cqlsh
> {code}
> Traceback (most recent call last):
> File "/usr/bin/cqlsh.py", line 1249, in perform_simple_statement
> result = future.result()
> File
> "/usr/share/cassandra/lib/cassandra-driver-internal-only-3.0.0-6af642d.zip/cassandra-driver-3.0.0-6af642d/cassandra/cluster.py",
> line 3122, in result
> raise self._final_exception
> ReadFailure: code=1300 [Replica(s) failed to execute read] message="Operation
> failed - received 0 responses and 1 failures" info={'failures': 1,
> 'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}
> {code}
> Node logs shows then:
> {code}
> [[AWARN [SharedPool-Worker-2] 2016-03-03 00:47:01,679
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-2,5,main]: {}
> java.lang.AssertionError: null
> at
> org.apache.cassandra.index.internal.composites.CompositesSearcher$1Transform.findEntry(CompositesSearcher.java:225)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.index.internal.composites.CompositesSearcher$1Transform.applyToRow(CompositesSearcher.java:215)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:116)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:133)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:89)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:79)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:294)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:134)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:127)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:123)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:65)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:292)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1789)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2457)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_66]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
> ~[apache-cassandra-3.3.0.jar:3.3.0]
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
> [apache-cassandra-3.3.0.jar:3.3.0]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [apache-cassandra-3.3.0.jar:3.3.0]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_66]
> {code}
> SStables are upgraded, I have tried repair and scrub. I have tried to rebuild
> indexes, and even remove them a
[jira] [Commented] (CASSANDRA-11403) Serializer/Version mismatch during upgrades to C* 3.0
[
https://issues.apache.org/jira/browse/CASSANDRA-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208076#comment-15208076
]
Sylvain Lebresne commented on CASSANDRA-11403:
--
Which exact version of C* is that? We've fixed bugs related to this already so
this is likely a duplicate, unless you can reproduce on the current branches.
> Serializer/Version mismatch during upgrades to C* 3.0
> -
>
> Key: CASSANDRA-11403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11403
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>
> The problem line seems to be:
> {code}
> MessageOut message =
> readCommand.createMessage(MessagingService.instance().getVersion(endpoint));
> {code}
> SinglePartitionReadCommand then picks the serializer based on the version:
> {code}
> return new MessageOut<>(MessagingService.Verb.READ, this, version <
> MessagingService.VERSION_30 ? legacyReadCommandSerializer : serializer);
> {code}
> However, OutboundTcpConnectionPool will test the payload size vs the version
> from its smallMessages connection:
> {code}
> return msg.payloadSize(smallMessages.getTargetVersion()) >
> LARGE_MESSAGE_THRESHOLD
> {code}
> Which is set when the connection/pool is created:
> {code}
> targetVersion = MessagingService.instance().getVersion(pool.endPoint());
> {code}
> During an upgrade, this state can change between these two calls leading the
> 3.0 serializer being used on 2.x packets and the following stacktrace:
> ERROR [OptionalTasks:1] 2016-03-07 19:53:06,445 CassandraDaemon.java:195 -
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:632)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:536)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.AbstractReadExecutor$NeverSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:214)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:918)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:251)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:212)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:77)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.QueryProcessor.proce
[jira] [Updated] (CASSANDRA-11408) simple compaction defaults for common scenarios
[
https://issues.apache.org/jira/browse/CASSANDRA-11408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sylvain Lebresne updated CASSANDRA-11408:
-
Priority: Minor (was: Major)
> simple compaction defaults for common scenarios
> ---
>
> Key: CASSANDRA-11408
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11408
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
>Reporter: Jonathan Shook
>Priority: Minor
>
> As compaction strategies get more flexible over time, some users might prefer
> to have a simple named profile for their settings.
> {code:title=example, syntax variant|borderStyle=solid}
> alter table foo.bar with compaction = 'timeseries-hourly-for-a-week';
> {code}
> {code:title=example, syntax variant |borderStyle=solid}
> alter table foo.bar with compaction = { 'profile' : 'key-value-balanced-ops'
> };
> {code}
> These would simply be a map into sets of well-tested and documented defaults
> across any of the core compaction strategies.
> This would simplify setting up compaction for well-understood workloads, but
> still allow for customization where desired.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9830) Option to disable bloom filter in highest level of LCS sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208137#comment-15208137 ] Marcus Eriksson commented on CASSANDRA-9830: (sorry for late reply) bq. why in Scenario B, the final amount of sstables is larger than Scenario A for the same dataset? No, not sure why, [~carlyeks] is it possible to run this on the [visualizer|https://issues.apache.org/jira/browse/CASSANDRA-10805] to see if we can figure it out? > Option to disable bloom filter in highest level of LCS sstables > --- > > Key: CASSANDRA-9830 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9830 > Project: Cassandra > Issue Type: New Feature > Components: Compaction >Reporter: Jonathan Ellis >Assignee: Paulo Motta >Priority: Minor > Labels: performance > Fix For: 3.x > > > We expect about 90% of data to be in the highest level of LCS in a fully > populated series. (See also CASSANDRA-9829.) > Thus if the user is primarily asking for data (partitions) that has actually > been inserted, the bloom filter on the highest level only helps reject > sstables about 10% of the time. > We should add an option that suppresses bloom filter creation on top-level > sstables. This will dramatically reduce memory usage for LCS and may even > improve performance as we no longer check a low-value filter. > (This is also an idea from RocksDB.) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9666) Provide an alternative to DTCS
[ https://issues.apache.org/jira/browse/CASSANDRA-9666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208149#comment-15208149 ] Sylvain Lebresne commented on CASSANDRA-9666: - I hate to be the buzzkill but I'm kind of -1 on just adding TWCS and calling it a day. TWCS and DTCS are targeting the exact same use case and having both in tree just doesn't make sense. It's a maintenance burden on the project and it's very confusing to users. If TWCS is noticably better than DTCS, then we should just admit it is, add TWCS and _deprecate_ DTCS. Note that I'm not saying it is, I haven't really much experience on any of the strategy, but *if* TWCS has same general performances than DTCS but the consensus is that it's more intuitive and operationally simpler, then it definitively qualifies as better (the code complexity between the 2 approaches could also be a factor to take into accounts)). That said, are we judging DTCS on its current state or on it's original one? > Provide an alternative to DTCS > -- > > Key: CASSANDRA-9666 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9666 > Project: Cassandra > Issue Type: Improvement >Reporter: Jeff Jirsa >Assignee: Jeff Jirsa > Fix For: 2.1.x, 2.2.x > > Attachments: dtcs-twcs-io.png, dtcs-twcs-load.png > > > DTCS is great for time series data, but it comes with caveats that make it > difficult to use in production (typical operator behaviors such as bootstrap, > removenode, and repair have MAJOR caveats as they relate to > max_sstable_age_days, and hints/read repair break the selection algorithm). > I'm proposing an alternative, TimeWindowCompactionStrategy, that sacrifices > the tiered nature of DTCS in order to address some of DTCS' operational > shortcomings. I believe it is necessary to propose an alternative rather than > simply adjusting DTCS, because it fundamentally removes the tiered nature in > order to remove the parameter max_sstable_age_days - the result is very very > different, even if it is heavily inspired by DTCS. > Specifically, rather than creating a number of windows of ever increasing > sizes, this strategy allows an operator to choose the window size, compact > with STCS within the first window of that size, and aggressive compact down > to a single sstable once that window is no longer current. The window size is > a combination of unit (minutes, hours, days) and size (1, etc), such that an > operator can expect all data using a block of that size to be compacted > together (that is, if your unit is hours, and size is 6, you will create > roughly 4 sstables per day, each one containing roughly 6 hours of data). > The result addresses a number of the problems with > DateTieredCompactionStrategy: > - At the present time, DTCS’s first window is compacted using an unusual > selection criteria, which prefers files with earlier timestamps, but ignores > sizes. In TimeWindowCompactionStrategy, the first window data will be > compacted with the well tested, fast, reliable STCS. All STCS options can be > passed to TimeWindowCompactionStrategy to configure the first window’s > compaction behavior. > - HintedHandoff may put old data in new sstables, but it will have little > impact other than slightly reduced efficiency (sstables will cover a wider > range, but the old timestamps will not impact sstable selection criteria > during compaction) > - ReadRepair may put old data in new sstables, but it will have little impact > other than slightly reduced efficiency (sstables will cover a wider range, > but the old timestamps will not impact sstable selection criteria during > compaction) > - Small, old sstables resulting from streams of any kind will be swiftly and > aggressively compacted with the other sstables matching their similar > maxTimestamp, without causing sstables in neighboring windows to grow in size. > - The configuration options are explicit and straightforward - the tuning > parameters leave little room for error. The window is set in common, easily > understandable terms such as “12 hours”, “1 Day”, “30 days”. The > minute/hour/day options are granular enough for users keeping data for hours, > and users keeping data for years. > - There is no explicitly configurable max sstable age, though sstables will > naturally stop compacting once new data is written in that window. > - Streaming operations can create sstables with old timestamps, and they'll > naturally be joined together with sstables in the same time bucket. This is > true for bootstrap/repair/sstableloader/removenode. > - It remains true that if old data and new data is written into the memtable > at the same time, the resulting sstables will be treated as if they were new > sstables, however, that no longer ne
[jira] [Commented] (CASSANDRA-8928) Add downgradesstables
[ https://issues.apache.org/jira/browse/CASSANDRA-8928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208166#comment-15208166 ] Kaide Mu commented on CASSANDRA-8928: - Hi, [~pauloricardomg] and [~yukim]. Sorry I couldn't reply earlier because I have been with exams during these weeks and thanks for reviewing the proposal. I agree that it's better to have first a basic sstabledowngrader than jumping directly to the framework, thus I have modified it according to your suggestions. I'll make another change if it's needed. Till now I have been looking for inspection tools, I guess I don't have any problems related with the usage. I'll keep on looking for sstablescrubber tool. In addition, do you mean with double cycle support that our sstabledowngrader should work correctly with a flow like "ma" -> "ka" -> "la"? > Add downgradesstables > - > > Key: CASSANDRA-8928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8928 > Project: Cassandra > Issue Type: New Feature > Components: Tools >Reporter: Jeremy Hanna >Priority: Minor > Labels: gsoc2016, mentor > > As mentioned in other places such as CASSANDRA-8047 and in the wild, > sometimes you need to go back. A downgrade sstables utility would be nice > for a lot of reasons and I don't know that supporting going back to the > previous major version format would be too much code since we already support > reading the previous version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8928) Add downgradesstables
[ https://issues.apache.org/jira/browse/CASSANDRA-8928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208166#comment-15208166 ] Kaide Mu edited comment on CASSANDRA-8928 at 3/23/16 10:21 AM: --- Hi, [~pauloricardomg] and [~yukim]. Sorry I couldn't reply earlier because I have been with exams during these weeks and thanks for reviewing the proposal. I agree that it's better to have first a basic sstabledowngrader than jumping directly to the framework, I have modified it according to your suggestions. I'll make another change if it's needed. Till now I have been looking for inspection tools, I guess I don't have any problems related with the usage. I'll keep on looking for sstablescrubber tool. In addition, do you mean with double cycle support that our sstabledowngrader should work correctly with a flow like "ma" -> "ka" -> "la"? was (Author: kdmu): Hi, [~pauloricardomg] and [~yukim]. Sorry I couldn't reply earlier because I have been with exams during these weeks and thanks for reviewing the proposal. I agree that it's better to have first a basic sstabledowngrader than jumping directly to the framework, thus I have modified it according to your suggestions. I'll make another change if it's needed. Till now I have been looking for inspection tools, I guess I don't have any problems related with the usage. I'll keep on looking for sstablescrubber tool. In addition, do you mean with double cycle support that our sstabledowngrader should work correctly with a flow like "ma" -> "ka" -> "la"? > Add downgradesstables > - > > Key: CASSANDRA-8928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8928 > Project: Cassandra > Issue Type: New Feature > Components: Tools >Reporter: Jeremy Hanna >Priority: Minor > Labels: gsoc2016, mentor > > As mentioned in other places such as CASSANDRA-8047 and in the wild, > sometimes you need to go back. A downgrade sstables utility would be nice > for a lot of reasons and I don't know that supporting going back to the > previous major version format would be too much code since we already support > reading the previous version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11343) Fix bloom filter sizing with LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-11343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson reassigned CASSANDRA-11343: --- Assignee: Marcus Eriksson > Fix bloom filter sizing with LCS > > > Key: CASSANDRA-11343 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11343 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.2.x, 3.0.x, 3.x > > > Since CASSANDRA-7272 we most often over allocate the bloom filter size with > LCS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11343) Fix bloom filter sizing with LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-11343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson reassigned CASSANDRA-11343: --- Assignee: (was: Marcus Eriksson) > Fix bloom filter sizing with LCS > > > Key: CASSANDRA-11343 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11343 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson > Fix For: 2.2.x, 3.0.x, 3.x > > > Since CASSANDRA-7272 we most often over allocate the bloom filter size with > LCS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11392) Add auto import java.util for UDF code block
[
https://issues.apache.org/jira/browse/CASSANDRA-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-11392:
-
Reviewer: Robert Stupp
> Add auto import java.util for UDF code block
>
>
> Key: CASSANDRA-11392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11392
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
>Priority: Minor
> Fix For: 3.6
>
> Attachments: patch.txt
>
>
> Right now, when creating Java source code for UDF, since we cannot define
> import, we need to use fully qualified class name, ex:
> {noformat}
> CREATE FUNCTION toSet(li list)
> CALLED ON NULL INPUT
> RETURNS text
> LANGUAGE java
> AS $$
> java.util.Set set = new java.util.HashSet();
> for(String txt: list) {
> set.add(txt);
> }
> return set;
> $$;
> {noformat}
> Classes from {{java.util}} package are so commonly used that it makes
> developer life easier to import automatically {{java.util.*}} in the
> {{JavaUDF}} base class so that developers don't need to use FQCN for common
> classes.
> The only drawback I can see is the risk of class name clash but since:
> 1. it is not allow to create new class
> 2. classes that can be used in UDF are restricted
> I don't see serious clash name issues either
> [~snazy] WDYT ?
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11405) Server encryption cannot be enabled with the IBM JRE 1.7
[
https://issues.apache.org/jira/browse/CASSANDRA-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208330#comment-15208330
]
Stefan Podkowinski commented on CASSANDRA-11405:
This has been addressed in CASSANDRA-10508, but likely won't be back ported to
2.2, as the issue doesn't effect supported JDKs.
> Server encryption cannot be enabled with the IBM JRE 1.7
>
>
> Key: CASSANDRA-11405
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11405
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
> Environment: Linux, IBM JRE 1.7
>Reporter: Guillermo Vega-Toro
> Fix For: 2.2.6
>
>
> When enabling server encryption with the IBM JRE (algorithm: IbmX509), an
> IllegalArgumentException is thrown from the IBM JSSE when the server is
> started:
> ERROR 10:04:37,326 Exception encountered during startup
> java.lang.IllegalArgumentException: SSLv2Hello
> at com.ibm.jsse2.qb.a(qb.java:50)
> at com.ibm.jsse2.pb.a(pb.java:101)
> at com.ibm.jsse2.pb.(pb.java:77)
> at com.ibm.jsse2.oc.setEnabledProtocols(oc.java:77)
> at
> org.apache.cassandra.security.SSLFactory.getServerSocket(SSLFactory.java:64)
> at
> org.apache.cassandra.net.MessagingService.getServerSockets(MessagingService.java:425)
> at
> org.apache.cassandra.net.MessagingService.listen(MessagingService.java:409)
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:693)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:623)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:515)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:437)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:656)
> The problem is that the IBM JSSE does not support SSLv2Hello, but this
> protocol is hard-coded in class org.apache.cassandra.security.SSLFactory:
> public static final String[] ACCEPTED_PROTOCOLS = new String[] {"SSLv2Hello",
> "TLSv1", "TLSv1.1", "TLSv1.2"};
> public static SSLServerSocket getServerSocket(EncryptionOptions options,
> InetAddress address, int port) throws IOException
> {
> SSLContext ctx = createSSLContext(options, true);
> SSLServerSocket serverSocket =
> (SSLServerSocket)ctx.getServerSocketFactory().createServerSocket();
> serverSocket.setReuseAddress(true);
> String[] suits =
> filterCipherSuites(serverSocket.getSupportedCipherSuites(),
> options.cipher_suites);
> serverSocket.setEnabledCipherSuites(suits);
> serverSocket.setNeedClientAuth(options.require_client_auth);
> serverSocket.setEnabledProtocols(ACCEPTED_PROTOCOLS);
> serverSocket.bind(new InetSocketAddress(address, port), 500);
> return serverSocket;
> }
> This ACCEPTED_PROTOCOLS array should not be hard-coded. It should rather read
> the protocols from configuration, or if the algorithm is IbmX509, simply do
> not call setEnabledProtocols - with the IBM JSSE, the enabled protocol is
> controlled by the protocol passed to SSLContext.getInstance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11349) MerkleTree mismatch when multiple range tombstones exists for the same partition and interval
[
https://issues.apache.org/jira/browse/CASSANDRA-11349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208345#comment-15208345
]
Stefan Podkowinski commented on CASSANDRA-11349:
I gave the patch some more thoughts and I'm now confident that the proposed
change is the best way to address the issue.
Basically what happens during validation compaction is that a scanner is
created for each sstable. The {{CompactionIterable.Reducer}} will then create a
{{LazilyCompactedRow}} with an iterable of {{OnDiskAtom}} for the same key in
each sstable. The purpose of {{LazilyCompactedRow}} during validation
compaction is to create a digest of the compacted version of all atoms that
would represent a single row. This is done cell by cell, where each collection
of atoms for a single cell name is consumed by {{LazilyCompactedRow.Reducer}}.
The decision on whether {{LazilyCompactedRow.Reducer}} should finish to merge
a cell and move to the next one is currently being done by
{{AbstractCellNameType.onDiskAtomComparator}}, as evaluated by
{{MergeIterator.ManyToOne}}. However, the comparator does not only compare by
name, but also by {{DeletionTime}} in case of {{RangeTombstone}}. As a
consequence, {{MergeIterator.ManyToOne}} will advance in case two
{{RangeTombstone}} with different deletion times are read, which breaks the
"_will be called one or more times with cells that share the same column name_"
contract in {{LazilyCompactedRow.Reducer}}.
The submitted patch will introduce a new {{Comparator}} that will
basically work like {{onDiskAtomComparator}}, but does not compare deletion
time. As simple as that.
||2.1||2.2||
|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-11349-2.1]|[branch|https://github.com/spodkowinski/cassandra/tree/CASSANDRA-11349-2.2]|
|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.1-testall/]|[testall|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.2-testall/]|
|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.1-dtest/]|[dtest|http://cassci.datastax.com/view/Dev/view/spodkowinski/job/spodkowinski-CASSANDRA-11349-2.2-dtest/]|
The only other places where {{LazilyCompactedRow}} is being used except
validation compaction are the cleanup and scrub functions, which shouldn't be
affected, as those are working on individual sstables and I assume that there's
no case where an sstable can have multiple identical range tombstones with
different timestamps.
> MerkleTree mismatch when multiple range tombstones exists for the same
> partition and interval
> -
>
> Key: CASSANDRA-11349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11349
> Project: Cassandra
> Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Stefan Podkowinski
>
> We observed that repair, for some of our clusters, streamed a lot of data and
> many partitions were "out of sync".
> Moreover, the read repair mismatch ratio is around 3% on those clusters,
> which is really high.
> After investigation, it appears that, if two range tombstones exists for a
> partition for the same range/interval, they're both included in the merkle
> tree computation.
> But, if for some reason, on another node, the two range tombstones were
> already compacted into a single range tombstone, this will result in a merkle
> tree difference.
> Currently, this is clearly bad because MerkleTree differences are dependent
> on compactions (and if a partition is deleted and created multiple times, the
> only way to ensure that repair "works correctly"/"don't overstream data" is
> to major compact before each repair... which is not really feasible).
> Below is a list of steps allowing to easily reproduce this case:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 float,
> c4 float,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 2);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush
> ccm node1 cqlsh
> USE test_rt;
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 3);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair
> # now grep the log and observe that there was some inconstencies detected
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> Consequences of this are a costly repair, accumu
[jira] [Updated] (CASSANDRA-11349) MerkleTree mismatch when multiple range tombstones exists for the same partition and interval
[
https://issues.apache.org/jira/browse/CASSANDRA-11349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Podkowinski updated CASSANDRA-11349:
---
Attachment: 11349-2.1.patch
> MerkleTree mismatch when multiple range tombstones exists for the same
> partition and interval
> -
>
> Key: CASSANDRA-11349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11349
> Project: Cassandra
> Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Stefan Podkowinski
> Attachments: 11349-2.1.patch
>
>
> We observed that repair, for some of our clusters, streamed a lot of data and
> many partitions were "out of sync".
> Moreover, the read repair mismatch ratio is around 3% on those clusters,
> which is really high.
> After investigation, it appears that, if two range tombstones exists for a
> partition for the same range/interval, they're both included in the merkle
> tree computation.
> But, if for some reason, on another node, the two range tombstones were
> already compacted into a single range tombstone, this will result in a merkle
> tree difference.
> Currently, this is clearly bad because MerkleTree differences are dependent
> on compactions (and if a partition is deleted and created multiple times, the
> only way to ensure that repair "works correctly"/"don't overstream data" is
> to major compact before each repair... which is not really feasible).
> Below is a list of steps allowing to easily reproduce this case:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 float,
> c4 float,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 2);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush
> ccm node1 cqlsh
> USE test_rt;
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 3);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair
> # now grep the log and observe that there was some inconstencies detected
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> Consequences of this are a costly repair, accumulating many small SSTables
> (up to thousands for a rather short period of time when using VNodes, the
> time for compaction to absorb those small files), but also an increased size
> on disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11349) MerkleTree mismatch when multiple range tombstones exists for the same partition and interval
[
https://issues.apache.org/jira/browse/CASSANDRA-11349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Podkowinski updated CASSANDRA-11349:
---
Labels: repair (was: )
Fix Version/s: 2.2.x
2.1.x
Status: Patch Available (was: In Progress)
> MerkleTree mismatch when multiple range tombstones exists for the same
> partition and interval
> -
>
> Key: CASSANDRA-11349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11349
> Project: Cassandra
> Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Stefan Podkowinski
> Labels: repair
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 11349-2.1.patch
>
>
> We observed that repair, for some of our clusters, streamed a lot of data and
> many partitions were "out of sync".
> Moreover, the read repair mismatch ratio is around 3% on those clusters,
> which is really high.
> After investigation, it appears that, if two range tombstones exists for a
> partition for the same range/interval, they're both included in the merkle
> tree computation.
> But, if for some reason, on another node, the two range tombstones were
> already compacted into a single range tombstone, this will result in a merkle
> tree difference.
> Currently, this is clearly bad because MerkleTree differences are dependent
> on compactions (and if a partition is deleted and created multiple times, the
> only way to ensure that repair "works correctly"/"don't overstream data" is
> to major compact before each repair... which is not really feasible).
> Below is a list of steps allowing to easily reproduce this case:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 float,
> c4 float,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 2);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush
> ccm node1 cqlsh
> USE test_rt;
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 3);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair
> # now grep the log and observe that there was some inconstencies detected
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> Consequences of this are a costly repair, accumulating many small SSTables
> (up to thousands for a rather short period of time when using VNodes, the
> time for compaction to absorb those small files), but also an increased size
> on disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11411) Reduce the amount of logging during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208388#comment-15208388 ] Yuki Morishita commented on CASSANDRA-11411: +1 > Reduce the amount of logging during repair > -- > > Key: CASSANDRA-11411 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11411 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.2.x, 3.0.x, 3.x > > > We should move some repair logging to trace - currently > [this|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3418] > generates 13MB of logs on a vnode cluster for a single table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11411) Reduce the amount of logging during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuki Morishita updated CASSANDRA-11411: --- Reviewer: Yuki Morishita > Reduce the amount of logging during repair > -- > > Key: CASSANDRA-11411 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11411 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.2.x, 3.0.x, 3.x > > > We should move some repair logging to trace - currently > [this|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3418] > generates 13MB of logs on a vnode cluster for a single table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11390) Too big MerkleTrees allocated during repair
[
https://issues.apache.org/jira/browse/CASSANDRA-11390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208412#comment-15208412
]
Marcus Olsson commented on CASSANDRA-11390:
---
There is a comment in the doValidationCompaction() log message saying
{code}
// MT serialize may take time
{code}
Which should probably be moved to createMerkleTrees() log message instead or
removed, whichever you prefer.
After that +1 :)
> Too big MerkleTrees allocated during repair
> ---
>
> Key: CASSANDRA-11390
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11390
> Project: Cassandra
> Issue Type: Bug
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
>
> Since CASSANDRA-5220 we create one merkle tree per range, but each of those
> trees is allocated to hold all the keys on the node, taking up too much memory
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[04/10] cassandra git commit: Reduce debug logging during repair
Reduce debug logging during repair
Patch by marcuse; reviewed by yukim for CASSANDRA-11411
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/97dbc7a2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/97dbc7a2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/97dbc7a2
Branch: refs/heads/trunk
Commit: 97dbc7a2aabc1d2b6eb311a764fcbb0f30dfef8f
Parents: b6d0a75
Author: Marcus Eriksson
Authored: Wed Mar 23 09:07:30 2016 +0100
Committer: Marcus Eriksson
Committed: Wed Mar 23 14:39:05 2016 +0100
--
src/java/org/apache/cassandra/service/StorageService.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/97dbc7a2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 9a0ba5d..5d29a5a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3184,8 +3184,8 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
*/
public List> getAllRanges(List sortedTokens)
{
-if (logger.isDebugEnabled())
-logger.debug("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
+if (logger.isTraceEnabled())
+logger.trace("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
if (sortedTokens.isEmpty())
return Collections.emptyList();
[10/10] cassandra git commit: Merge branch 'cassandra-3.5' into trunk
Merge branch 'cassandra-3.5' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/df1ff74e Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/df1ff74e Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/df1ff74e Branch: refs/heads/trunk Commit: df1ff74e205de1151c4acbdf17a686f882e5df06 Parents: a1940a1 4509934 Author: Marcus Eriksson Authored: Wed Mar 23 14:41:11 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 14:41:11 2016 +0100 -- src/java/org/apache/cassandra/service/StorageService.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) --
[01/10] cassandra git commit: Reduce debug logging during repair
Repository: cassandra
Updated Branches:
refs/heads/cassandra-2.2 b6d0a752e -> 97dbc7a2a
refs/heads/cassandra-3.0 5e722ee0a -> d479b8d68
refs/heads/cassandra-3.5 76241eb02 -> 4509934f9
refs/heads/trunk a1940a157 -> df1ff74e2
Reduce debug logging during repair
Patch by marcuse; reviewed by yukim for CASSANDRA-11411
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/97dbc7a2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/97dbc7a2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/97dbc7a2
Branch: refs/heads/cassandra-2.2
Commit: 97dbc7a2aabc1d2b6eb311a764fcbb0f30dfef8f
Parents: b6d0a75
Author: Marcus Eriksson
Authored: Wed Mar 23 09:07:30 2016 +0100
Committer: Marcus Eriksson
Committed: Wed Mar 23 14:39:05 2016 +0100
--
src/java/org/apache/cassandra/service/StorageService.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/97dbc7a2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 9a0ba5d..5d29a5a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3184,8 +3184,8 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
*/
public List> getAllRanges(List sortedTokens)
{
-if (logger.isDebugEnabled())
-logger.debug("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
+if (logger.isTraceEnabled())
+logger.trace("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
if (sortedTokens.isEmpty())
return Collections.emptyList();
[jira] [Updated] (CASSANDRA-11411) Reduce the amount of logging during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-11411: Resolution: Fixed Fix Version/s: (was: 3.0.x) (was: 2.2.x) (was: 3.x) 3.5 3.0.5 2.2.6 Status: Resolved (was: Patch Available) committed, thanks > Reduce the amount of logging during repair > -- > > Key: CASSANDRA-11411 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11411 > Project: Cassandra > Issue Type: Improvement >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Minor > Fix For: 2.2.6, 3.0.5, 3.5 > > > We should move some repair logging to trace - currently > [this|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/StorageService.java#L3418] > generates 13MB of logs on a vnode cluster for a single table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[02/10] cassandra git commit: Reduce debug logging during repair
Reduce debug logging during repair
Patch by marcuse; reviewed by yukim for CASSANDRA-11411
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/97dbc7a2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/97dbc7a2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/97dbc7a2
Branch: refs/heads/cassandra-3.0
Commit: 97dbc7a2aabc1d2b6eb311a764fcbb0f30dfef8f
Parents: b6d0a75
Author: Marcus Eriksson
Authored: Wed Mar 23 09:07:30 2016 +0100
Committer: Marcus Eriksson
Committed: Wed Mar 23 14:39:05 2016 +0100
--
src/java/org/apache/cassandra/service/StorageService.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/97dbc7a2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 9a0ba5d..5d29a5a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3184,8 +3184,8 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
*/
public List> getAllRanges(List sortedTokens)
{
-if (logger.isDebugEnabled())
-logger.debug("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
+if (logger.isTraceEnabled())
+logger.trace("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
if (sortedTokens.isEmpty())
return Collections.emptyList();
[08/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5
Merge branch 'cassandra-3.0' into cassandra-3.5 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4509934f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4509934f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4509934f Branch: refs/heads/cassandra-3.5 Commit: 4509934f9e73a92ad9d79fbc10b753a73fd9cf98 Parents: 76241eb d479b8d Author: Marcus Eriksson Authored: Wed Mar 23 14:40:55 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 14:40:55 2016 +0100 -- src/java/org/apache/cassandra/service/StorageService.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4509934f/src/java/org/apache/cassandra/service/StorageService.java --
[09/10] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5
Merge branch 'cassandra-3.0' into cassandra-3.5 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/4509934f Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/4509934f Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/4509934f Branch: refs/heads/trunk Commit: 4509934f9e73a92ad9d79fbc10b753a73fd9cf98 Parents: 76241eb d479b8d Author: Marcus Eriksson Authored: Wed Mar 23 14:40:55 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 14:40:55 2016 +0100 -- src/java/org/apache/cassandra/service/StorageService.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/4509934f/src/java/org/apache/cassandra/service/StorageService.java --
[07/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d479b8d6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d479b8d6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d479b8d6 Branch: refs/heads/trunk Commit: d479b8d68ee69fa1b2b2cce79d186e7f93e4f197 Parents: 5e722ee 97dbc7a Author: Marcus Eriksson Authored: Wed Mar 23 14:40:39 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 14:40:39 2016 +0100 -- src/java/org/apache/cassandra/service/StorageService.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d479b8d6/src/java/org/apache/cassandra/service/StorageService.java --
[03/10] cassandra git commit: Reduce debug logging during repair
Reduce debug logging during repair
Patch by marcuse; reviewed by yukim for CASSANDRA-11411
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/97dbc7a2
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/97dbc7a2
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/97dbc7a2
Branch: refs/heads/cassandra-3.5
Commit: 97dbc7a2aabc1d2b6eb311a764fcbb0f30dfef8f
Parents: b6d0a75
Author: Marcus Eriksson
Authored: Wed Mar 23 09:07:30 2016 +0100
Committer: Marcus Eriksson
Committed: Wed Mar 23 14:39:05 2016 +0100
--
src/java/org/apache/cassandra/service/StorageService.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/97dbc7a2/src/java/org/apache/cassandra/service/StorageService.java
--
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 9a0ba5d..5d29a5a 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -3184,8 +3184,8 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
*/
public List> getAllRanges(List sortedTokens)
{
-if (logger.isDebugEnabled())
-logger.debug("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
+if (logger.isTraceEnabled())
+logger.trace("computing ranges for {}",
StringUtils.join(sortedTokens, ", "));
if (sortedTokens.isEmpty())
return Collections.emptyList();
[06/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d479b8d6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d479b8d6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d479b8d6 Branch: refs/heads/cassandra-3.0 Commit: d479b8d68ee69fa1b2b2cce79d186e7f93e4f197 Parents: 5e722ee 97dbc7a Author: Marcus Eriksson Authored: Wed Mar 23 14:40:39 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 14:40:39 2016 +0100 -- src/java/org/apache/cassandra/service/StorageService.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d479b8d6/src/java/org/apache/cassandra/service/StorageService.java --
[05/10] cassandra git commit: Merge branch 'cassandra-2.2' into cassandra-3.0
Merge branch 'cassandra-2.2' into cassandra-3.0 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/d479b8d6 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/d479b8d6 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/d479b8d6 Branch: refs/heads/cassandra-3.5 Commit: d479b8d68ee69fa1b2b2cce79d186e7f93e4f197 Parents: 5e722ee 97dbc7a Author: Marcus Eriksson Authored: Wed Mar 23 14:40:39 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 14:40:39 2016 +0100 -- src/java/org/apache/cassandra/service/StorageService.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/d479b8d6/src/java/org/apache/cassandra/service/StorageService.java --
cassandra git commit: Add auto import java.util for UDF code block
Repository: cassandra
Updated Branches:
refs/heads/trunk df1ff74e2 -> 03b42a299
Add auto import java.util for UDF code block
patch by DOAN DuyHai; reviewed by Robert Stupp for CASSANDRA-11392
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/03b42a29
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/03b42a29
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/03b42a29
Branch: refs/heads/trunk
Commit: 03b42a299b878264479068a3fae03aa2ca28d6b7
Parents: df1ff74
Author: DOAN DuyHai
Authored: Wed Mar 23 14:48:26 2016 +0100
Committer: Robert Stupp
Committed: Wed Mar 23 14:48:26 2016 +0100
--
CHANGES.txt | 1 +
.../cassandra/cql3/functions/JavaSourceUDF.txt | 2 +-
.../cql3/validation/entities/UFTest.java| 20
3 files changed, 22 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/03b42a29/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 9051909..9ca76a7 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.6
+ * Add auto import java.util for UDF code block (CASSANDRA-11392)
* Add --hex-format option to nodetool getsstables (CASSANDRA-11337)
* sstablemetadata should print sstable min/max token (CASSANDRA-7159)
* Do not wrap CassandraException in TriggerExecutor (CASSANDRA-9421)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/03b42a29/src/resources/org/apache/cassandra/cql3/functions/JavaSourceUDF.txt
--
diff --git
a/src/resources/org/apache/cassandra/cql3/functions/JavaSourceUDF.txt
b/src/resources/org/apache/cassandra/cql3/functions/JavaSourceUDF.txt
index 4bd3601..f0e9317 100644
--- a/src/resources/org/apache/cassandra/cql3/functions/JavaSourceUDF.txt
+++ b/src/resources/org/apache/cassandra/cql3/functions/JavaSourceUDF.txt
@@ -1,7 +1,7 @@
package #package_name#;
import java.nio.ByteBuffer;
-import java.util.List;
+import java.util.*;
import org.apache.cassandra.cql3.functions.JavaUDF;
http://git-wip-us.apache.org/repos/asf/cassandra/blob/03b42a29/test/unit/org/apache/cassandra/cql3/validation/entities/UFTest.java
--
diff --git
a/test/unit/org/apache/cassandra/cql3/validation/entities/UFTest.java
b/test/unit/org/apache/cassandra/cql3/validation/entities/UFTest.java
index f482d54..90dedd4 100644
--- a/test/unit/org/apache/cassandra/cql3/validation/entities/UFTest.java
+++ b/test/unit/org/apache/cassandra/cql3/validation/entities/UFTest.java
@@ -2529,4 +2529,24 @@ public class UFTest extends CQLTester
"$$");
}
+
+@Test
+public void testImportJavaUtil() throws Throwable
+{
+createTable("CREATE TABLE %s (key int primary key, sval text)");
+
+String f = createFunction(KEYSPACE, "text",
+"CREATE OR REPLACE FUNCTION %s(listText list) "
+
+"CALLED ON NULL INPUT " +
+"RETURNS set " +
+"LANGUAGE JAVA\n"+
+"AS $$\n" +
+" Set set = new HashSet(); " +
+" for (String s : listtext) {" +
+"set.add(s);" +
+" }" +
+" return set;" +
+"$$");
+
+}
}
[jira] [Updated] (CASSANDRA-11392) Add auto import java.util for UDF code block
[
https://issues.apache.org/jira/browse/CASSANDRA-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-11392:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
+1
Committed as 03b42a299b878264479068a3fae03aa2ca28d6b7 to trunk for 3.6
> Add auto import java.util for UDF code block
>
>
> Key: CASSANDRA-11392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11392
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
>Priority: Minor
> Fix For: 3.6
>
> Attachments: patch.txt
>
>
> Right now, when creating Java source code for UDF, since we cannot define
> import, we need to use fully qualified class name, ex:
> {noformat}
> CREATE FUNCTION toSet(li list)
> CALLED ON NULL INPUT
> RETURNS text
> LANGUAGE java
> AS $$
> java.util.Set set = new java.util.HashSet();
> for(String txt: list) {
> set.add(txt);
> }
> return set;
> $$;
> {noformat}
> Classes from {{java.util}} package are so commonly used that it makes
> developer life easier to import automatically {{java.util.*}} in the
> {{JavaUDF}} base class so that developers don't need to use FQCN for common
> classes.
> The only drawback I can see is the risk of class name clash but since:
> 1. it is not allow to create new class
> 2. classes that can be used in UDF are restricted
> I don't see serious clash name issues either
> [~snazy] WDYT ?
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[2/6] cassandra git commit: Allocate merkletrees with the correct size
Allocate merkletrees with the correct size
Patch by marcuse; reviewed by Marcus Olsson for CASSANDRA-11390
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d1bfae5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d1bfae5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d1bfae5
Branch: refs/heads/cassandra-3.5
Commit: 1d1bfae580d44d3b8a4678c5af5767ff17102128
Parents: d479b8d
Author: Marcus Eriksson
Authored: Mon Mar 21 15:41:32 2016 +0100
Committer: Marcus Eriksson
Committed: Wed Mar 23 14:54:39 2016 +0100
--
CHANGES.txt | 1 +
.../db/compaction/CompactionManager.java| 49 ++--
2 files changed, 35 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d1bfae5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 904206b..cf36047 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.5
+ * Allocate merkletrees with the correct size (CASSANDRA-11390)
* Support streaming pre-3.0 sstables (CASSANDRA-10990)
* Add backpressure to compressed commit log (CASSANDRA-10971)
* SSTableExport supports secondary index tables (CASSANDRA-11330)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d1bfae5/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 891f976..7c46fcb 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1078,16 +1078,8 @@ public class CompactionManager implements
CompactionManagerMBean
// Create Merkle trees suitable to hold estimated partitions for
the given ranges.
// We blindly assume that a partition is evenly distributed on all
sstables for now.
-long numPartitions = 0;
-for (SSTableReader sstable : sstables)
-{
-numPartitions +=
sstable.estimatedKeysForRanges(validator.desc.ranges);
-}
// determine tree depth from number of partitions, but cap at 20
to prevent large tree.
-int depth = numPartitions > 0 ? (int)
Math.min(Math.floor(Math.log(numPartitions)), 20) : 0;
-MerkleTrees tree = new MerkleTrees(cfs.getPartitioner());
-tree.addMerkleTrees((int) Math.pow(2, depth),
validator.desc.ranges);
-
+MerkleTrees tree = createMerkleTrees(sstables,
validator.desc.ranges, cfs);
long start = System.nanoTime();
try (AbstractCompactionStrategy.ScannerList scanners =
cfs.getCompactionStrategyManager().getScanners(sstables, validator.desc.ranges);
ValidationCompactionController controller = new
ValidationCompactionController(cfs, gcBefore);
@@ -1114,15 +1106,11 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-if (logger.isTraceEnabled())
+if (logger.isDebugEnabled())
{
-// MT serialize may take time
long duration =
TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);
-logger.trace("Validation finished in {} msec, depth {} for {}
keys, serialized size {} bytes for {}",
+logger.debug("Validation finished in {} msec, for {}",
duration,
- depth,
- numPartitions,
- MerkleTrees.serializer.serializedSize(tree, 0),
validator.desc);
}
}
@@ -1133,6 +1121,37 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
+private static MerkleTrees createMerkleTrees(Iterable
sstables, Collection> ranges, ColumnFamilyStore cfs)
+{
+MerkleTrees tree = new MerkleTrees(cfs.getPartitioner());
+long allPartitions = 0;
+Map, Long> rangePartitionCounts = new HashMap<>();
+for (Range range : ranges)
+{
+long numPartitions = 0;
+for (SSTableReader sstable : sstables)
+numPartitions +=
sstable.estimatedKeysForRanges(Collections.singleton(range));
+rangePartitionCounts.put(range, numPartitions);
+allPartitions += numPartitions;
+}
+
+for (Range range : ranges)
+{
+long numPartitions = rangePartitionCounts.get(range);
+double rangeOwningRatio = allPartitio
[5/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5
Merge branch 'cassandra-3.0' into cassandra-3.5 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f47d9769 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f47d9769 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f47d9769 Branch: refs/heads/cassandra-3.5 Commit: f47d9769ffc20c29d611972bddae12c44a4df35f Parents: 4509934 1d1bfae Author: Marcus Eriksson Authored: Wed Mar 23 15:00:34 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 15:00:34 2016 +0100 -- CHANGES.txt | 1 + .../db/compaction/CompactionManager.java| 49 ++-- 2 files changed, 35 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f47d9769/CHANGES.txt -- diff --cc CHANGES.txt index 7d74b6e,cf36047..85fca0f --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,7 -1,7 +1,8 @@@ -3.0.5 +3.5 +Merged from 3.0: + * Allocate merkletrees with the correct size (CASSANDRA-11390) * Support streaming pre-3.0 sstables (CASSANDRA-10990) - * Add backpressure to compressed commit log (CASSANDRA-10971) + * Add backpressure to compressed or encrypted commit log (CASSANDRA-10971) * SSTableExport supports secondary index tables (CASSANDRA-11330) * Fix sstabledump to include missing info in debug output (CASSANDRA-11321) * Establish and implement canonical bulk reading workload(s) (CASSANDRA-10331) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f47d9769/src/java/org/apache/cassandra/db/compaction/CompactionManager.java --
[6/6] cassandra git commit: Merge branch 'cassandra-3.5' into trunk
Merge branch 'cassandra-3.5' into trunk Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/9b7d5734 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/9b7d5734 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/9b7d5734 Branch: refs/heads/trunk Commit: 9b7d5734e44155a310167642b6d303d5a0e88cc8 Parents: 03b42a2 f47d976 Author: Marcus Eriksson Authored: Wed Mar 23 15:00:41 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 15:00:41 2016 +0100 -- CHANGES.txt | 1 + .../db/compaction/CompactionManager.java| 49 ++-- 2 files changed, 35 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/9b7d5734/CHANGES.txt -- diff --cc CHANGES.txt index 9ca76a7,85fca0f..5db2ba7 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,27 -1,6 +1,28 @@@ +3.6 + * Add auto import java.util for UDF code block (CASSANDRA-11392) + * Add --hex-format option to nodetool getsstables (CASSANDRA-11337) + * sstablemetadata should print sstable min/max token (CASSANDRA-7159) + * Do not wrap CassandraException in TriggerExecutor (CASSANDRA-9421) + * COPY TO should have higher double precision (CASSANDRA-11255) + * Stress should exit with non-zero status after failure (CASSANDRA-10340) + * Add client to cqlsh SHOW_SESSION (CASSANDRA-8958) + * Fix nodetool tablestats keyspace level metrics (CASSANDRA-11226) + * Store repair options in parent_repair_history (CASSANDRA-11244) + * Print current leveling in sstableofflinerelevel (CASSANDRA-9588) + * Change repair message for keyspaces with RF 1 (CASSANDRA-11203) + * Remove hard-coded SSL cipher suites and protocols (CASSANDRA-10508) + * Improve concurrency in CompactionStrategyManager (CASSANDRA-10099) + * (cqlsh) interpret CQL type for formatting blobs (CASSANDRA-11274) + * Refuse to start and print txn log information in case of disk + corruption (CASSANDRA-10112) + * Resolve some eclipse-warnings (CASSANDRA-11086) + * (cqlsh) Show static columns in a different color (CASSANDRA-11059) + * Allow to remove TTLs on table with default_time_to_live (CASSANDRA-11207) + + 3.5 Merged from 3.0: + * Allocate merkletrees with the correct size (CASSANDRA-11390) * Support streaming pre-3.0 sstables (CASSANDRA-10990) * Add backpressure to compressed or encrypted commit log (CASSANDRA-10971) * SSTableExport supports secondary index tables (CASSANDRA-11330)
[jira] [Updated] (CASSANDRA-11390) Too big MerkleTrees allocated during repair
[ https://issues.apache.org/jira/browse/CASSANDRA-11390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-11390: Resolution: Fixed Fix Version/s: (was: 3.0.x) (was: 3.x) 3.5 3.0.5 Status: Resolved (was: Patch Available) committed with comment moved, thanks! > Too big MerkleTrees allocated during repair > --- > > Key: CASSANDRA-11390 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11390 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 3.0.5, 3.5 > > > Since CASSANDRA-5220 we create one merkle tree per range, but each of those > trees is allocated to hold all the keys on the node, taking up too much memory -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[3/6] cassandra git commit: Allocate merkletrees with the correct size
Allocate merkletrees with the correct size
Patch by marcuse; reviewed by Marcus Olsson for CASSANDRA-11390
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d1bfae5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d1bfae5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d1bfae5
Branch: refs/heads/trunk
Commit: 1d1bfae580d44d3b8a4678c5af5767ff17102128
Parents: d479b8d
Author: Marcus Eriksson
Authored: Mon Mar 21 15:41:32 2016 +0100
Committer: Marcus Eriksson
Committed: Wed Mar 23 14:54:39 2016 +0100
--
CHANGES.txt | 1 +
.../db/compaction/CompactionManager.java| 49 ++--
2 files changed, 35 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d1bfae5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 904206b..cf36047 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.5
+ * Allocate merkletrees with the correct size (CASSANDRA-11390)
* Support streaming pre-3.0 sstables (CASSANDRA-10990)
* Add backpressure to compressed commit log (CASSANDRA-10971)
* SSTableExport supports secondary index tables (CASSANDRA-11330)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d1bfae5/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 891f976..7c46fcb 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1078,16 +1078,8 @@ public class CompactionManager implements
CompactionManagerMBean
// Create Merkle trees suitable to hold estimated partitions for
the given ranges.
// We blindly assume that a partition is evenly distributed on all
sstables for now.
-long numPartitions = 0;
-for (SSTableReader sstable : sstables)
-{
-numPartitions +=
sstable.estimatedKeysForRanges(validator.desc.ranges);
-}
// determine tree depth from number of partitions, but cap at 20
to prevent large tree.
-int depth = numPartitions > 0 ? (int)
Math.min(Math.floor(Math.log(numPartitions)), 20) : 0;
-MerkleTrees tree = new MerkleTrees(cfs.getPartitioner());
-tree.addMerkleTrees((int) Math.pow(2, depth),
validator.desc.ranges);
-
+MerkleTrees tree = createMerkleTrees(sstables,
validator.desc.ranges, cfs);
long start = System.nanoTime();
try (AbstractCompactionStrategy.ScannerList scanners =
cfs.getCompactionStrategyManager().getScanners(sstables, validator.desc.ranges);
ValidationCompactionController controller = new
ValidationCompactionController(cfs, gcBefore);
@@ -1114,15 +1106,11 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-if (logger.isTraceEnabled())
+if (logger.isDebugEnabled())
{
-// MT serialize may take time
long duration =
TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);
-logger.trace("Validation finished in {} msec, depth {} for {}
keys, serialized size {} bytes for {}",
+logger.debug("Validation finished in {} msec, for {}",
duration,
- depth,
- numPartitions,
- MerkleTrees.serializer.serializedSize(tree, 0),
validator.desc);
}
}
@@ -1133,6 +1121,37 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
+private static MerkleTrees createMerkleTrees(Iterable
sstables, Collection> ranges, ColumnFamilyStore cfs)
+{
+MerkleTrees tree = new MerkleTrees(cfs.getPartitioner());
+long allPartitions = 0;
+Map, Long> rangePartitionCounts = new HashMap<>();
+for (Range range : ranges)
+{
+long numPartitions = 0;
+for (SSTableReader sstable : sstables)
+numPartitions +=
sstable.estimatedKeysForRanges(Collections.singleton(range));
+rangePartitionCounts.put(range, numPartitions);
+allPartitions += numPartitions;
+}
+
+for (Range range : ranges)
+{
+long numPartitions = rangePartitionCounts.get(range);
+double rangeOwningRatio = allPartitions > 0 ?
[1/6] cassandra git commit: Allocate merkletrees with the correct size
Repository: cassandra
Updated Branches:
refs/heads/cassandra-3.0 d479b8d68 -> 1d1bfae58
refs/heads/cassandra-3.5 4509934f9 -> f47d9769f
refs/heads/trunk 03b42a299 -> 9b7d5734e
Allocate merkletrees with the correct size
Patch by marcuse; reviewed by Marcus Olsson for CASSANDRA-11390
Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo
Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/1d1bfae5
Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/1d1bfae5
Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/1d1bfae5
Branch: refs/heads/cassandra-3.0
Commit: 1d1bfae580d44d3b8a4678c5af5767ff17102128
Parents: d479b8d
Author: Marcus Eriksson
Authored: Mon Mar 21 15:41:32 2016 +0100
Committer: Marcus Eriksson
Committed: Wed Mar 23 14:54:39 2016 +0100
--
CHANGES.txt | 1 +
.../db/compaction/CompactionManager.java| 49 ++--
2 files changed, 35 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d1bfae5/CHANGES.txt
--
diff --git a/CHANGES.txt b/CHANGES.txt
index 904206b..cf36047 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.5
+ * Allocate merkletrees with the correct size (CASSANDRA-11390)
* Support streaming pre-3.0 sstables (CASSANDRA-10990)
* Add backpressure to compressed commit log (CASSANDRA-10971)
* SSTableExport supports secondary index tables (CASSANDRA-11330)
http://git-wip-us.apache.org/repos/asf/cassandra/blob/1d1bfae5/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
--
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 891f976..7c46fcb 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -1078,16 +1078,8 @@ public class CompactionManager implements
CompactionManagerMBean
// Create Merkle trees suitable to hold estimated partitions for
the given ranges.
// We blindly assume that a partition is evenly distributed on all
sstables for now.
-long numPartitions = 0;
-for (SSTableReader sstable : sstables)
-{
-numPartitions +=
sstable.estimatedKeysForRanges(validator.desc.ranges);
-}
// determine tree depth from number of partitions, but cap at 20
to prevent large tree.
-int depth = numPartitions > 0 ? (int)
Math.min(Math.floor(Math.log(numPartitions)), 20) : 0;
-MerkleTrees tree = new MerkleTrees(cfs.getPartitioner());
-tree.addMerkleTrees((int) Math.pow(2, depth),
validator.desc.ranges);
-
+MerkleTrees tree = createMerkleTrees(sstables,
validator.desc.ranges, cfs);
long start = System.nanoTime();
try (AbstractCompactionStrategy.ScannerList scanners =
cfs.getCompactionStrategyManager().getScanners(sstables, validator.desc.ranges);
ValidationCompactionController controller = new
ValidationCompactionController(cfs, gcBefore);
@@ -1114,15 +1106,11 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
-if (logger.isTraceEnabled())
+if (logger.isDebugEnabled())
{
-// MT serialize may take time
long duration =
TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - start);
-logger.trace("Validation finished in {} msec, depth {} for {}
keys, serialized size {} bytes for {}",
+logger.debug("Validation finished in {} msec, for {}",
duration,
- depth,
- numPartitions,
- MerkleTrees.serializer.serializedSize(tree, 0),
validator.desc);
}
}
@@ -1133,6 +1121,37 @@ public class CompactionManager implements
CompactionManagerMBean
}
}
+private static MerkleTrees createMerkleTrees(Iterable
sstables, Collection> ranges, ColumnFamilyStore cfs)
+{
+MerkleTrees tree = new MerkleTrees(cfs.getPartitioner());
+long allPartitions = 0;
+Map, Long> rangePartitionCounts = new HashMap<>();
+for (Range range : ranges)
+{
+long numPartitions = 0;
+for (SSTableReader sstable : sstables)
+numPartitions +=
sstable.estimatedKeysForRanges(Collections.singleton(range));
+rangePartitionCounts.put(range, numPartitions);
+allPartitions += numParti
[4/6] cassandra git commit: Merge branch 'cassandra-3.0' into cassandra-3.5
Merge branch 'cassandra-3.0' into cassandra-3.5 Project: http://git-wip-us.apache.org/repos/asf/cassandra/repo Commit: http://git-wip-us.apache.org/repos/asf/cassandra/commit/f47d9769 Tree: http://git-wip-us.apache.org/repos/asf/cassandra/tree/f47d9769 Diff: http://git-wip-us.apache.org/repos/asf/cassandra/diff/f47d9769 Branch: refs/heads/trunk Commit: f47d9769ffc20c29d611972bddae12c44a4df35f Parents: 4509934 1d1bfae Author: Marcus Eriksson Authored: Wed Mar 23 15:00:34 2016 +0100 Committer: Marcus Eriksson Committed: Wed Mar 23 15:00:34 2016 +0100 -- CHANGES.txt | 1 + .../db/compaction/CompactionManager.java| 49 ++-- 2 files changed, 35 insertions(+), 15 deletions(-) -- http://git-wip-us.apache.org/repos/asf/cassandra/blob/f47d9769/CHANGES.txt -- diff --cc CHANGES.txt index 7d74b6e,cf36047..85fca0f --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,7 -1,7 +1,8 @@@ -3.0.5 +3.5 +Merged from 3.0: + * Allocate merkletrees with the correct size (CASSANDRA-11390) * Support streaming pre-3.0 sstables (CASSANDRA-10990) - * Add backpressure to compressed commit log (CASSANDRA-10971) + * Add backpressure to compressed or encrypted commit log (CASSANDRA-10971) * SSTableExport supports secondary index tables (CASSANDRA-11330) * Fix sstabledump to include missing info in debug output (CASSANDRA-11321) * Establish and implement canonical bulk reading workload(s) (CASSANDRA-10331) http://git-wip-us.apache.org/repos/asf/cassandra/blob/f47d9769/src/java/org/apache/cassandra/db/compaction/CompactionManager.java --
[jira] [Updated] (CASSANDRA-11392) Add auto import java.util for UDF code block
[
https://issues.apache.org/jira/browse/CASSANDRA-11392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Stupp updated CASSANDRA-11392:
-
Assignee: DOAN DuyHai
> Add auto import java.util for UDF code block
>
>
> Key: CASSANDRA-11392
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11392
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
> Environment: C* 3.4
>Reporter: DOAN DuyHai
>Assignee: DOAN DuyHai
>Priority: Minor
> Fix For: 3.6
>
> Attachments: patch.txt
>
>
> Right now, when creating Java source code for UDF, since we cannot define
> import, we need to use fully qualified class name, ex:
> {noformat}
> CREATE FUNCTION toSet(li list)
> CALLED ON NULL INPUT
> RETURNS text
> LANGUAGE java
> AS $$
> java.util.Set set = new java.util.HashSet();
> for(String txt: list) {
> set.add(txt);
> }
> return set;
> $$;
> {noformat}
> Classes from {{java.util}} package are so commonly used that it makes
> developer life easier to import automatically {{java.util.*}} in the
> {{JavaUDF}} base class so that developers don't need to use FQCN for common
> classes.
> The only drawback I can see is the risk of class name clash but since:
> 1. it is not allow to create new class
> 2. classes that can be used in UDF are restricted
> I don't see serious clash name issues either
> [~snazy] WDYT ?
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11279) dtest failure in disk_balance_test.TestDiskBalance.disk_balance_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208468#comment-15208468
]
Marcus Eriksson commented on CASSANDRA-11279:
-
[~philipthompson] yeah since we only write 10k keys it could be, can you try it?
> dtest failure in disk_balance_test.TestDiskBalance.disk_balance_bootstrap_test
> --
>
> Key: CASSANDRA-11279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11279
> Project: Cassandra
> Issue Type: Test
>Reporter: Russ Hatch
>Assignee: Philip Thompson
> Labels: dtest
>
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1011/testReport/disk_balance_test/TestDiskBalance/disk_balance_bootstrap_test
> Failed on CassCI build trunk_dtest #1011
> This looks likely to be a test issue, perhaps we need to relax the assertion
> here a bit:
> {noformat}
> values not within 20.00% of the max: (474650, 382235, 513385) (node1)
> {noformat}
> This is flaky with several flaps in the last few weeks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208474#comment-15208474
]
Marcus Eriksson commented on CASSANDRA-11172:
-
[~marco.zattoo] I think your problem will be fixed with CASSANDRA-11373
> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.0.4, 3.4
>
> Attachments: beep.txt, tpstats.txt, tpstats_compaction.txt,
> trapped_in_compaction.txt, trapped_in_compaction_mixed.txt
>
>
> Observed that after a large repair on LCS that sometimes the system will
> enter an infinite loop with vast amounts of logs lines recording, "Adding
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam
> quickly rotates out possibly useful earlier debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11373) Cancelled compaction leading to infinite loop in compaction strategy getNextBackgroundTask
[
https://issues.apache.org/jira/browse/CASSANDRA-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-11373:
Summary: Cancelled compaction leading to infinite loop in compaction
strategy getNextBackgroundTask (was: nodetool repair might get stalled when
running major compactions at the same time with LCS)
> Cancelled compaction leading to infinite loop in compaction strategy
> getNextBackgroundTask
> --
>
> Key: CASSANDRA-11373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11373
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Eduard Tudenhoefner
>Assignee: Marcus Eriksson
> Fix For: 3.0.x, 3.x
>
> Attachments: jstack.txt
>
>
> Our test is basically running *nodetool repair* on specific keyspaces (such
> as keyspace1) and the test is also triggering *nodetool compact keyspace1
> standard1* in the background.
> And so it looks like running major compactions & repairs lead to that issue
> when using *LCS*.
> Below is an excerpt from the *thread dump* (the rest is attached)
> {code}
> "CompactionExecutor:2" #33 daemon prio=1 os_prio=4 tid=0x7f5363e64f10
> nid=0x3c4e waiting for monitor entry [0x7f53340d8000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:252)
> - waiting to lock <0x0006c9362c80> (a
> org.apache.cassandra.db.compaction.CompactionStrategyManager)
> at
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTableRepairedStatusChanged(Tracker.java:434)
> at
> org.apache.cassandra.db.compaction.CompactionManager.performAnticompaction(CompactionManager.java:550)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:465)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - <0x0006c9362ca8> (a
> java.util.concurrent.ThreadPoolExecutor$Worker)
> "CompactionExecutor:1" #32 daemon prio=1 os_prio=4 tid=0x7f5363e618b0
> nid=0x3c4d runnable [0x7f5334119000]
>java.lang.Thread.State: RUNNABLE
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:650)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at com.google.common.collect.Iterators.addAll(Iterators.java:361)
> at com.google.common.collect.Iterables.addAll(Iterables.java:354)
> at
> org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:589)
> at
> org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:349)
> - locked <0x0006d0f7a6a8> (a
> org.apache.cassandra.db.compaction.LeveledManifest)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:98)
> - locked <0x0006d0f7a568> (a
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:95)
> - locked <0x0006c9362c80> (a
> org.apache.cassandra.db.compaction.CompactionStrategyManager)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:257)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> *CPU usage is at 100%*
> {code}
> top -p 15386
> top - 12:12:40 up 1:28, 1 user, load average: 1.08, 1.11, 1.16
> Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
> %Cpu(s): 0.3 us, 0.0 sy, 12.4 ni, 8
[jira] [Updated] (CASSANDRA-11373) Cancelled compaction leading to infinite loop in compaction strategy getNextBackgroundTask
[
https://issues.apache.org/jira/browse/CASSANDRA-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-11373:
Fix Version/s: 2.2.x
> Cancelled compaction leading to infinite loop in compaction strategy
> getNextBackgroundTask
> --
>
> Key: CASSANDRA-11373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11373
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Eduard Tudenhoefner
>Assignee: Marcus Eriksson
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: jstack.txt
>
>
> Our test is basically running *nodetool repair* on specific keyspaces (such
> as keyspace1) and the test is also triggering *nodetool compact keyspace1
> standard1* in the background.
> And so it looks like running major compactions & repairs lead to that issue
> when using *LCS*.
> Below is an excerpt from the *thread dump* (the rest is attached)
> {code}
> "CompactionExecutor:2" #33 daemon prio=1 os_prio=4 tid=0x7f5363e64f10
> nid=0x3c4e waiting for monitor entry [0x7f53340d8000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:252)
> - waiting to lock <0x0006c9362c80> (a
> org.apache.cassandra.db.compaction.CompactionStrategyManager)
> at
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTableRepairedStatusChanged(Tracker.java:434)
> at
> org.apache.cassandra.db.compaction.CompactionManager.performAnticompaction(CompactionManager.java:550)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:465)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - <0x0006c9362ca8> (a
> java.util.concurrent.ThreadPoolExecutor$Worker)
> "CompactionExecutor:1" #32 daemon prio=1 os_prio=4 tid=0x7f5363e618b0
> nid=0x3c4d runnable [0x7f5334119000]
>java.lang.Thread.State: RUNNABLE
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:650)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at com.google.common.collect.Iterators.addAll(Iterators.java:361)
> at com.google.common.collect.Iterables.addAll(Iterables.java:354)
> at
> org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:589)
> at
> org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:349)
> - locked <0x0006d0f7a6a8> (a
> org.apache.cassandra.db.compaction.LeveledManifest)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:98)
> - locked <0x0006d0f7a568> (a
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:95)
> - locked <0x0006c9362c80> (a
> org.apache.cassandra.db.compaction.CompactionStrategyManager)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:257)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> *CPU usage is at 100%*
> {code}
> top -p 15386
> top - 12:12:40 up 1:28, 1 user, load average: 1.08, 1.11, 1.16
> Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
> %Cpu(s): 0.3 us, 0.0 sy, 12.4 ni, 87.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0
> st
> KiB Mem: 16433792 total, 8947336 used, 7486456 free,89552 buffers
> KiB Swap:0 total,0 used,0
[jira] [Comment Edited] (CASSANDRA-11172) Infinite loop bug adding high-level SSTableReader in compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-11172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208474#comment-15208474
]
Marcus Eriksson edited comment on CASSANDRA-11172 at 3/23/16 2:33 PM:
--
[~marco.zattoo] I think your problem will be fixed with CASSANDRA-11373 - the
exception you see will abort the compaction and cause the issues mentioned there
was (Author: krummas):
[~marco.zattoo] I think your problem will be fixed with CASSANDRA-11373
> Infinite loop bug adding high-level SSTableReader in compaction
> ---
>
> Key: CASSANDRA-11172
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11172
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
> Environment: DSE 4.x / Cassandra 2.1.11.969
>Reporter: Jeff Ferland
>Assignee: Marcus Eriksson
> Fix For: 2.1.14, 2.2.6, 3.0.4, 3.4
>
> Attachments: beep.txt, tpstats.txt, tpstats_compaction.txt,
> trapped_in_compaction.txt, trapped_in_compaction_mixed.txt
>
>
> Observed that after a large repair on LCS that sometimes the system will
> enter an infinite loop with vast amounts of logs lines recording, "Adding
> high-level (L${LEVEL}) SSTableReader(path='${TABLE}') to candidates"
> This results in an outage of the node and eventual crashing. The log spam
> quickly rotates out possibly useful earlier debugging.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10134) Always require replace_address to replace existing address
[
https://issues.apache.org/jira/browse/CASSANDRA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208507#comment-15208507
]
Paulo Motta commented on CASSANDRA-10134:
-
Will this disallow genuine (but unsafe) scenarios where you want to manually
replace a node with the same tokens without going through bootstrap, such as
when you lost part of the data due to a [JBOD disk
failure|https://docs.datastax.com/en/cassandra/2.0/cassandra/operations/opsRecoverUsingJBOD.html]?
If so, we should probably update that doc and/or add yet another flag to
support those.
> Always require replace_address to replace existing address
> --
>
> Key: CASSANDRA-10134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10134
> Project: Cassandra
> Issue Type: Improvement
> Components: Distributed Metadata
>Reporter: Tyler Hobbs
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> Normally, when a node is started from a clean state with the same address as
> an existing down node, it will fail to start with an error like this:
> {noformat}
> ERROR [main] 2015-08-19 15:07:51,577 CassandraDaemon.java:554 - Exception
> encountered during startup
> java.lang.RuntimeException: A node with address /127.0.0.3 already exists,
> cancelling join. Use cassandra.replace_address if you want to replace this
> node.
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:543)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:720)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:611)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:537)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:626)
> [main/:na]
> {noformat}
> However, if {{auto_bootstrap}} is set to false or the node is in its own seed
> list, it will not throw this error and will start normally. The new node
> then takes over the host ID of the old node (even if the tokens are
> different), and the only message you will see is a warning in the other
> nodes' logs:
> {noformat}
> logger.warn("Changing {}'s host ID from {} to {}", endpoint, storedId,
> hostId);
> {noformat}
> This could cause an operator to accidentally wipe out the token information
> for a down node without replacing it. To fix this, we should check for an
> endpoint collision even if {{auto_bootstrap}} is false or the node is a seed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11343) Fix bloom filter sizing with LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-11343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremiah Jordan updated CASSANDRA-11343: Component/s: Compaction > Fix bloom filter sizing with LCS > > > Key: CASSANDRA-11343 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11343 > Project: Cassandra > Issue Type: Bug > Components: Compaction >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > Fix For: 2.2.x, 3.0.x, 3.x > > > Since CASSANDRA-7272 we most often over allocate the bloom filter size with > LCS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11373) Cancelled compaction leading to infinite loop in compaction strategy getNextBackgroundTask
[
https://issues.apache.org/jira/browse/CASSANDRA-11373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-11373:
Status: Patch Available (was: Open)
> Cancelled compaction leading to infinite loop in compaction strategy
> getNextBackgroundTask
> --
>
> Key: CASSANDRA-11373
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11373
> Project: Cassandra
> Issue Type: Bug
> Components: Compaction
>Reporter: Eduard Tudenhoefner
>Assignee: Marcus Eriksson
> Fix For: 2.2.x, 3.0.x, 3.x
>
> Attachments: jstack.txt
>
>
> Our test is basically running *nodetool repair* on specific keyspaces (such
> as keyspace1) and the test is also triggering *nodetool compact keyspace1
> standard1* in the background.
> And so it looks like running major compactions & repairs lead to that issue
> when using *LCS*.
> Below is an excerpt from the *thread dump* (the rest is attached)
> {code}
> "CompactionExecutor:2" #33 daemon prio=1 os_prio=4 tid=0x7f5363e64f10
> nid=0x3c4e waiting for monitor entry [0x7f53340d8000]
>java.lang.Thread.State: BLOCKED (on object monitor)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.handleNotification(CompactionStrategyManager.java:252)
> - waiting to lock <0x0006c9362c80> (a
> org.apache.cassandra.db.compaction.CompactionStrategyManager)
> at
> org.apache.cassandra.db.lifecycle.Tracker.notifySSTableRepairedStatusChanged(Tracker.java:434)
> at
> org.apache.cassandra.db.compaction.CompactionManager.performAnticompaction(CompactionManager.java:550)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:465)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
>Locked ownable synchronizers:
> - <0x0006c9362ca8> (a
> java.util.concurrent.ThreadPoolExecutor$Worker)
> "CompactionExecutor:1" #32 daemon prio=1 os_prio=4 tid=0x7f5363e618b0
> nid=0x3c4d runnable [0x7f5334119000]
>java.lang.Thread.State: RUNNABLE
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:650)
> at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:143)
> at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:138)
> at com.google.common.collect.Iterators.addAll(Iterators.java:361)
> at com.google.common.collect.Iterables.addAll(Iterables.java:354)
> at
> org.apache.cassandra.db.compaction.LeveledManifest.getCandidatesFor(LeveledManifest.java:589)
> at
> org.apache.cassandra.db.compaction.LeveledManifest.getCompactionCandidates(LeveledManifest.java:349)
> - locked <0x0006d0f7a6a8> (a
> org.apache.cassandra.db.compaction.LeveledManifest)
> at
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:98)
> - locked <0x0006d0f7a568> (a
> org.apache.cassandra.db.compaction.LeveledCompactionStrategy)
> at
> org.apache.cassandra.db.compaction.CompactionStrategyManager.getNextBackgroundTask(CompactionStrategyManager.java:95)
> - locked <0x0006c9362c80> (a
> org.apache.cassandra.db.compaction.CompactionStrategyManager)
> at
> org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:257)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> *CPU usage is at 100%*
> {code}
> top -p 15386
> top - 12:12:40 up 1:28, 1 user, load average: 1.08, 1.11, 1.16
> Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
> %Cpu(s): 0.3 us, 0.0 sy, 12.4 ni, 87.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0
> st
> KiB Mem: 16433792 total, 8947336 used, 7486456 free,89552 buffers
> KiB Swap:0 total,0
[jira] [Deleted] (CASSANDRA-11343) Fix bloom filter sizing with LCS
[ https://issues.apache.org/jira/browse/CASSANDRA-11343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko deleted CASSANDRA-11343: -- > Fix bloom filter sizing with LCS > > > Key: CASSANDRA-11343 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11343 > Project: Cassandra > Issue Type: Bug >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson > > Since CASSANDRA-7272 we most often over allocate the bloom filter size with > LCS -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11403) Serializer/Version mismatch during upgrades to C* 3.0
[
https://issues.apache.org/jira/browse/CASSANDRA-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208532#comment-15208532
]
Jeremiah Jordan commented on CASSANDRA-11403:
-
This is on cassandra-3.0 hash 9cfbc31bc29685bd60355a823e0cf261a89858f0
> Serializer/Version mismatch during upgrades to C* 3.0
> -
>
> Key: CASSANDRA-11403
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11403
> Project: Cassandra
> Issue Type: Bug
> Components: Streaming and Messaging
>Reporter: Anthony Cozzie
>
> The problem line seems to be:
> {code}
> MessageOut message =
> readCommand.createMessage(MessagingService.instance().getVersion(endpoint));
> {code}
> SinglePartitionReadCommand then picks the serializer based on the version:
> {code}
> return new MessageOut<>(MessagingService.Verb.READ, this, version <
> MessagingService.VERSION_30 ? legacyReadCommandSerializer : serializer);
> {code}
> However, OutboundTcpConnectionPool will test the payload size vs the version
> from its smallMessages connection:
> {code}
> return msg.payloadSize(smallMessages.getTargetVersion()) >
> LARGE_MESSAGE_THRESHOLD
> {code}
> Which is set when the connection/pool is created:
> {code}
> targetVersion = MessagingService.instance().getVersion(pool.endPoint());
> {code}
> During an upgrade, this state can change between these two calls leading the
> 3.0 serializer being used on 2.x packets and the following stacktrace:
> ERROR [OptionalTasks:1] 2016-03-07 19:53:06,445 CassandraDaemon.java:195 -
> Exception in thread Thread[OptionalTasks:1,5,main]
> java.lang.AssertionError: null
> at
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:632)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:536)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.AbstractReadExecutor$NeverSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:214)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:918)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:251)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:212)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:77)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252)
> ~[cassandra-all-3.0.3.903.jar:3.0.3.903]
> at
> org
[jira] [Commented] (CASSANDRA-10134) Always require replace_address to replace existing address
[
https://issues.apache.org/jira/browse/CASSANDRA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208551#comment-15208551
]
Joel Knighton commented on CASSANDRA-10134:
---
Good point - I don't think the implementation of this needs to. The current
behavior is:
no bootstrap/seed - no collision check
bootstrap - collision check
replace_address - collision check + require bootstrapping
The implementation could separate this to make replacement/bootstrap
orthogonal, such that replace_address with bootstrap goes through the same
machinery as currently, but replace_bootstrap with no bootstrap simply
overrides the collision check.
> Always require replace_address to replace existing address
> --
>
> Key: CASSANDRA-10134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10134
> Project: Cassandra
> Issue Type: Improvement
> Components: Distributed Metadata
>Reporter: Tyler Hobbs
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> Normally, when a node is started from a clean state with the same address as
> an existing down node, it will fail to start with an error like this:
> {noformat}
> ERROR [main] 2015-08-19 15:07:51,577 CassandraDaemon.java:554 - Exception
> encountered during startup
> java.lang.RuntimeException: A node with address /127.0.0.3 already exists,
> cancelling join. Use cassandra.replace_address if you want to replace this
> node.
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:543)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:720)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:611)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:537)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:626)
> [main/:na]
> {noformat}
> However, if {{auto_bootstrap}} is set to false or the node is in its own seed
> list, it will not throw this error and will start normally. The new node
> then takes over the host ID of the old node (even if the tokens are
> different), and the only message you will see is a warning in the other
> nodes' logs:
> {noformat}
> logger.warn("Changing {}'s host ID from {} to {}", endpoint, storedId,
> hostId);
> {noformat}
> This could cause an operator to accidentally wipe out the token information
> for a down node without replacing it. To fix this, we should check for an
> endpoint collision even if {{auto_bootstrap}} is false or the node is a seed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208544#comment-15208544 ] Jeremy Hanna commented on CASSANDRA-8969: - Sorry about the lag in responsiveness on my part. The idea was that if you set it to 2x, then you could potentially have 2x in-flight requests being kept in memory on the server in cases where requests get queued up. I think that's not immediately obvious to people who increase it to just avoid client-side timeouts. > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-10134) Always require replace_address to replace existing address
[
https://issues.apache.org/jira/browse/CASSANDRA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208551#comment-15208551
]
Joel Knighton edited comment on CASSANDRA-10134 at 3/23/16 3:06 PM:
Good point - I don't think the implementation of this needs to. The current
behavior is:
no bootstrap/seed - no collision check
bootstrap - collision check
replace_address - collision check + require bootstrapping
The implementation could separate this to make replacement/bootstrap
orthogonal, such that replace_address with bootstrap goes through the same
machinery as currently, but replace_address with no bootstrap simply overrides
the collision check.
was (Author: jkni):
Good point - I don't think the implementation of this needs to. The current
behavior is:
no bootstrap/seed - no collision check
bootstrap - collision check
replace_address - collision check + require bootstrapping
The implementation could separate this to make replacement/bootstrap
orthogonal, such that replace_address with bootstrap goes through the same
machinery as currently, but replace_bootstrap with no bootstrap simply
overrides the collision check.
> Always require replace_address to replace existing address
> --
>
> Key: CASSANDRA-10134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10134
> Project: Cassandra
> Issue Type: Improvement
> Components: Distributed Metadata
>Reporter: Tyler Hobbs
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> Normally, when a node is started from a clean state with the same address as
> an existing down node, it will fail to start with an error like this:
> {noformat}
> ERROR [main] 2015-08-19 15:07:51,577 CassandraDaemon.java:554 - Exception
> encountered during startup
> java.lang.RuntimeException: A node with address /127.0.0.3 already exists,
> cancelling join. Use cassandra.replace_address if you want to replace this
> node.
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:543)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:720)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:611)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:537)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:626)
> [main/:na]
> {noformat}
> However, if {{auto_bootstrap}} is set to false or the node is in its own seed
> list, it will not throw this error and will start normally. The new node
> then takes over the host ID of the old node (even if the tokens are
> different), and the only message you will see is a warning in the other
> nodes' logs:
> {noformat}
> logger.warn("Changing {}'s host ID from {} to {}", endpoint, storedId,
> hostId);
> {noformat}
> This could cause an operator to accidentally wipe out the token information
> for a down node without replacing it. To fix this, we should check for an
> endpoint collision even if {{auto_bootstrap}} is false or the node is a seed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10134) Always require replace_address to replace existing address
[
https://issues.apache.org/jira/browse/CASSANDRA-10134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208556#comment-15208556
]
Paulo Motta commented on CASSANDRA-10134:
-
good idea, +1.
> Always require replace_address to replace existing address
> --
>
> Key: CASSANDRA-10134
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10134
> Project: Cassandra
> Issue Type: Improvement
> Components: Distributed Metadata
>Reporter: Tyler Hobbs
>Assignee: Sam Tunnicliffe
> Fix For: 3.x
>
>
> Normally, when a node is started from a clean state with the same address as
> an existing down node, it will fail to start with an error like this:
> {noformat}
> ERROR [main] 2015-08-19 15:07:51,577 CassandraDaemon.java:554 - Exception
> encountered during startup
> java.lang.RuntimeException: A node with address /127.0.0.3 already exists,
> cancelling join. Use cassandra.replace_address if you want to replace this
> node.
> at
> org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:543)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:720)
> ~[main/:na]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:611)
> ~[main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:378)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:537)
> [main/:na]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:626)
> [main/:na]
> {noformat}
> However, if {{auto_bootstrap}} is set to false or the node is in its own seed
> list, it will not throw this error and will start normally. The new node
> then takes over the host ID of the old node (even if the tokens are
> different), and the only message you will see is a warning in the other
> nodes' logs:
> {noformat}
> logger.warn("Changing {}'s host ID from {} to {}", endpoint, storedId,
> hostId);
> {noformat}
> This could cause an operator to accidentally wipe out the token information
> for a down node without replacing it. To fix this, we should check for an
> endpoint collision even if {{auto_bootstrap}} is false or the node is a seed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-10769) "received out of order wrt DecoratedKey" after scrub
[
https://issues.apache.org/jira/browse/CASSANDRA-10769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208597#comment-15208597
]
Jean-Francois Gosselin commented on CASSANDRA-10769:
Based on the comments in CASSANDRA-9935 the "AssertionError: row DecoratedKey"
is still present in 2.1.13.
> "received out of order wrt DecoratedKey" after scrub
>
>
> Key: CASSANDRA-10769
> URL: https://issues.apache.org/jira/browse/CASSANDRA-10769
> Project: Cassandra
> Issue Type: Bug
> Environment: C* 2.1.11, Debian Wheezy
>Reporter: mlowicki
>
> After running scrub and cleanup on all nodes in single data center I'm
> getting:
> {code}
> ERROR [ValidationExecutor:103] 2015-11-25 06:28:21,530 Validator.java:245 -
> Failed creating a merkle tree for [repair
> #89fa2b70-933d-11e5-b036-75bb514ae072 on sync/entity_by_id2,
> (-5867793819051725444,-5865919628027816979]], /10.210.3.221 (see log for
> details)
> ERROR [ValidationExecutor:103] 2015-11-25 06:28:21,531
> CassandraDaemon.java:227 - Exception in thread
> Thread[ValidationExecutor:103,1,main]
> java.lang.AssertionError: row DecoratedKey(-5867787467868737053,
> 000932373633313036313204808800) received out of order wrt
> DecoratedKey(-5865937851627253360, 000933313230313737333204c3c700)
> at org.apache.cassandra.repair.Validator.add(Validator.java:127)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:1010)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:94)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.db.compaction.CompactionManager$9.call(CompactionManager.java:622)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ~[na:1.7.0_80]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> ~[na:1.7.0_80]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_80]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> {code}
> What I did is to run repair on other node:
> {code}
> time nodetool repair --in-local-dc
> {code}
> Corresponding log on the node where repair has been started:
> {code}
> ERROR [AntiEntropySessions:414] 2015-11-25 06:28:21,533
> RepairSession.java:303 - [repair #89fa2b70-933d-11e5-b036-75bb514ae072]
> session completed with the following error
> org.apache.cassandra.exceptions.RepairException: [repair
> #89fa2b70-933d-11e5-b036-75bb514ae072 on sync/entity_by_id2,
> (-5867793819051725444,-5865919628027816979]] Validation failed in
> /10.210.3.117
> at
> org.apache.cassandra.repair.RepairSession.validationComplete(RepairSession.java:166)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.service.ActiveRepairService.handleMessage(ActiveRepairService.java:406)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:134)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_80]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_80]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_80]
> INFO [AntiEntropySessions:415] 2015-11-25 06:28:21,533
> RepairSession.java:260 - [repair #b9458fa0-933d-11e5-b036-75bb514ae072] new
> session: will sync /10.210.3.221, /10.210.3.118, /10.210.3.117 on range
> (7119703141488009983,7129744584776466802] for sync.[device_token, entity2,
> user_stats, user_device, user_quota, user_store, user_device_progress,
> entity_by_id2]
> ERROR [AntiEntropySessions:414] 2015-11-25 06:28:21,533
> CassandraDaemon.java:227 - Exception in thread
> Thread[AntiEntropySessions:414,5,RMI Runtime]
> java.lang.RuntimeException: org.apache.cassandra.exceptions.RepairException:
> [repair #89fa2b70-933d-11e5-b036-75bb514ae072 on sync/entity_by_id2,
> (-5867793819051725444,-5865919628027816979]] Validation failed in
> /10.210.3.117
> at com.google.common.base.Throwables.propagate(Throwables.java:160)
> ~[guava-16.0.jar:na]
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32)
> ~[apache-cassandra-2.1.11.jar:2.1.11]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> ~[na:1.7.0_80]
[jira] [Commented] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208605#comment-15208605 ] Sylvain Lebresne commented on CASSANDRA-8969: - bq. you could potentially have 2x in-flight requests being kept in memory Not totally sure I follow. Are we talking about the request objects? Because those are really tiny and I don't that being that relevant. Besides, the number of max in-flight queries is currently really limited by the number of native transport threads, and I'm not sure to see in which way a bigger rpc timeouts changes much here. Anyway, feel free to check the attached patch to see if we're talking of the same thing with different words. But if we aren't, I'm not sure to understand the problem you're mentioning. > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-9692) Print sensible units for all log messages
[ https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Giampaolo updated CASSANDRA-9692: - Attachment: ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch > Print sensible units for all log messages > - > > Key: CASSANDRA-9692 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9692 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict >Assignee: Giampaolo >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: > Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff, > Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff, > ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch, > dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch > > > Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts > log analysis. I've introduced some improvements to the bits I touched for > CASSANDRA-9681, but we should do this across the codebase. It's a small > investment for a lot of long term clarity in the logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-7962) usage counters on prepared statements, summary and details
[ https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-7962: - Component/s: Observability > usage counters on prepared statements, summary and details > -- > > Key: CASSANDRA-7962 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7962 > Project: Cassandra > Issue Type: New Feature > Components: Observability >Reporter: Jonathan Shook >Priority: Minor > > Count the usages of prepared statements on the server side, and expose them > via JMX. The original query structure should be included. Provide a nodetool > command to see the counts. Expose a meaningful id which can be used in other > places, such as with other JMX clients. If enumerations are used to identify > these for easier discussion, enumerate according to the statement id. > Allow for "since last access" deltas, or "since server start", and require > this parameter to be specified. > Show the number of seconds represented by the current data. > This would allow easier identification of access pattern distribution in the > case that prepared statements are being used. It would provide useful metrics > for diagnosis, testing, and monitoring. > nodetool command syntax: > nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [ > ks [ table ] ] > Example nodetool outputs: > nodetool spusagecountsummary deltas appks usertable > (count, statement id), since 234233 seconds ago > 56304 24ad327f9bb2578de663fc92336703dd > 5 8743b52063cd84097a65d1633f5c74f5 > 1 663fc92336703dd24ad327f9bb2578de > 0 92336703dd24ad327f9bb2578de663fc > 0 bb2578de663fc922336703dd24ad327f9 > nodetool spusagecountdetails alltime appks usertable > (count, statement id,\n query\n\n), since 234234233 seconds ago > 56320304 24ad327f9bb2578de663fc92336703dd > select user from usertable where userid=?; > 265 8743b52063cd84097a65d1633f5c74f5 > insert into usertable (userid,...) ... > 11 663fc92336703dd24ad327f9bb2578de > select tags from tagcloud where userid=?; > ... and so forth ... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-7962) usage counters on prepared statements, summary and details
[ https://issues.apache.org/jira/browse/CASSANDRA-7962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208658#comment-15208658 ] Robert Stupp commented on CASSANDRA-7962: - CASSANDRA-8831 seems to be related to this one. +10 on having metrics on prepared statements. That would help a lot in practice to answer the question: "what's going on in my cluster". (oh - seems i just volunteered ;) ) > usage counters on prepared statements, summary and details > -- > > Key: CASSANDRA-7962 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7962 > Project: Cassandra > Issue Type: New Feature > Components: Observability >Reporter: Jonathan Shook >Priority: Minor > > Count the usages of prepared statements on the server side, and expose them > via JMX. The original query structure should be included. Provide a nodetool > command to see the counts. Expose a meaningful id which can be used in other > places, such as with other JMX clients. If enumerations are used to identify > these for easier discussion, enumerate according to the statement id. > Allow for "since last access" deltas, or "since server start", and require > this parameter to be specified. > Show the number of seconds represented by the current data. > This would allow easier identification of access pattern distribution in the > case that prepared statements are being used. It would provide useful metrics > for diagnosis, testing, and monitoring. > nodetool command syntax: > nodetool ( spusagecountsummary | spusagecountdetails ) ( deltas | alltime ) [ > ks [ table ] ] > Example nodetool outputs: > nodetool spusagecountsummary deltas appks usertable > (count, statement id), since 234233 seconds ago > 56304 24ad327f9bb2578de663fc92336703dd > 5 8743b52063cd84097a65d1633f5c74f5 > 1 663fc92336703dd24ad327f9bb2578de > 0 92336703dd24ad327f9bb2578de663fc > 0 bb2578de663fc922336703dd24ad327f9 > nodetool spusagecountdetails alltime appks usertable > (count, statement id,\n query\n\n), since 234234233 seconds ago > 56320304 24ad327f9bb2578de663fc92336703dd > select user from usertable where userid=?; > 265 8743b52063cd84097a65d1633f5c74f5 > insert into usertable (userid,...) ... > 11 663fc92336703dd24ad327f9bb2578de > select tags from tagcloud where userid=?; > ... and so forth ... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-8467) Monitoring UDFs
[ https://issues.apache.org/jira/browse/CASSANDRA-8467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksey Yeschenko updated CASSANDRA-8467: - Component/s: Observability > Monitoring UDFs > --- > > Key: CASSANDRA-8467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8467 > Project: Cassandra > Issue Type: New Feature > Components: Observability >Reporter: Robert Stupp >Priority: Minor > Labels: tracing, udf > > This thicket's about to add UDF executions to session-tracing. > Tracing these parameters for UDF invocations could become very interesting. > * name of UDF > * # of invocations > * # of rejected executions > * min/max/avg execution times > "Rejected executions" would count UDFs that are not executed because an input > parameter is null/empty (CASSANDRA-8374). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208680#comment-15208680 ] Jeremiah Jordan commented on CASSANDRA-8969: [~slebresne] the issue he is talking about is when a node gets overloaded such that requests start backing up. If I am issuing 100 req/s and something happens to stall things for 10 seconds (say some other node that my requests go to stalls), with a 1 second timeout I only ever have 100 requests in flight. With a 10 second timeout I could have 1000 requests in flight on the coordinator. > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208691#comment-15208691 ] J.B. Langston edited comment on CASSANDRA-8969 at 3/23/16 4:19 PM: --- I agree this could be a good warning to have. I've seen a lot of users naively increase the timeout. Usually it's caused by I/O not keeping up with requests, but a lot of users won't take the time to figure that out. They just see their application timing out and they see something in cassandra.yaml called timeout so they increase it without thinking of the cost. Now they have GC death spiral and OOM to contend with in addition to the original problem. was (Author: jblangs...@datastax.com): I agree this could be a good warning to have. I've seen a lot of customers naively increase the timeout. Usually it's caused by I/O not keeping up with requests, but a lot of users won't take the time to figure that out. They just see their application timing out and they see something in cassandra.yaml called timeout so they increase it without thinking of the cost. Now they have GC death spiral and OOM to contend with in addition to the original problem. > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[ https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208691#comment-15208691 ] J.B. Langston commented on CASSANDRA-8969: -- I agree this could be a good warning to have. I've seen a lot of customers naively increase the timeout. Usually it's caused by I/O not keeping up with requests, but a lot of users won't take the time to figure that out. They just see their application timing out and they see something in cassandra.yaml called timeout so they increase it without thinking of the cost. Now they have GC death spiral and OOM to contend with in addition to the original problem. > Add indication in cassandra.yaml that rpc timeouts going too high will cause > memory build up > > > Key: CASSANDRA-8969 > URL: https://issues.apache.org/jira/browse/CASSANDRA-8969 > Project: Cassandra > Issue Type: Improvement > Components: Configuration >Reporter: Jeremy Hanna >Assignee: Jeremy Hanna >Priority: Minor > Labels: lhf > Fix For: 3.x > > Attachments: 8969.txt > > > It would be helpful to communicate that setting the rpc timeouts too high may > cause memory problems on the server as it can become overloaded and has to > retain the in flight requests in memory. I'll get this done but just adding > the ticket as a placeholder for memory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11405) Server encryption cannot be enabled with the IBM JRE 1.7
[
https://issues.apache.org/jira/browse/CASSANDRA-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Brown resolved CASSANDRA-11405.
-
Resolution: Won't Fix
> Server encryption cannot be enabled with the IBM JRE 1.7
>
>
> Key: CASSANDRA-11405
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11405
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
> Environment: Linux, IBM JRE 1.7
>Reporter: Guillermo Vega-Toro
> Fix For: 2.2.6
>
>
> When enabling server encryption with the IBM JRE (algorithm: IbmX509), an
> IllegalArgumentException is thrown from the IBM JSSE when the server is
> started:
> ERROR 10:04:37,326 Exception encountered during startup
> java.lang.IllegalArgumentException: SSLv2Hello
> at com.ibm.jsse2.qb.a(qb.java:50)
> at com.ibm.jsse2.pb.a(pb.java:101)
> at com.ibm.jsse2.pb.(pb.java:77)
> at com.ibm.jsse2.oc.setEnabledProtocols(oc.java:77)
> at
> org.apache.cassandra.security.SSLFactory.getServerSocket(SSLFactory.java:64)
> at
> org.apache.cassandra.net.MessagingService.getServerSockets(MessagingService.java:425)
> at
> org.apache.cassandra.net.MessagingService.listen(MessagingService.java:409)
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:693)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:623)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:515)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:437)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:656)
> The problem is that the IBM JSSE does not support SSLv2Hello, but this
> protocol is hard-coded in class org.apache.cassandra.security.SSLFactory:
> public static final String[] ACCEPTED_PROTOCOLS = new String[] {"SSLv2Hello",
> "TLSv1", "TLSv1.1", "TLSv1.2"};
> public static SSLServerSocket getServerSocket(EncryptionOptions options,
> InetAddress address, int port) throws IOException
> {
> SSLContext ctx = createSSLContext(options, true);
> SSLServerSocket serverSocket =
> (SSLServerSocket)ctx.getServerSocketFactory().createServerSocket();
> serverSocket.setReuseAddress(true);
> String[] suits =
> filterCipherSuites(serverSocket.getSupportedCipherSuites(),
> options.cipher_suites);
> serverSocket.setEnabledCipherSuites(suits);
> serverSocket.setNeedClientAuth(options.require_client_auth);
> serverSocket.setEnabledProtocols(ACCEPTED_PROTOCOLS);
> serverSocket.bind(new InetSocketAddress(address, port), 500);
> return serverSocket;
> }
> This ACCEPTED_PROTOCOLS array should not be hard-coded. It should rather read
> the protocols from configuration, or if the algorithm is IbmX509, simply do
> not call setEnabledProtocols - with the IBM JSSE, the enabled protocol is
> controlled by the protocol passed to SSLContext.getInstance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11405) Server encryption cannot be enabled with the IBM JRE 1.7
[
https://issues.apache.org/jira/browse/CASSANDRA-11405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208700#comment-15208700
]
Jason Brown commented on CASSANDRA-11405:
-
+1 to Stefan's assessment; we will not back port to < 3.x. That being said, the
patch is pretty trivial to backport, so you could apply it your own deployment.
> Server encryption cannot be enabled with the IBM JRE 1.7
>
>
> Key: CASSANDRA-11405
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11405
> Project: Cassandra
> Issue Type: Bug
> Components: Configuration
> Environment: Linux, IBM JRE 1.7
>Reporter: Guillermo Vega-Toro
> Fix For: 2.2.6
>
>
> When enabling server encryption with the IBM JRE (algorithm: IbmX509), an
> IllegalArgumentException is thrown from the IBM JSSE when the server is
> started:
> ERROR 10:04:37,326 Exception encountered during startup
> java.lang.IllegalArgumentException: SSLv2Hello
> at com.ibm.jsse2.qb.a(qb.java:50)
> at com.ibm.jsse2.pb.a(pb.java:101)
> at com.ibm.jsse2.pb.(pb.java:77)
> at com.ibm.jsse2.oc.setEnabledProtocols(oc.java:77)
> at
> org.apache.cassandra.security.SSLFactory.getServerSocket(SSLFactory.java:64)
> at
> org.apache.cassandra.net.MessagingService.getServerSockets(MessagingService.java:425)
> at
> org.apache.cassandra.net.MessagingService.listen(MessagingService.java:409)
> at
> org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:693)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:623)
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:515)
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:437)
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:567)
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:656)
> The problem is that the IBM JSSE does not support SSLv2Hello, but this
> protocol is hard-coded in class org.apache.cassandra.security.SSLFactory:
> public static final String[] ACCEPTED_PROTOCOLS = new String[] {"SSLv2Hello",
> "TLSv1", "TLSv1.1", "TLSv1.2"};
> public static SSLServerSocket getServerSocket(EncryptionOptions options,
> InetAddress address, int port) throws IOException
> {
> SSLContext ctx = createSSLContext(options, true);
> SSLServerSocket serverSocket =
> (SSLServerSocket)ctx.getServerSocketFactory().createServerSocket();
> serverSocket.setReuseAddress(true);
> String[] suits =
> filterCipherSuites(serverSocket.getSupportedCipherSuites(),
> options.cipher_suites);
> serverSocket.setEnabledCipherSuites(suits);
> serverSocket.setNeedClientAuth(options.require_client_auth);
> serverSocket.setEnabledProtocols(ACCEPTED_PROTOCOLS);
> serverSocket.bind(new InetSocketAddress(address, port), 500);
> return serverSocket;
> }
> This ACCEPTED_PROTOCOLS array should not be hard-coded. It should rather read
> the protocols from configuration, or if the algorithm is IbmX509, simply do
> not call setEnabledProtocols - with the IBM JSSE, the enabled protocol is
> controlled by the protocol passed to SSLContext.getInstance.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-9692) Print sensible units for all log messages
[
https://issues.apache.org/jira/browse/CASSANDRA-9692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208719#comment-15208719
]
Giampaolo commented on CASSANDRA-9692:
--
I added a patch for {{dtest}}. The strange thing was that I had a negative
compaction ratio like the one below, so I added also the minus sign to the regex
{code}
Compacted (3c496851-f111-11e5-bd49-13f05e23c5c9) 9 sstables to
[/tmp/dtest-8P7lxi/test/node1/data1/keyspace1/standard1-6881b590f11011e5bd4913f05e23c5c9/ma-28-big,]
to level=0. 43.792MiB to 43.868MiB (~100% of original) in 18,517ms =
-4.186KiB/s. 0 total partitions merged to 199,890. Partition merge counts
were {1:199890, }
{code}
To make things work, I added also a little change to the {{ccm}}. It's not
clear to me at the moment if ccm was to change or not. If this is the case, I
will do a PR on github.
You can find patches on github also:
[dtest|https://github.com/radicalbit/cassandra-dtest/commit/8a1017398ab55a4648fcc307a9be0644c02602dd]
and
[ccm|https://github.com/radicalbit/ccm/commit/bb08b6798f3fda39217f2daf710116a84a3ede84]
> Print sensible units for all log messages
> -
>
> Key: CASSANDRA-9692
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9692
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Benedict
>Assignee: Giampaolo
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
> Attachments:
> Cassandra9692-Rev1-trunk-giampaolo-trapasso-at-radicalbit-io.diff,
> Cassandra9692-Rev2-trunk-giampaolo.trapasso-at-radicalbit-io.diff,
> Cassandra9692-trunk-giampaolo-trapasso-at-radicalbit-io.diff,
> ccm-bb08b6798f3fda39217f2daf710116a84a3ede84.patch,
> dtests-8a1017398ab55a4648fcc307a9be0644c02602dd.patch
>
>
> Like CASSANDRA-9691, this has bugged me too long. it also adversely impacts
> log analysis. I've introduced some improvements to the bits I touched for
> CASSANDRA-9681, but we should do this across the codebase. It's a small
> investment for a lot of long term clarity in the logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11310) Allow filtering on clustering columns for queries without secondary indexes
[
https://issues.apache.org/jira/browse/CASSANDRA-11310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208743#comment-15208743
]
Alex Petrov commented on CASSANDRA-11310:
-
I hope this time it should be a bit better.
Unfortunately, making a {{useFiltering}} boolean variable similar to
{{usesSecondaryIndexing}} would mean splitting some logic within
{{StatementRestrictions::needFiltering}} into several places and adding several
iterations (for example check for whether clustering columns cause indexing and
whether adding the contains restriction would cause filtering).
Although other than that - there was one redundant check (for
{{usesSecondaryIndexing || clusteringColumnRestrictions}} that already happens
in {{processClusteringColumnRestrictions}}). Clustering column restrictions are
added without influencing indexing flag now, as {{needFiltering}} is now on
{{ClusteringColumnRestrictions}}.
I've added test covering {{ORDER}}, {{COMPACT}}, {{static}} columns, overall
in/across partitions, with different combinations of slices, with frozen
collections in PK.
New branch is here: https://github.com/ifesdjeen/cassandra/commits/11310-trunk
Thank you.
> Allow filtering on clustering columns for queries without secondary indexes
> ---
>
> Key: CASSANDRA-11310
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11310
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL
>Reporter: Benjamin Lerer
>Assignee: Alex Petrov
> Labels: doc-impacting
> Fix For: 3.x
>
>
> Since CASSANDRA-6377 queries without index filtering non-primary key columns
> are fully supported.
> It makes sense to also support filtering on clustering-columns.
> {code}
> CREATE TABLE emp_table2 (
> empID int,
> firstname text,
> lastname text,
> b_mon text,
> b_day text,
> b_yr text,
> PRIMARY KEY (empID, b_yr, b_mon, b_day));
> SELECT b_mon,b_day,b_yr,firstname,lastname FROM emp_table2
> WHERE b_mon='oct' ALLOW FILTERING;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-8969) Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up
[
https://issues.apache.org/jira/browse/CASSANDRA-8969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208747#comment-15208747
]
Jeremy Hanna commented on CASSANDRA-8969:
-
Not to bikeshed too much but what do you think of making it a little more
explicit?
{quote}
+# For this reason, you should avoid putting these settings too high. Of course
+# putting them too low is equally ill-advised since clients could get timeouts
even
+# for successful operations just because the timeout setting is too tight.
+# In other words, if you are timing out requests because of underlying
resource constraints
+# then increasing the timeout will just cause more problems.
{quote}
> Add indication in cassandra.yaml that rpc timeouts going too high will cause
> memory build up
>
>
> Key: CASSANDRA-8969
> URL: https://issues.apache.org/jira/browse/CASSANDRA-8969
> Project: Cassandra
> Issue Type: Improvement
> Components: Configuration
>Reporter: Jeremy Hanna
>Assignee: Jeremy Hanna
>Priority: Minor
> Labels: lhf
> Fix For: 3.x
>
> Attachments: 8969.txt
>
>
> It would be helpful to communicate that setting the rpc timeouts too high may
> cause memory problems on the server as it can become overloaded and has to
> retain the in flight requests in memory. I'll get this done but just adding
> the ticket as a placeholder for memory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11413) dtest failure in cql_tests.AbortedQueriesTester.remote_query_test
Philip Thompson created CASSANDRA-11413: --- Summary: dtest failure in cql_tests.AbortedQueriesTester.remote_query_test Key: CASSANDRA-11413 URL: https://issues.apache.org/jira/browse/CASSANDRA-11413 Project: Cassandra Issue Type: Test Components: Testing Reporter: Philip Thompson Assignee: Philip Thompson example failure: http://cassci.datastax.com/job/trunk_dtest/1076/testReport/cql_tests/AbortedQueriesTester/remote_query_test Failed on CassCI build trunk_dtest #1076 Also breaking: cql_tests.AbortedQueriesTester.materialized_view_test topology_test.TestTopology.do_not_join_ring_test Broken by https://github.com/pcmanus/ccm/pull/479 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11413) dtest failure in cql_tests.AbortedQueriesTester.remote_query_test
[ https://issues.apache.org/jira/browse/CASSANDRA-11413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson resolved CASSANDRA-11413. - Resolution: Fixed Reverted the ccm PR with https://github.com/pcmanus/ccm/pull/480 > dtest failure in cql_tests.AbortedQueriesTester.remote_query_test > - > > Key: CASSANDRA-11413 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11413 > Project: Cassandra > Issue Type: Test > Components: Testing >Reporter: Philip Thompson >Assignee: Philip Thompson > Labels: dtest > > example failure: > http://cassci.datastax.com/job/trunk_dtest/1076/testReport/cql_tests/AbortedQueriesTester/remote_query_test > Failed on CassCI build trunk_dtest #1076 > Also breaking: > cql_tests.AbortedQueriesTester.materialized_view_test > topology_test.TestTopology.do_not_join_ring_test > Broken by https://github.com/pcmanus/ccm/pull/479 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11414) dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
Philip Thompson created CASSANDRA-11414:
---
Summary: dtest failure in
bootstrap_test.TestBootstrap.resumable_bootstrap_test
Key: CASSANDRA-11414
URL: https://issues.apache.org/jira/browse/CASSANDRA-11414
Project: Cassandra
Issue Type: Test
Components: Testing
Reporter: Philip Thompson
Assignee: DS Test Eng
Stress is failing to read back all data. We can see this output from the stress
read
{code}
java.io.IOException: Operation x0 on key(s) [314c384f304f4c325030]: Data
returned was not validated
at org.apache.cassandra.stress.Operation.error(Operation.java:138)
at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
at
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
at
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
at
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
java.io.IOException: Operation x0 on key(s) [33383438363931353131]: Data
returned was not validated
at org.apache.cassandra.stress.Operation.error(Operation.java:138)
at
org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
at
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
at
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
at
org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
at
org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
FAILURE
{code}
Started happening with build 1075. Does not appear flaky on CI.
example failure:
http://cassci.datastax.com/job/trunk_dtest/1076/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test
Failed on CassCI build trunk_dtest #1076
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11415) dtest failure in jmx_test.TestJMX.cfhistograms_test
Philip Thompson created CASSANDRA-11415:
---
Summary: dtest failure in jmx_test.TestJMX.cfhistograms_test
Key: CASSANDRA-11415
URL: https://issues.apache.org/jira/browse/CASSANDRA-11415
Project: Cassandra
Issue Type: Test
Reporter: Philip Thompson
Assignee: DS Test Eng
Fix For: 2.1.x
We are seeing the following stacktrace when running nodetool cfhistograms
{code}
java.lang.AssertionError
at
org.apache.cassandra.utils.EstimatedHistogram.(EstimatedHistogram.java:66)
at
org.apache.cassandra.tools.NodeProbe.metricPercentilesAsArray(NodeProbe.java:1260)
at
org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:1109)
at
org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:292)
at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:206)
{code}
example failure:
http://cassci.datastax.com/job/cassandra-2.1_dtest/437/testReport/jmx_test/TestJMX/cfhistograms_test
Failed on CassCI build cassandra-2.1_dtest #437
This doesn't appear to be flaky. I can repro locally. It seems like a product
issue, but if someone could confirm if it happens out of the test, that would
be great.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11414) dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208826#comment-15208826
]
Paulo Motta commented on CASSANDRA-11414:
-
I'd say changing the stress read and/or write CL to QUORUM would fix this.
> dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
> --
>
> Key: CASSANDRA-11414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11414
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Philip Thompson
>Assignee: DS Test Eng
> Labels: dtest
>
> Stress is failing to read back all data. We can see this output from the
> stress read
> {code}
> java.io.IOException: Operation x0 on key(s) [314c384f304f4c325030]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> java.io.IOException: Operation x0 on key(s) [33383438363931353131]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> FAILURE
> {code}
> Started happening with build 1075. Does not appear flaky on CI.
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1076/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test
> Failed on CassCI build trunk_dtest #1076
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11416) No longer able to load old backups into new cluster if there was a dropped column
[
https://issues.apache.org/jira/browse/CASSANDRA-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-11416:
Fix Version/s: 3.x
3.0.x
> No longer able to load old backups into new cluster if there was a dropped
> column
> -
>
> Key: CASSANDRA-11416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11416
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremiah Jordan
> Fix For: 3.0.x, 3.x
>
>
> The following change to the sstableloader test works in 2.1/2.2 but fails in
> 3.0+
> https://github.com/JeremiahDJordan/cassandra-dtest/commit/7dc66efb8d24239f0a488ec5a613240531aeb7db
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text, c4
> text)
> ...insert data...
> ALTER TABLE test_drop DROP c4
> ...insert more data...
> {code}
> Make a snapshot and save off a describe to backup table test_drop.
> Decide to restore the snapshot to a new cluster. First restore the schema
> from describe. (column c4 isn't there)
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text)
> {code}
> sstableload the snapshot data.
> Works in 2.1/2.2. Fails in 3.0+ with:
> {code}
> java.lang.RuntimeException: Unknown column c4 during deserialization
> java.lang.RuntimeException: Failed to list files in
> /var/folders/t4/rlc2b6450qbg92762l9l4mt8gn/T/dtest-3eKv_g/test/node1/data1_copy/ks/drop_one-bcef5280f11b11e5825a43f0253f18b5
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:53)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:544)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:104)
> Caused by: java.lang.RuntimeException: Unknown column c4 during
> deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:430)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$193(SSTableLoader.java:121)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$184(LogAwareFileLister.java:75)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
> at
> java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2965)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:77)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:49)
> ... 4 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (CASSANDRA-11414) dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208826#comment-15208826
]
Paulo Motta edited comment on CASSANDRA-11414 at 3/23/16 5:35 PM:
--
I'd say changing the stress write CL to TWO would fix this (we must keep the
read CL at ONE though)
was (Author: pauloricardomg):
I'd say changing the stress read and/or write CL to QUORUM would fix this.
> dtest failure in bootstrap_test.TestBootstrap.resumable_bootstrap_test
> --
>
> Key: CASSANDRA-11414
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11414
> Project: Cassandra
> Issue Type: Test
> Components: Testing
>Reporter: Philip Thompson
>Assignee: DS Test Eng
> Labels: dtest
>
> Stress is failing to read back all data. We can see this output from the
> stress read
> {code}
> java.io.IOException: Operation x0 on key(s) [314c384f304f4c325030]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> java.io.IOException: Operation x0 on key(s) [33383438363931353131]: Data
> returned was not validated
> at org.apache.cassandra.stress.Operation.error(Operation.java:138)
> at
> org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:116)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
> at
> org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:261)
> at
> org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:327)
> FAILURE
> {code}
> Started happening with build 1075. Does not appear flaky on CI.
> example failure:
> http://cassci.datastax.com/job/trunk_dtest/1076/testReport/bootstrap_test/TestBootstrap/resumable_bootstrap_test
> Failed on CassCI build trunk_dtest #1076
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (CASSANDRA-11416) No longer able to load old backups into new cluster if there was a dropped column
Jeremiah Jordan created CASSANDRA-11416:
---
Summary: No longer able to load old backups into new cluster if
there was a dropped column
Key: CASSANDRA-11416
URL: https://issues.apache.org/jira/browse/CASSANDRA-11416
Project: Cassandra
Issue Type: Bug
Reporter: Jeremiah Jordan
The following change to the sstableloader test works in 2.1/2.2 but fails in
3.0+
https://github.com/JeremiahDJordan/cassandra-dtest/commit/7dc66efb8d24239f0a488ec5a613240531aeb7db
{code}
CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text, c4
text)
...insert data...
ALTER TABLE test_drop DROP c4
...insert more data...
{code}
Make a snapshot and save off a describe to backup table test_drop.
Decide to restore the snapshot to a new cluster. First restore the schema
from describe. (column c4 isn't there)
{code}
CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text)
{code}
sstableload the snapshot data.
Works in 2.1/2.2. Fails in 3.0+ with:
{code}
java.lang.RuntimeException: Unknown column c4 during deserialization
java.lang.RuntimeException: Failed to list files in
/var/folders/t4/rlc2b6450qbg92762l9l4mt8gn/T/dtest-3eKv_g/test/node1/data1_copy/ks/drop_one-bcef5280f11b11e5825a43f0253f18b5
at
org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:53)
at
org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:544)
at
org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
at
org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:104)
Caused by: java.lang.RuntimeException: Unknown column c4 during deserialization
at
org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
at
org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:430)
at
org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$193(SSTableLoader.java:121)
at
org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$184(LogAwareFileLister.java:75)
at
java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
at
java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2965)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
at
org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:77)
at
org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:49)
... 4 more
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11416) No longer able to load backups into new cluster if there was a dropped column
[
https://issues.apache.org/jira/browse/CASSANDRA-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeremiah Jordan updated CASSANDRA-11416:
Summary: No longer able to load backups into new cluster if there was a
dropped column (was: No longer able to load old backups into new cluster if
there was a dropped column)
> No longer able to load backups into new cluster if there was a dropped column
> -
>
> Key: CASSANDRA-11416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11416
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremiah Jordan
> Fix For: 3.0.x, 3.x
>
>
> The following change to the sstableloader test works in 2.1/2.2 but fails in
> 3.0+
> https://github.com/JeremiahDJordan/cassandra-dtest/commit/7dc66efb8d24239f0a488ec5a613240531aeb7db
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text, c4
> text)
> ...insert data...
> ALTER TABLE test_drop DROP c4
> ...insert more data...
> {code}
> Make a snapshot and save off a describe to backup table test_drop.
> Decide to restore the snapshot to a new cluster. First restore the schema
> from describe. (column c4 isn't there)
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text)
> {code}
> sstableload the snapshot data.
> Works in 2.1/2.2. Fails in 3.0+ with:
> {code}
> java.lang.RuntimeException: Unknown column c4 during deserialization
> java.lang.RuntimeException: Failed to list files in
> /var/folders/t4/rlc2b6450qbg92762l9l4mt8gn/T/dtest-3eKv_g/test/node1/data1_copy/ks/drop_one-bcef5280f11b11e5825a43f0253f18b5
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:53)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:544)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:104)
> Caused by: java.lang.RuntimeException: Unknown column c4 during
> deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:430)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$193(SSTableLoader.java:121)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$184(LogAwareFileLister.java:75)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
> at
> java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2965)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:77)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:49)
> ... 4 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11415) dtest failure in jmx_test.TestJMX.cfhistograms_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208846#comment-15208846
]
Yuki Morishita commented on CASSANDRA-11415:
I think it is from CASSANDRA-9598 change. It added '-ea' to the script.
We need CASSANDRA-10859 backported to cassandra-2.1 and maybe 2.2.
> dtest failure in jmx_test.TestJMX.cfhistograms_test
> ---
>
> Key: CASSANDRA-11415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11415
> Project: Cassandra
> Issue Type: Test
>Reporter: Philip Thompson
>Assignee: DS Test Eng
> Labels: dtest
> Fix For: 2.1.x
>
>
> We are seeing the following stacktrace when running nodetool cfhistograms
> {code}
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.EstimatedHistogram.(EstimatedHistogram.java:66)
> at
> org.apache.cassandra.tools.NodeProbe.metricPercentilesAsArray(NodeProbe.java:1260)
> at
> org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:1109)
> at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:292)
> at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:206)
> {code}
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/437/testReport/jmx_test/TestJMX/cfhistograms_test
> Failed on CassCI build cassandra-2.1_dtest #437
> This doesn't appear to be flaky. I can repro locally. It seems like a product
> issue, but if someone could confirm if it happens out of the test, that would
> be great.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (CASSANDRA-11415) dtest failure in jmx_test.TestJMX.cfhistograms_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita reassigned CASSANDRA-11415:
--
Assignee: Yuki Morishita (was: DS Test Eng)
> dtest failure in jmx_test.TestJMX.cfhistograms_test
> ---
>
> Key: CASSANDRA-11415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11415
> Project: Cassandra
> Issue Type: Test
>Reporter: Philip Thompson
>Assignee: Yuki Morishita
> Labels: dtest
> Fix For: 2.1.x
>
>
> We are seeing the following stacktrace when running nodetool cfhistograms
> {code}
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.EstimatedHistogram.(EstimatedHistogram.java:66)
> at
> org.apache.cassandra.tools.NodeProbe.metricPercentilesAsArray(NodeProbe.java:1260)
> at
> org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:1109)
> at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:292)
> at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:206)
> {code}
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/437/testReport/jmx_test/TestJMX/cfhistograms_test
> Failed on CassCI build cassandra-2.1_dtest #437
> This doesn't appear to be flaky. I can repro locally. It seems like a product
> issue, but if someone could confirm if it happens out of the test, that would
> be great.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11415) dtest failure in jmx_test.TestJMX.cfhistograms_test
[
https://issues.apache.org/jira/browse/CASSANDRA-11415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuki Morishita updated CASSANDRA-11415:
---
Issue Type: Bug (was: Test)
> dtest failure in jmx_test.TestJMX.cfhistograms_test
> ---
>
> Key: CASSANDRA-11415
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11415
> Project: Cassandra
> Issue Type: Bug
>Reporter: Philip Thompson
>Assignee: Yuki Morishita
> Labels: dtest
> Fix For: 2.1.x
>
>
> We are seeing the following stacktrace when running nodetool cfhistograms
> {code}
> java.lang.AssertionError
> at
> org.apache.cassandra.utils.EstimatedHistogram.(EstimatedHistogram.java:66)
> at
> org.apache.cassandra.tools.NodeProbe.metricPercentilesAsArray(NodeProbe.java:1260)
> at
> org.apache.cassandra.tools.NodeTool$CfHistograms.execute(NodeTool.java:1109)
> at
> org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:292)
> at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:206)
> {code}
> example failure:
> http://cassci.datastax.com/job/cassandra-2.1_dtest/437/testReport/jmx_test/TestJMX/cfhistograms_test
> Failed on CassCI build cassandra-2.1_dtest #437
> This doesn't appear to be flaky. I can repro locally. It seems like a product
> issue, but if someone could confirm if it happens out of the test, that would
> be great.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11349) MerkleTree mismatch when multiple range tombstones exists for the same partition and interval
[
https://issues.apache.org/jira/browse/CASSANDRA-11349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208851#comment-15208851
]
Fabien Rousseau commented on CASSANDRA-11349:
-
Nice patch.
I will be able to test it on a dev environment either this week or at the
beginning of next week.
There is still one case not covered (though not sure it can happen).
Suppose that in SSTable 1, there is a range tombstone covering the columns "a"
through "g" at time t1, and in SSTable 2, there is a range tombstone covering
the columns "c" through "d" at time t2.
If those two SSTable are merged (for example on another replica), it will be
split in 3 range tombstones in one SSTable (one range tombstone "a" -> "b" at
t1, "c" -> "d" at t2, "e" -> "f" at t1).
Computing the merkle tree for those two hosts will still be different (not the
same range tombstones).
As said above, not sure if it can happen, and anyway, this patch is a good
improvement and probably fits 99% cases.
> MerkleTree mismatch when multiple range tombstones exists for the same
> partition and interval
> -
>
> Key: CASSANDRA-11349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11349
> Project: Cassandra
> Issue Type: Bug
>Reporter: Fabien Rousseau
>Assignee: Stefan Podkowinski
> Labels: repair
> Fix For: 2.1.x, 2.2.x
>
> Attachments: 11349-2.1.patch
>
>
> We observed that repair, for some of our clusters, streamed a lot of data and
> many partitions were "out of sync".
> Moreover, the read repair mismatch ratio is around 3% on those clusters,
> which is really high.
> After investigation, it appears that, if two range tombstones exists for a
> partition for the same range/interval, they're both included in the merkle
> tree computation.
> But, if for some reason, on another node, the two range tombstones were
> already compacted into a single range tombstone, this will result in a merkle
> tree difference.
> Currently, this is clearly bad because MerkleTree differences are dependent
> on compactions (and if a partition is deleted and created multiple times, the
> only way to ensure that repair "works correctly"/"don't overstream data" is
> to major compact before each repair... which is not really feasible).
> Below is a list of steps allowing to easily reproduce this case:
> {noformat}
> ccm create test -v 2.1.13 -n 2 -s
> ccm node1 cqlsh
> CREATE KEYSPACE test_rt WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 2};
> USE test_rt;
> CREATE TABLE IF NOT EXISTS table1 (
> c1 text,
> c2 text,
> c3 float,
> c4 float,
> PRIMARY KEY ((c1), c2)
> );
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 2);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> # now flush only one of the two nodes
> ccm node1 flush
> ccm node1 cqlsh
> USE test_rt;
> INSERT INTO table1 (c1, c2, c3, c4) VALUES ( 'a', 'b', 1, 3);
> DELETE FROM table1 WHERE c1 = 'a' AND c2 = 'b';
> ctrl ^d
> ccm node1 repair
> # now grep the log and observe that there was some inconstencies detected
> between nodes (while it shouldn't have detected any)
> ccm node1 showlog | grep "out of sync"
> {noformat}
> Consequences of this are a costly repair, accumulating many small SSTables
> (up to thousands for a rather short period of time when using VNodes, the
> time for compaction to absorb those small files), but also an increased size
> on disk.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11179) Parallel cleanup can lead to disk space exhaustion
[
https://issues.apache.org/jira/browse/CASSANDRA-11179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208855#comment-15208855
]
Carl Yeksigian commented on CASSANDRA-11179:
Looks good. Just a couple of comments:
- Would be nice to add a comment to {{parallelAllSSTableOperation}} explaining
that jobs = 0 means using all compactor threads, so that we remember to
propagate that to our argument explanations.
- Also, it's not clear what would happen if you specified a jobs higher than
the number of concurrent compactors. The expectation is probably that it would
override that selection, so either a warning or the inability to do that would
be helpful.
> Parallel cleanup can lead to disk space exhaustion
> --
>
> Key: CASSANDRA-11179
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11179
> Project: Cassandra
> Issue Type: Improvement
> Components: Compaction, Tools
>Reporter: Tyler Hobbs
>Assignee: Marcus Eriksson
> Labels: doc-impacting
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.x
>
>
> In CASSANDRA-5547, we made cleanup (among other things) run in parallel
> across multiple sstables. There have been reports on IRC of this leading to
> disk space exhaustion, because multiple sstables are (almost entirely)
> rewritten at the same time. This seems particularly problematic because
> cleanup is frequently run after a cluster is expanded due to low disk space.
> I'm not really familiar with how we perform free disk space checks now, but
> it sounds like we can make some improvements here. It would be good to
> reduce the concurrency of cleanup operations if there isn't enough free disk
> space to support this.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11362) dtest failure in snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address
[
https://issues.apache.org/jira/browse/CASSANDRA-11362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Philip Thompson resolved CASSANDRA-11362.
-
Resolution: Cannot Reproduce
http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/33/
Did not show up over 300 separate runs. Closing as Cannot Repro
> dtest failure in
> snitch_test.TestGossipingPropertyFileSnitch.test_prefer_local_reconnect_on_listen_address
> --
>
> Key: CASSANDRA-11362
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11362
> Project: Cassandra
> Issue Type: Test
>Reporter: Jim Witschey
>Assignee: Philip Thompson
> Labels: dtest
>
> This is a weird one:
> http://cassci.datastax.com/job/cassandra-2.2_dtest/540/testReport/snitch_test/TestGossipingPropertyFileSnitch/test_prefer_local_reconnect_on_listen_address
> Not sure why the keyspace isn't getting created. At first I thought it was
> expecting {{keyspace1}} but finding {{Keyspace1}}, but then the test would
> just never pass.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (CASSANDRA-11318) dtest failure in replication_test.SnitchConfigurationUpdateTest.test_rf_collapse_property_file_snitch
[ https://issues.apache.org/jira/browse/CASSANDRA-11318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson resolved CASSANDRA-11318. - Resolution: Cannot Reproduce Did not show up in 300 separate runs: http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/34/ Closing as Cannot Repro. > dtest failure in > replication_test.SnitchConfigurationUpdateTest.test_rf_collapse_property_file_snitch > - > > Key: CASSANDRA-11318 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11318 > Project: Cassandra > Issue Type: Test >Reporter: Jim Witschey >Assignee: Philip Thompson > Labels: dtest > > example failure: > http://cassci.datastax.com/job/cassandra-3.0_dtest/592/testReport/replication_test/SnitchConfigurationUpdateTest/test_rf_collapse_property_file_snitch > Failed on CassCI build cassandra-3.0_dtest #592 > The test failed on this line: > https://github.com/riptano/cassandra-dtest/blob/7a331cda807c96ae107b58017854f0e57996d8c3/replication_test.py#L567 > So, a node's expected move from one rack to another doesn't happen in the > allotted timeout time. This is the only flap I've seen. Maybe the thing to do > here is increase the timeout and keep an eye on it? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11225) dtest failure in consistency_test.TestAccuracy.test_simple_strategy_counters
[ https://issues.apache.org/jira/browse/CASSANDRA-11225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208875#comment-15208875 ] Philip Thompson commented on CASSANDRA-11225: - I ran this 300 times: http://cassci.datastax.com/view/Parameterized/job/parameterized_dtest_multiplexer/36/ > dtest failure in consistency_test.TestAccuracy.test_simple_strategy_counters > > > Key: CASSANDRA-11225 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11225 > Project: Cassandra > Issue Type: Test >Reporter: Russ Hatch >Assignee: Philip Thompson > Labels: dtest > > example failure: > http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/209/testReport/consistency_test/TestAccuracy/test_simple_strategy_counters > Failed on CassCI build cassandra-2.1_novnode_dtest #209 > error: "AssertionError: Failed to read value from sufficient number of nodes, > required 2 but got 1 - [574, 2]" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (CASSANDRA-11225) dtest failure in consistency_test.TestAccuracy.test_simple_strategy_counters
[ https://issues.apache.org/jira/browse/CASSANDRA-11225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Philip Thompson updated CASSANDRA-11225: Assignee: DS Test Eng (was: Philip Thompson) > dtest failure in consistency_test.TestAccuracy.test_simple_strategy_counters > > > Key: CASSANDRA-11225 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11225 > Project: Cassandra > Issue Type: Test >Reporter: Russ Hatch >Assignee: DS Test Eng > Labels: dtest > > example failure: > http://cassci.datastax.com/job/cassandra-2.1_novnode_dtest/209/testReport/consistency_test/TestAccuracy/test_simple_strategy_counters > Failed on CassCI build cassandra-2.1_novnode_dtest #209 > error: "AssertionError: Failed to read value from sufficient number of nodes, > required 2 but got 1 - [574, 2]" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11416) No longer able to load backups into new cluster if there was a dropped column
[
https://issues.apache.org/jira/browse/CASSANDRA-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208892#comment-15208892
]
Aleksey Yeschenko commented on CASSANDRA-11416:
---
I'm assuming you can work around it, at least for now, by adding and dropping
those columns manually in the new cluster?
> No longer able to load backups into new cluster if there was a dropped column
> -
>
> Key: CASSANDRA-11416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11416
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremiah Jordan
> Fix For: 3.0.x, 3.x
>
>
> The following change to the sstableloader test works in 2.1/2.2 but fails in
> 3.0+
> https://github.com/JeremiahDJordan/cassandra-dtest/commit/7dc66efb8d24239f0a488ec5a613240531aeb7db
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text, c4
> text)
> ...insert data...
> ALTER TABLE test_drop DROP c4
> ...insert more data...
> {code}
> Make a snapshot and save off a describe to backup table test_drop.
> Decide to restore the snapshot to a new cluster. First restore the schema
> from describe. (column c4 isn't there)
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text)
> {code}
> sstableload the snapshot data.
> Works in 2.1/2.2. Fails in 3.0+ with:
> {code}
> java.lang.RuntimeException: Unknown column c4 during deserialization
> java.lang.RuntimeException: Failed to list files in
> /var/folders/t4/rlc2b6450qbg92762l9l4mt8gn/T/dtest-3eKv_g/test/node1/data1_copy/ks/drop_one-bcef5280f11b11e5825a43f0253f18b5
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:53)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:544)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:104)
> Caused by: java.lang.RuntimeException: Unknown column c4 during
> deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:430)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$193(SSTableLoader.java:121)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$184(LogAwareFileLister.java:75)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
> at
> java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2965)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:77)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:49)
> ... 4 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (CASSANDRA-10643) Implement compaction for a specific token range
[ https://issues.apache.org/jira/browse/CASSANDRA-10643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishy Kasar updated CASSANDRA-10643: Attachment: 10643-trunk-REV01.txt > Implement compaction for a specific token range > --- > > Key: CASSANDRA-10643 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10643 > Project: Cassandra > Issue Type: Improvement > Components: Compaction >Reporter: Vishy Kasar >Assignee: Vishy Kasar > Attachments: 10643-trunk-REV01.txt > > > We see repeated cases in production (using LCS) where small number of users > generate a large number repeated updates or tombstones. Reading data of such > users brings in large amounts of data in to java process. Apart from the read > itself being slow for the user, the excessive GC affects other users as well. > Our solution so far is to move from LCS to SCS and back. This takes long and > is an over kill if the number of outliers is small. For such cases, we can > implement the point compaction of a token range. We make the nodetool compact > take a starting and ending token range and compact all the SSTables that fall > with in that range. We can refuse to compact if the number of sstables is > beyond a max_limit. > Example: > nodetool -st 3948291562518219268 -et 3948291562518219269 compact keyspace > table -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (CASSANDRA-11417) dtest failure in replication_test.SnitchConfigurationUpdateTest.test_rf_expand_gossiping_property_file_snitch_multi_dc
Philip Thompson created CASSANDRA-11417:
---
Summary: dtest failure in
replication_test.SnitchConfigurationUpdateTest.test_rf_expand_gossiping_property_file_snitch_multi_dc
Key: CASSANDRA-11417
URL: https://issues.apache.org/jira/browse/CASSANDRA-11417
Project: Cassandra
Issue Type: Test
Reporter: Philip Thompson
Assignee: DS Test Eng
Error is
{code}
Unknown table 'rf_test' in keyspace 'testing'
{code}
Just seems like a schema disagreement problem. Presumably we just need to have
the driver block until schema agreement.
example failure:
http://cassci.datastax.com/job/trunk_offheap_dtest/90/testReport/replication_test/SnitchConfigurationUpdateTest/test_rf_expand_gossiping_property_file_snitch_multi_dc
Failed on CassCI build trunk_offheap_dtest #90
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-11416) No longer able to load backups into new cluster if there was a dropped column
[
https://issues.apache.org/jira/browse/CASSANDRA-11416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15208905#comment-15208905
]
Jeremiah Jordan commented on CASSANDRA-11416:
-
Yes, you can work around it by replaying your original schema and all changes
in the new cluster. But that is a pain for automated backups, so we should
change this so it doesn't crash out. A snapshot+describe output should be
enough to restore a table.
> No longer able to load backups into new cluster if there was a dropped column
> -
>
> Key: CASSANDRA-11416
> URL: https://issues.apache.org/jira/browse/CASSANDRA-11416
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jeremiah Jordan
> Fix For: 3.0.x, 3.x
>
>
> The following change to the sstableloader test works in 2.1/2.2 but fails in
> 3.0+
> https://github.com/JeremiahDJordan/cassandra-dtest/commit/7dc66efb8d24239f0a488ec5a613240531aeb7db
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text, c4
> text)
> ...insert data...
> ALTER TABLE test_drop DROP c4
> ...insert more data...
> {code}
> Make a snapshot and save off a describe to backup table test_drop.
> Decide to restore the snapshot to a new cluster. First restore the schema
> from describe. (column c4 isn't there)
> {code}
> CREATE TABLE test_drop (key text PRIMARY KEY, c1 text, c2 text, c3 text)
> {code}
> sstableload the snapshot data.
> Works in 2.1/2.2. Fails in 3.0+ with:
> {code}
> java.lang.RuntimeException: Unknown column c4 during deserialization
> java.lang.RuntimeException: Failed to list files in
> /var/folders/t4/rlc2b6450qbg92762l9l4mt8gn/T/dtest-3eKv_g/test/node1/data1_copy/ks/drop_one-bcef5280f11b11e5825a43f0253f18b5
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:53)
> at
> org.apache.cassandra.db.lifecycle.LifecycleTransaction.getFiles(LifecycleTransaction.java:544)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.openSSTables(SSTableLoader.java:76)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.stream(SSTableLoader.java:165)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:104)
> Caused by: java.lang.RuntimeException: Unknown column c4 during
> deserialization
> at
> org.apache.cassandra.db.SerializationHeader$Component.toHeader(SerializationHeader.java:331)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.openForBatch(SSTableReader.java:430)
> at
> org.apache.cassandra.io.sstable.SSTableLoader.lambda$openSSTables$193(SSTableLoader.java:121)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.lambda$innerList$184(LogAwareFileLister.java:75)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
> at
> java.util.TreeMap$EntrySpliterator.forEachRemaining(TreeMap.java:2965)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.innerList(LogAwareFileLister.java:77)
> at
> org.apache.cassandra.db.lifecycle.LogAwareFileLister.list(LogAwareFileLister.java:49)
> ... 4 more
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
