[jira] [Updated] (CASSANDRA-17065) Introduce separate rate limiting settings for entire SSTable streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-17065: Change Category: Operability Complexity: Normal Status: Open (was: Triage Needed) > Introduce separate rate limiting settings for entire SSTable streaming > -- > > Key: CASSANDRA-17065 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17065 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode, Tool/nodetool >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > With the introduction of entire SSTable streaming, it is desirable to > introduce new settings for the rate limit for entire SSTable streaming > operations. > Currently, regular streaming and SSTable streaming are rate limited by the > same setting. However, zero-copy streaming reduces load in the JVM and we can > have a setting specifically for this use case. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435713#comment-17435713
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
Rebase and cleaning done successfully.
I cleaned Unit tests and distributed In-jvm tests, passing up to current trunk
state. I should finish updating the Python DTests for the new config
parameters' names and check for failures when the java upgrade tests are fixed.
Topic:
* the upgrade tests will be testing with the new names and config parameters
provided with the units for trunk. We have unit tests that check the backward
compatibility parsing. I guess we don't plan to run the upgrades also with the
old config?
I plan to publish the full patch for new round of review probably early next
week.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.x
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16349) SSTableLoader reports error when SSTable(s) do not have data for some nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-16349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435675#comment-17435675
]
Ekaterina Dimitrova commented on CASSANDRA-16349:
-
New [Jenkins run|
https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1252/] as
I pushed with wrong image yesterday, I am sorry about that.
> SSTableLoader reports error when SSTable(s) do not have data for some nodes
> ---
>
> Key: CASSANDRA-16349
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16349
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Serban Teodorescu
>Assignee: Serban Teodorescu
>Priority: Normal
> Fix For: 4.0.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Running SSTableLoader in verbose mode will show error(s) if there are node(s)
> that do not own any data from the SSTable(s). This can happen in at least 2
> cases:
> # SSTableLoader is used to stream backups while keeping the same token ranges
> # SSTable(s) are created with CQLSSTableWriter to match token ranges (this
> can bring better performance by using ZeroCopy streaming)
> Partial output of the SSTableLoader:
> {quote}ERROR 02:47:47,842 [Stream #fa8e73b0-3da5-11eb-9c47-c5d27ae8fe47]
> Remote peer /127.0.0.4:7000 failed stream session.
> ERROR 02:47:47,842 [Stream #fa8e73b0-3da5-11eb-9c47-c5d27ae8fe47] Remote peer
> /127.0.0.3:7000 failed stream session.
> progress: [/127.0.0.4:7000]0:0/1 100% [/127.0.0.3:7000]0:0/1 100%
> [/127.0.0.2:7000]0:7/7 100% [/127.0.0.1:7000]0:7/7 100% total: 100%
> 0.000KiB/s (avg: 1.611KiB/s)
> progress: [/127.0.0.4:7000]0:0/1 100% [/127.0.0.3:7000]0:0/1 100%
> [/127.0.0.2:7000]0:7/7 100% [/127.0.0.1:7000]0:7/7 100% total: 100%
> 0.000KiB/s (avg: 1.611KiB/s)
> progress: [/127.0.0.4:7000]0:0/1 100% [/127.0.0.3:7000]0:0/1 100%
> [/127.0.0.2:7000]0:7/7 100% [/127.0.0.1:7000]0:7/7 100% total: 100%
> 0.000KiB/s (avg: 1.515KiB/s)
> progress: [/127.0.0.4:7000]0:0/1 100% [/127.0.0.3:7000]0:0/1 100%
> [/127.0.0.2:7000]0:7/7 100% [/127.0.0.1:7000]0:7/7 100% total: 100%
> 0.000KiB/s (avg: 1.427KiB/s)
> {quote}
>
> Stack trace:
> {quote}java.util.concurrent.ExecutionException:
> org.apache.cassandra.streaming.StreamException: Stream failed
> at
> com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:552)
> at
> com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:533)
> at org.apache.cassandra.tools.BulkLoader.load(BulkLoader.java:99)
> at org.apache.cassandra.tools.BulkLoader.main(BulkLoader.java:49)
> Caused by: org.apache.cassandra.streaming.StreamException: Stream failed
> at
> org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:88)
> at
> com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1056)
> at
> com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30)
> at
> com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138)
> at
> com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:958)
> at
> com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:748)
> at
> org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:220)
> at
> org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:196)
> at
> org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:505)
> at
> org.apache.cassandra.streaming.StreamSession.complete(StreamSession.java:819)
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:595)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:189)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:844)
> {quote}
> To reproduce create a cluster with ccm with more nodes than the RF, put some
> data into it copy a SSTable and stream it.
>
> The error originates on the nodes, the following stack trace is shown in the
> logs:
> {quote}java.lang.IllegalStateException: Stream hasn't been read yet
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:507)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.getSize(CassandraIncomingFile.java:96)
> at
> org.apache.cassandra.streaming.StreamSession.receive(StreamSession.java:789)
> at
> org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:587)
> at
>
[jira] [Commented] (CASSANDRA-17064) dtest-api: Option to start nodes with blank gossip state
[ https://issues.apache.org/jira/browse/CASSANDRA-17064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435670#comment-17435670 ] Michael Semb Wever commented on CASSANDRA-17064: bq. dtest-api +1 > dtest-api: Option to start nodes with blank gossip state > > > Key: CASSANDRA-17064 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17064 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > dtest-api should permit starting nodes with neither regular gossip nor > artificially injected gossip, so that the Gossip state may be modified by the > tests from a blank slate -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17050) Upgrade tests fail with InvocationTargetException
[ https://issues.apache.org/jira/browse/CASSANDRA-17050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435668#comment-17435668 ] Michael Semb Wever commented on CASSANDRA-17050: bq. Additional necessary changes to dtest-api +1 And i reckon the "update SNAPSHOT" commit should be ninja'd to trunk. That was supposed to have happened in [2139b4c85e319b17afbdea2f653152d1e1895fc6|https://github.com/apache/cassandra-in-jvm-dtest-api/commit/2139b4c85e319b17afbdea2f653152d1e1895fc6]. > Upgrade tests fail with InvocationTargetException > - > > Key: CASSANDRA-17050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17050 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Urgent > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x, 4.x > > > Upgrade tests are currently failing due to the new dtest-api changes and > their integration with trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17050) Upgrade tests fail with InvocationTargetException
[ https://issues.apache.org/jira/browse/CASSANDRA-17050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435661#comment-17435661 ] Michael Semb Wever edited comment on CASSANDRA-17050 at 10/28/21, 9:16 PM: --- bq. I wonder if we should aim to reduce the friction for internal dependencies by using the apache snapshots repository for non-release builds. Not sure if somebody has a better idea, but today it is more than a little annoying to have to release the dtest-api before merging any change that uses it. The problem may become more common as we accumulate more sub-projects. Yes. This was discussed a little [here|https://issues.apache.org/jira/browse/CASSANDRA-16649?focusedCommentId=17341901=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17341901] and the comment after it. I'm not confident yet of any other approach but the "throwaway" commits introducing the snapshot repository in patches (which are akin to the circle "throwaway" commits). was (Author: michaelsembwever): bq. I wonder if we should aim to reduce the friction for internal dependencies by using the apache snapshots repository for non-release builds. Not sure if somebody has a better idea, but today it is more than a little annoying to have to release the dtest-api before merging any change that uses it. The problem may become more common as we accumulate more sub-projects. Yes. This was discuss a little [here|https://issues.apache.org/jira/browse/CASSANDRA-16649?focusedCommentId=17341901=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17341901] and the comment after it. I'm not confident yet of any other approach but the "throwaway" commits introducing the snapshot repository in patches (which are akin to the circle "throwaway" commits). > Upgrade tests fail with InvocationTargetException > - > > Key: CASSANDRA-17050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17050 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Urgent > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x, 4.x > > > Upgrade tests are currently failing due to the new dtest-api changes and > their integration with trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17050) Upgrade tests fail with InvocationTargetException
[ https://issues.apache.org/jira/browse/CASSANDRA-17050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435661#comment-17435661 ] Michael Semb Wever commented on CASSANDRA-17050: bq. I wonder if we should aim to reduce the friction for internal dependencies by using the apache snapshots repository for non-release builds. Not sure if somebody has a better idea, but today it is more than a little annoying to have to release the dtest-api before merging any change that uses it. The problem may become more common as we accumulate more sub-projects. Yes. This was discuss a little [here|https://issues.apache.org/jira/browse/CASSANDRA-16649?focusedCommentId=17341901=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17341901] and the comment after it. I'm not confident yet of any other approach but the "throwaway" commits introducing the snapshot repository in patches (which are akin to the circle "throwaway" commits). > Upgrade tests fail with InvocationTargetException > - > > Key: CASSANDRA-17050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17050 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Urgent > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x, 4.x > > > Upgrade tests are currently failing due to the new dtest-api changes and > their integration with trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17031) Add support for PEM based key material for SSL
[
https://issues.apache.org/jira/browse/CASSANDRA-17031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435648#comment-17435648
]
Maulin Vasavada commented on CASSANDRA-17031:
-
Nope. Not urgent. Hope you get some relaxing time over the weekend before
starting another week :)
On Thu, Oct 28, 2021 at 12:06 AM Stefan Miklosovic (Jira)
> Add support for PEM based key material for SSL
> --
>
> Key: CASSANDRA-17031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17031
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Maulin Vasavada
>Assignee: Maulin Vasavada
>Priority: Normal
>
> h1. Scope
> Currently Cassandra supports standard keystore types for SSL
> keys/certificates. The scope of this enhancement is to add support for PEM
> based key material (keys/certificate) given that PEM is widely used common
> format for the same. We intend to add support for Unencrypted and Password
> Based Encrypted (PBE) PKCS#8 formatted Private Keys in PEM format with
> standard algorithms (RSA, DSA and EC) along with the certificate chain for
> the private key and PEM based X509 certificates. The work here is going to be
> built on top of [CEP-9: Make SSLContext creation
> pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable]
> for which the code is merged for Apache Cassandra 4.1 release.
> We intend to support the key material be configured as direct PEM values
> input OR via the file (configured with keystore and truststore configurations
> today). We are not going to model PEM as a valid 'store_type' given that
> 'store_type' has a [specific
> definition|https://docs.oracle.com/en/java/javase/11/security/java-cryptography-architecture-jca-reference-guide.html#GUID-AB51DEFD-5238-4F96-967F-082F6D34FBEA].
>
> h1. Approach
> Create an implementation for
> [ISslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/ISslContextFactory.java]
> extending
> [FileBasedSslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java]
> implementation to add PEM formatted key/certificates.
> h1. Motivation
> PEM is a widely used format for encoding Private Keys and X.509 Certificates
> and Apache Cassandra's current implementation lacks the support for
> specifying the PEM formatted key material for SSL configurations. This means
> operators have to re-create the key material to comply to the supported
> formats (using key/trust store types - jks, pkcs12 etc) and deal with an
> operational task for managing it. This is an operational overhead we can
> avoid by supporting the PEM format making Apache Cassandra even more customer
> friendly and drive more adoption.
> h1. Proposed Changes
> # A new implementation for ISslContextFactory - PEMBasedSslContextFactory
> with the following supported configuration
> {panel:title=New configurations}
> {panel}
> |{{encryption_options: }}
> {{}}{{ssl_context_factory:}}
> {{}}{{class_name:
> org.apache.cassandra.security.PEMBasedSslContextFactory}}
> {{}}{{parameters:}}
> {{ }}{{private_key: certificate chain>}}
> {{ }}{{private_key_password: }}{{private}} {{key }}{{if}} {{it is encrypted>}}
> {{ }}{{trusted_certificates: }}|
> *NOTE:* We could reuse 'keystore_password' instead of the
> 'private_key_password'. However PEM encoded private key is not a 'keystore'
> in itself hence it would be inappropriate to piggyback on that other than
> avoid duplicating similar fields.
> # The PEMBasedSslContextFactory will also support file based key material
> (and the corresponding HOT Reloading based on file timestamp updates) for the
> PEM format via existing 'keystore' and 'truststore' encryption options.
> However in that case the 'truststore_password' configuration won't be used
> since generally PEM formatted certificates for truststore don't get encrypted
> with a password.
> # The PEMBasedSslContextFactory will internally create PKCS12 keystore for
> private key and the trusted certificates. However, this doesn't impact the
> user of the implementation in anyway and it is mentioned for clarity only.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17082) Make nodes more resilient to stale JSON files during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17082: -- Test and Documentation Plan: No documentation change needed Status: Patch Available (was: Open) > Make nodes more resilient to stale JSON files during startup > > > Key: CASSANDRA-17082 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17082 > Project: Cassandra > Issue Type: Improvement > Components: Local/Startup and Shutdown >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > Fix For: 4.x > > > There's a few things we can protect against being in the data dir on startup > that might be around from older activity, tool usage, exports, etc on a 3.x > -> 4.x update: > a) file ending with *-old.json > b) file ending with *.json or *idx.json > A trivial update to the filter on SSTableHeaderFix.java should protect > against hitting these types of files on startup and throwing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17082) Make nodes more resilient to stale JSON files during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435642#comment-17435642 ] Josh McKenzie commented on CASSANDRA-17082: --- [Branch|https://github.com/apache/cassandra/compare/trunk...josh-mckenzie:cassandra-17082?expand=1] Super trivial and changes the filter on processFileOrDirectory to tolerate failed Descriptor.fromFilenameWithComponent calls; not sure it's worth the resources to run the test suite against this (and get the barrage of known failures... /grumble) > Make nodes more resilient to stale JSON files during startup > > > Key: CASSANDRA-17082 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17082 > Project: Cassandra > Issue Type: Improvement > Components: Local/Startup and Shutdown >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > Fix For: 4.x > > > There's a few things we can protect against being in the data dir on startup > that might be around from older activity, tool usage, exports, etc on a 3.x > -> 4.x update: > a) file ending with *-old.json > b) file ending with *.json or *idx.json > A trivial update to the filter on SSTableHeaderFix.java should protect > against hitting these types of files on startup and throwing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17082) Make nodes more resilient to stale JSON files during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17082: -- Description: There's a few things we can protect against being in the data dir on startup that might be around from older activity, tool usage, exports, etc on a 3.x -> 4.x update: a) file ending with *-old.json b) file ending with *.json or *idx.json A trivial update to the filter on SSTableHeaderFix.java should protect against hitting these types of files on startup and throwing. was: There's a few things we can protect against being in the data dir on startup that might be around from older activity, tool usage, exports, etc: a) file ending with *-old.json b) file ending with *.json or *idx.json A trivial update to the filter on SSTableHeaderFix.java should protect against hitting these types of files on startup and throwing. > Make nodes more resilient to stale JSON files during startup > > > Key: CASSANDRA-17082 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17082 > Project: Cassandra > Issue Type: Improvement > Components: Local/Startup and Shutdown >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > Fix For: 4.x > > > There's a few things we can protect against being in the data dir on startup > that might be around from older activity, tool usage, exports, etc on a 3.x > -> 4.x update: > a) file ending with *-old.json > b) file ending with *.json or *idx.json > A trivial update to the filter on SSTableHeaderFix.java should protect > against hitting these types of files on startup and throwing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17082) Make nodes more resilient to stale JSON files during startup
[ https://issues.apache.org/jira/browse/CASSANDRA-17082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17082: -- Change Category: Operability Complexity: Low Hanging Fruit Fix Version/s: 4.x Status: Open (was: Triage Needed) > Make nodes more resilient to stale JSON files during startup > > > Key: CASSANDRA-17082 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17082 > Project: Cassandra > Issue Type: Improvement > Components: Local/Startup and Shutdown >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > Fix For: 4.x > > > There's a few things we can protect against being in the data dir on startup > that might be around from older activity, tool usage, exports, etc: > a) file ending with *-old.json > b) file ending with *.json or *idx.json > A trivial update to the filter on SSTableHeaderFix.java should protect > against hitting these types of files on startup and throwing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17082) Make nodes more resilient to stale JSON files during startup
Josh McKenzie created CASSANDRA-17082: - Summary: Make nodes more resilient to stale JSON files during startup Key: CASSANDRA-17082 URL: https://issues.apache.org/jira/browse/CASSANDRA-17082 Project: Cassandra Issue Type: Improvement Components: Local/Startup and Shutdown Reporter: Josh McKenzie Assignee: Josh McKenzie There's a few things we can protect against being in the data dir on startup that might be around from older activity, tool usage, exports, etc: a) file ending with *-old.json b) file ending with *.json or *idx.json A trivial update to the filter on SSTableHeaderFix.java should protect against hitting these types of files on startup and throwing. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17069) Refactor normal/preview/IR repair to standardize repair cleanup and error handling of failed RepairJobs
[ https://issues.apache.org/jira/browse/CASSANDRA-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-17069: -- Test and Documentation Plan: added tests and rely on existing ones Status: Patch Available (was: In Progress) > Refactor normal/preview/IR repair to standardize repair cleanup and error > handling of failed RepairJobs > --- > > Key: CASSANDRA-17069 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17069 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.x > > > Right now we have 3 different implementations of repair: normal, preview, and > incremental (IR); all 3 handle RepairJob failures differently and offer > different state cleanup. To make sure that we consistently handle errors the > same way and cleanup, we should move these responsibilities outside of the > repair task itself and move these into common APIs and move some logic into > the repair coordination its self. > This work relates with CASSANDRA-15399 as special handling each task makes > the work more complex. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435635#comment-17435635
]
Josh McKenzie commented on CASSANDRA-12106:
---
Ok, quite a few commits back and forth in on that PR and it's pretty tidied up.
I'll pull a comment I left there here as it's the one outstanding problem with
the design we're not going to fix on this initial commit (it's a tradeoff):
{quote}The last outstanding thing I can't think of a good solution to is what
to do if a node is gc thrashing, read and write CQL stages are a mess, and
denylisting requires CQL mutation to work. While we _could_ append a new deny
listed key to the DenylistEntry's in the cache independently of, or before, the
CQL mutation that adds it to the dedicated store for the denylist, that means
we'll likely have a disjoint between what's deny listed in our cache vs. what's
in CQL and makes the in-memory cache the authority vs. what's in the DB. I'm
actually somewhat receptive to that, since the JMX call queries what's in the
cache and actively denylisting, but I think we should probably leave that up to
a follow-up ticket rather than trying to tackle that on the tail end of this...
marathon.
{quote}
The tests are kind of a mess:
[JDK
8|https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/134/workflows/2cd807fb-2393-4fd5-89c8-aa5c71eef491]
[JDK
11|https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/134/workflows/3c36936f-97c7-4d08-9383-e9afc476fd36]
Here's where I've gotten on debugging the failing test front:
* j8 + 11 dtest no vnodes: test_bootstrap_with_reset_bootstrap_state: Created
CASSANDRA-17076; happening on trunk
* j8 + 11 dtest with vnodes: test_bootstrap_binary_disabled: Created
CASSANDRA-17077; happening on trunk
* j8 unit: testPersistentStatistics: Created CASSANDRA-17078; can't reproduce
locally on branch or trunk
* j8 dtest novnode:
** test_rolling_upgrade_with_internode_ssl x3
** test_rolling_upgrade x2: Created CASSANDRA-17079; also happening on trunk
** test_drop_compact_storage_mixed_cluster: Created CASSANDRA-17080
** test_bootstrap_with_reset_bootstrap_state: Created CASSANDRA-17081
* j8 jvm upgrade: InvocationTargetException - being fixed in CASSANDRA-17050
* j11 unit: testNoTreesRetainedAfterDifference - happening on trunk, tracked
in CASSANDRA-17039
I'm going to give it a couple days for us to talk through what's going on w/the
testing situation, maybe see CASSANDRA-17050 cleared out, before merging this
in. I also want to read through {{MigrationManager.evolveSystemKeyspace}} again
and think about the behavior in mixed version cluster states for generation
changes in SystemDistributed (have some other changes queued for that in the
auth domain as well).
> Add ability to blocklist / denylist a CQL partition so all requests are
> ignored
> ---
>
> Key: CASSANDRA-12106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12106
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/Local Write-Read Paths, Local/Config
>Reporter: Geoffrey Yu
>Assignee: Josh McKenzie
>Priority: Low
> Fix For: 4.x
>
> Attachments: 12106-trunk.txt
>
>
> Sometimes reads/writes to a given partition may cause problems due to the
> data present. It would be useful to have a manual way to blocklist / denylist
> such partitions so all read and write requests to them are rejected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17081) Fix test: bootstrap_test.py::TestBootstrap::test_bootstrap_with_reset_bootstrap_state
Josh McKenzie created CASSANDRA-17081:
-
Summary: Fix test:
bootstrap_test.py::TestBootstrap::test_bootstrap_with_reset_bootstrap_state
Key: CASSANDRA-17081
URL: https://issues.apache.org/jira/browse/CASSANDRA-17081
Project: Cassandra
Issue Type: Bug
Reporter: Josh McKenzie
Seeing in circle and locally on trunk:
Looks like it's timing out waiting for the bootstrap to complete.
{code:java}
test_bootstrap_with_reset_bootstrap_state failed (1 runs remaining out of 2).
28 Oct 2021 19:03:53 [node3] after 120.39/120 seconds Missing:
['127.0.0.1:7000.* is now UP'] not found in system.log:
Head: ERROR [Stream-Deserializer-/127.0.0.1:7000-20b885c
Tail: ...b336de0e72/nb-1-big-Data.db
ERROR [Stream-Deserializer-/127.0.0.1:7000-29a7cdb5] 2021-10-28 15:01:36,578
StorageService.java:483 - Stopping gossiper
[, , , , ]
test_bootstrap_with_reset_bootstrap_state failed; it passed 0 out of the
required 1 times.
28 Oct 2021 19:08:23 [node3] after 120.41/120 seconds Missing:
['127.0.0.1:7000.* is now UP'] not found in system.log:
Head:
Tail: ...
[, , , , ]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435630#comment-17435630
]
Ekaterina Dimitrova commented on CASSANDRA-17070:
-
+1 on the proposed change to use a super class similar to #16777, it was on my
list for some time but it was a lower priority. Honestly, we should have done
it at first place like that. Thanks [~adelapena]!
> ViewComplexTest hardening
> -
>
> Key: CASSANDRA-17070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17070
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> I have seen a number of times already the {{ViewComplexTest}} family timeout
> on test method teardown. This leaves a dirty env behind triggering the
> following test methods to fail on it. This ticket aims at hardening them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17080) Fix test: dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test.TestDropCompactStorage.test_drop_compact_storage_mixed_cluster
[
https://issues.apache.org/jira/browse/CASSANDRA-17080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-17080:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
> Fix test:
> dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test.TestDropCompactStorage.test_drop_compact_storage_mixed_cluster
> --
>
> Key: CASSANDRA-17080
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17080
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
>
> !https://ci-cassandra.apache.org/static/a177fe56/images/32x32/health-80plus.png!
> Failed 28 times in the last 28 runs. Flakiness: 0%, Stability: 0%
>
> Example of failure:
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/801/testReport/junit/dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test/TestDropCompactStorage/test_drop_compact_storage_mixed_cluster/]
>
> {code:java}
> upgrade_tests/drop_compact_storage_upgrade_test.py:149:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> object at 0x7fa0e7f1ceb0>
> session =
> assert_msg = 'Cannot DROP COMPACT STORAGE as some nodes in the cluster
> ([/127.0.0.2:7000, /127.0.0.1:7000]) are not on 4.0+ yet. Please upgrade
> those nodes and run `upgradesstables` before retrying.'
> def drop_compact_storage(self, session, assert_msg):
> try:
> session.execute("ALTER TABLE drop_compact_storage_test.test DROP
> COMPACT STORAGE")
> pytest.fail("No exception has been thrown")
> except InvalidRequest as e:
> > assert assert_msg in str(e)
> E assert 'Cannot DROP COMPACT STORAGE as some nodes in the cluster
> ([/127.0.0.2:7000, /127.0.0.1:7000]) are not on 4.0+ yet. Please upgrade
> those nodes and run `upgradesstables` before retrying.' in 'Error from
> server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as
> some nodes in the cluster ([/1271:7000, /127.0.0.2:7000]) are not on 4.0+
> yet. Please upgrade those nodes and run `upgradesstables` before retrying."'
> E+ where 'Error from server: code=2200 [Invalid query]
> message="Cannot DROP COMPACT STORAGE as some nodes in the cluster
> ([/1271:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those
> nodes and run `upgradesstables` before retrying."' =
> str(InvalidRequest('Error from server: code=2200 [Invalid query]
> message="Cannot DROP COMPACT STORAGE as some nodes in the...1:7000,
> /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those nodes and run
> `upgradesstables` before retrying."'))
> upgrade_tests/drop_compact_storage_upgrade_test.py:45: AssertionError
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17080) Fix test: dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test.TestDropCompactStorage.test_drop_compact_storage_mixed_cluster
Josh McKenzie created CASSANDRA-17080:
-
Summary: Fix test:
dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test.TestDropCompactStorage.test_drop_compact_storage_mixed_cluster
Key: CASSANDRA-17080
URL: https://issues.apache.org/jira/browse/CASSANDRA-17080
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/python
Reporter: Josh McKenzie
!https://ci-cassandra.apache.org/static/a177fe56/images/32x32/health-80plus.png!
Failed 28 times in the last 28 runs. Flakiness: 0%, Stability: 0%
Example of failure:
[https://ci-cassandra.apache.org/job/Cassandra-trunk/801/testReport/junit/dtest-upgrade.upgrade_tests.drop_compact_storage_upgrade_test/TestDropCompactStorage/test_drop_compact_storage_mixed_cluster/]
{code:java}
upgrade_tests/drop_compact_storage_upgrade_test.py:149:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
session =
assert_msg = 'Cannot DROP COMPACT STORAGE as some nodes in the cluster
([/127.0.0.2:7000, /127.0.0.1:7000]) are not on 4.0+ yet. Please upgrade those
nodes and run `upgradesstables` before retrying.'
def drop_compact_storage(self, session, assert_msg):
try:
session.execute("ALTER TABLE drop_compact_storage_test.test DROP
COMPACT STORAGE")
pytest.fail("No exception has been thrown")
except InvalidRequest as e:
> assert assert_msg in str(e)
E assert 'Cannot DROP COMPACT STORAGE as some nodes in the cluster
([/127.0.0.2:7000, /127.0.0.1:7000]) are not on 4.0+ yet. Please upgrade those
nodes and run `upgradesstables` before retrying.' in 'Error from server:
code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as some nodes in
the cluster ([/1271:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please
upgrade those nodes and run `upgradesstables` before retrying."'
E+ where 'Error from server: code=2200 [Invalid query]
message="Cannot DROP COMPACT STORAGE as some nodes in the cluster
([/1271:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please upgrade those
nodes and run `upgradesstables` before retrying."' = str(InvalidRequest('Error
from server: code=2200 [Invalid query] message="Cannot DROP COMPACT STORAGE as
some nodes in the...1:7000, /127.0.0.2:7000]) are not on 4.0+ yet. Please
upgrade those nodes and run `upgradesstables` before retrying."'))
upgrade_tests/drop_compact_storage_upgrade_test.py:45: AssertionError
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17079) Fix test failure: dtest-upgrade.upgrade_tests.upgrade_through_versions_test.TestProtoV3Upgrade_AllVersions_RandomPartitioner_EndsAt_3_11_X_HEAD.test_rolling_upgrade
Josh McKenzie created CASSANDRA-17079:
-
Summary: Fix test failure:
dtest-upgrade.upgrade_tests.upgrade_through_versions_test.TestProtoV3Upgrade_AllVersions_RandomPartitioner_EndsAt_3_11_X_HEAD.test_rolling_upgrade
Key: CASSANDRA-17079
URL: https://issues.apache.org/jira/browse/CASSANDRA-17079
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/python
Reporter: Josh McKenzie
!https://ci-cassandra.apache.org/static/a177fe56/images/32x32/health-80plus.png!
Failed 28 times in the last 28 runs. Flakiness: 0%, Stability: 0%
Example failure:
https://ci-cassandra.apache.org/job/Cassandra-trunk/801/testReport/junit/dtest-upgrade.upgrade_tests.upgrade_through_versions_test/TestProtoV3Upgrade_AllVersions_RandomPartitioner_EndsAt_3_11_X_HEAD/test_rolling_upgrade/
{code:java}
self =
@pytest.mark.timeout(3000)
def test_rolling_upgrade(self):
"""
Test rolling upgrade of the cluster, so we have mixed versions part
way through.
"""
> self.upgrade_scenario(rolling=True)
upgrade_tests/upgrade_through_versions_test.py:320:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
upgrade_tests/upgrade_through_versions_test.py:398: in upgrade_scenario
self._check_on_subprocs(self.fixture_dtest_setup.subprocs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
subprocs = [, ]
def _check_on_subprocs(self, subprocs):
"""
Check on given subprocesses.
If any are not alive, we'll go ahead and terminate any remaining
alive subprocesses since this test is going to fail.
"""
subproc_statuses = [s.is_alive() for s in subprocs]
if not all(subproc_statuses):
message = "A subprocess has terminated early. Subprocess statuses: "
for s in subprocs:
message += "{name} (is_alive: {aliveness}),
".format(name=s.name, aliveness=s.is_alive())
message += "attempting to terminate remaining subprocesses now."
self._terminate_subprocs()
> raise RuntimeError(message)
E RuntimeError: A subprocess has terminated early. Subprocess
statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting
to terminate remaining subprocesses now.
upgrade_tests/upgrade_through_versions_test.py:456: RuntimeError
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17078) Fix failing test: SSTableReaderTest.testPersistentStatistics
[
https://issues.apache.org/jira/browse/CASSANDRA-17078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-17078:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
> Fix failing test: SSTableReaderTest.testPersistentStatistics
>
>
> Key: CASSANDRA-17078
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17078
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Priority: Normal
>
> JDK8 unit test failure
> See it on CircleCI but not on Jenkins ASF infra right now.
> {code:java}
> java.lang.RuntimeException: Failed importing sstables
> at
> org.apache.cassandra.db.SSTableImporter.importNewSSTables(SSTableImporter.java:164)
> at
> org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:734)
> at
> org.apache.cassandra.io.sstable.SSTableReaderTest.clearAndLoad(SSTableReaderTest.java:222)
> at
> org.apache.cassandra.io.sstable.SSTableReaderTest.testPersistentStatistics(SSTableReaderTest.java:215)
> Caused by: java.lang.RuntimeException: Failed to rename
> /tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-9-big-Digest.crc32
> to
> /tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-13-big-Digest.crc32
> at org.apache.cassandra.io.util.PathUtils.rename(PathUtils.java:385)
> at org.apache.cassandra.io.util.File.move(File.java:227)
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:704)
> at
> org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:698)
> at
> org.apache.cassandra.io.sstable.format.SSTableWriter.rename(SSTableWriter.java:337)
> at
> org.apache.cassandra.io.sstable.format.SSTableReader.moveAndOpenSSTable(SSTableReader.java:2339)
> at
> org.apache.cassandra.db.SSTableImporter.importNewSSTables(SSTableImporter.java:139)
> Caused by: java.nio.file.NoSuchFileException:
> /tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-9-big-Digest.crc32
> ->
> /tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-13-big-Digest.crc32
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
> at
> sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
> at java.nio.file.Files.move(Files.java:1395)
> at
> org.apache.cassandra.io.util.PathUtils.atomicMoveWithFallback(PathUtils.java:396)
> at org.apache.cassandra.io.util.PathUtils.rename(PathUtils.java:377)
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17078) Fix failing test: SSTableReaderTest.testPersistentStatistics
Josh McKenzie created CASSANDRA-17078:
-
Summary: Fix failing test:
SSTableReaderTest.testPersistentStatistics
Key: CASSANDRA-17078
URL: https://issues.apache.org/jira/browse/CASSANDRA-17078
Project: Cassandra
Issue Type: Bug
Components: Test/unit
Reporter: Josh McKenzie
JDK8 unit test failure
See it on CircleCI but not on Jenkins ASF infra right now.
{code:java}
java.lang.RuntimeException: Failed importing sstables
at
org.apache.cassandra.db.SSTableImporter.importNewSSTables(SSTableImporter.java:164)
at
org.apache.cassandra.db.ColumnFamilyStore.loadNewSSTables(ColumnFamilyStore.java:734)
at
org.apache.cassandra.io.sstable.SSTableReaderTest.clearAndLoad(SSTableReaderTest.java:222)
at
org.apache.cassandra.io.sstable.SSTableReaderTest.testPersistentStatistics(SSTableReaderTest.java:215)
Caused by: java.lang.RuntimeException: Failed to rename
/tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-9-big-Digest.crc32
to
/tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-13-big-Digest.crc32
at org.apache.cassandra.io.util.PathUtils.rename(PathUtils.java:385)
at org.apache.cassandra.io.util.File.move(File.java:227)
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:704)
at
org.apache.cassandra.io.util.FileUtils.renameWithConfirm(FileUtils.java:698)
at
org.apache.cassandra.io.sstable.format.SSTableWriter.rename(SSTableWriter.java:337)
at
org.apache.cassandra.io.sstable.format.SSTableReader.moveAndOpenSSTable(SSTableReader.java:2339)
at
org.apache.cassandra.db.SSTableImporter.importNewSSTables(SSTableImporter.java:139)
Caused by: java.nio.file.NoSuchFileException:
/tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-9-big-Digest.crc32
->
/tmp/cassandra/build/test/cassandra/data/SSTableReaderTest/Standard1-da581e50380611ecaad41173773221f9/nb-13-big-Digest.crc32
at
sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:396)
at
sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:262)
at java.nio.file.Files.move(Files.java:1395)
at
org.apache.cassandra.io.util.PathUtils.atomicMoveWithFallback(PathUtils.java:396)
at org.apache.cassandra.io.util.PathUtils.rename(PathUtils.java:377)
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17069) Refactor normal/preview/IR repair to standardize repair cleanup and error handling of failed RepairJobs
[ https://issues.apache.org/jira/browse/CASSANDRA-17069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435590#comment-17435590 ] David Capwell commented on CASSANDRA-17069: --- patch is almost ready for review; finishing testing. > Refactor normal/preview/IR repair to standardize repair cleanup and error > handling of failed RepairJobs > --- > > Key: CASSANDRA-17069 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17069 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.x > > > Right now we have 3 different implementations of repair: normal, preview, and > incremental (IR); all 3 handle RepairJob failures differently and offer > different state cleanup. To make sure that we consistently handle errors the > same way and cleanup, we should move these responsibilities outside of the > repair task itself and move these into common APIs and move some logic into > the repair coordination its self. > This work relates with CASSANDRA-15399 as special handling each task makes > the work more complex. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17077) Fix failing test: dtest.bootstrap_test.TestBootstrap.test_bootstrap_binary_disabled
Josh McKenzie created CASSANDRA-17077:
-
Summary: Fix failing test:
dtest.bootstrap_test.TestBootstrap.test_bootstrap_binary_disabled
Key: CASSANDRA-17077
URL: https://issues.apache.org/jira/browse/CASSANDRA-17077
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/python
Reporter: Josh McKenzie
JDK8 and JDK11
[https://ci-cassandra.apache.org/job/Cassandra-trunk/801/testReport/junit/dtest.bootstrap_test/TestBootstrap/test_bootstrap_binary_disabled/]
{code:java}
bootstrap_test.py:1014:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../venv/lib/python3.8/site-packages/ccmlib/node.py:1005: in nodetool
return handle_external_tool_process(p, ['nodetool', '-h', 'localhost',
'-p', str(self.jmx_port)] + shlex.split(cmd))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
process =
cmd_args = ['nodetool', '-h', 'localhost', '-p', '7300', 'bootstrap', ...]
def handle_external_tool_process(process, cmd_args):
out, err = process.communicate()
if (out is not None) and isinstance(out, bytes):
out = out.decode()
if (err is not None) and isinstance(err, bytes):
err = err.decode()
rc = process.returncode
if rc != 0:
> raise ToolError(cmd_args, rc, out, err)
E ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost',
'-p', '7300', 'bootstrap', 'resume'] exited with non-zero status; exit status:
1;
E stderr: nodetool: Failed to connect to 'localhost:7300' -
ConnectException: 'Connection refused (Connection refused)'.
../venv/lib/python3.8/site-packages/ccmlib/node.py:2305: ToolError
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17077) Fix failing test: dtest.bootstrap_test.TestBootstrap.test_bootstrap_binary_disabled
[
https://issues.apache.org/jira/browse/CASSANDRA-17077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-17077:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
> Fix failing test:
> dtest.bootstrap_test.TestBootstrap.test_bootstrap_binary_disabled
> ---
>
> Key: CASSANDRA-17077
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17077
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
>
> JDK8 and JDK11
> [https://ci-cassandra.apache.org/job/Cassandra-trunk/801/testReport/junit/dtest.bootstrap_test/TestBootstrap/test_bootstrap_binary_disabled/]
>
> {code:java}
> bootstrap_test.py:1014:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../venv/lib/python3.8/site-packages/ccmlib/node.py:1005: in nodetool
> return handle_external_tool_process(p, ['nodetool', '-h', 'localhost',
> '-p', str(self.jmx_port)] + shlex.split(cmd))
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> process =
> cmd_args = ['nodetool', '-h', 'localhost', '-p', '7300', 'bootstrap', ...]
> def handle_external_tool_process(process, cmd_args):
> out, err = process.communicate()
> if (out is not None) and isinstance(out, bytes):
> out = out.decode()
> if (err is not None) and isinstance(err, bytes):
> err = err.decode()
> rc = process.returncode
>
> if rc != 0:
> > raise ToolError(cmd_args, rc, out, err)
> E ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost',
> '-p', '7300', 'bootstrap', 'resume'] exited with non-zero status; exit
> status: 1;
> E stderr: nodetool: Failed to connect to 'localhost:7300' -
> ConnectException: 'Connection refused (Connection refused)'.
> ../venv/lib/python3.8/site-packages/ccmlib/node.py:2305: ToolError
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17076) Fix failing test: test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap
[
https://issues.apache.org/jira/browse/CASSANDRA-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-17076:
--
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
> Fix failing test: test_bootstrap_with_reset_bootstrap_state -
> bootstrap_test.TestBootstrap
> --
>
> Key: CASSANDRA-17076
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17076
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Josh McKenzie
>Priority: Normal
>
> Both JDK8 and JDK11
> Times out
> {code:java}
> bootstrap_test.py:483:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:895: in start
> node.watch_log_for_alive(self, from_mark=mark)
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:664: in
> watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:592: in watch_log_for
> head=reads[:50], tail="..."+reads[len(reads)-150:]))
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> start = 1635437046.9280019, timeout = 120
> msg = "Missing: ['127.0.0.1:7000.* is now UP'] not found in system.log:\n
> Head: \n Tail: ..."
> node = 'node3'
> @staticmethod
> def raise_if_passed(start, timeout, msg, node=None):
> if start + timeout < time.time():
> > raise TimeoutError.create(start, timeout, msg, node)
> E ccmlib.node.TimeoutError: 28 Oct 2021 16:06:07 [node3] after
> 120.12/120 seconds Missing: ['127.0.0.1:7000.* is now UP'] not found in
> system.log:
> EHead:
> ETail: ...
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:56: TimeoutError
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17076) Fix failing test: test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap
Josh McKenzie created CASSANDRA-17076:
-
Summary: Fix failing test:
test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap
Key: CASSANDRA-17076
URL: https://issues.apache.org/jira/browse/CASSANDRA-17076
Project: Cassandra
Issue Type: Bug
Components: Test/dtest/python
Reporter: Josh McKenzie
Both JDK8 and JDK11
Times out
{code:java}
bootstrap_test.py:483:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../env3.6/lib/python3.6/site-packages/ccmlib/node.py:895: in start
node.watch_log_for_alive(self, from_mark=mark)
../env3.6/lib/python3.6/site-packages/ccmlib/node.py:664: in watch_log_for_alive
self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
filename=filename)
../env3.6/lib/python3.6/site-packages/ccmlib/node.py:592: in watch_log_for
head=reads[:50], tail="..."+reads[len(reads)-150:]))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
start = 1635437046.9280019, timeout = 120
msg = "Missing: ['127.0.0.1:7000.* is now UP'] not found in system.log:\n Head:
\n Tail: ..."
node = 'node3'
@staticmethod
def raise_if_passed(start, timeout, msg, node=None):
if start + timeout < time.time():
> raise TimeoutError.create(start, timeout, msg, node)
E ccmlib.node.TimeoutError: 28 Oct 2021 16:06:07 [node3] after
120.12/120 seconds Missing: ['127.0.0.1:7000.* is now UP'] not found in
system.log:
EHead:
ETail: ...
../env3.6/lib/python3.6/site-packages/ccmlib/node.py:56: TimeoutError
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17075) Unable to update token metadata if replacement host set and unresolvable in DNS
[
https://issues.apache.org/jira/browse/CASSANDRA-17075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Meredith updated CASSANDRA-17075:
-
Bug Category: Parent values: Availability(12983)Level 1 values: Response
Crash(12991)
Complexity: Normal
Discovered By: User Report
Fix Version/s: 3.0.x
Severity: Low
Status: Open (was: Triage Needed)
> Unable to update token metadata if replacement host set and unresolvable in
> DNS
> ---
>
> Key: CASSANDRA-17075
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17075
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 3.0.x
>
>
> Related to CASSANDRA-16873. If a host is set for the
> {{cassandra.replace_address*}} properties and does not resolve in DNS it
> prevents updating token metadata with an unknown host exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17075) Unable to update token metadata if replacement host set and unresolvable in DNS
Jon Meredith created CASSANDRA-17075:
Summary: Unable to update token metadata if replacement host set
and unresolvable in DNS
Key: CASSANDRA-17075
URL: https://issues.apache.org/jira/browse/CASSANDRA-17075
Project: Cassandra
Issue Type: Bug
Components: Cluster/Gossip
Reporter: Jon Meredith
Assignee: Jon Meredith
Related to CASSANDRA-16873. If a host is set for the
{{cassandra.replace_address*}} properties and does not resolve in DNS it
prevents updating token metadata with an unknown host exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435564#comment-17435564
]
Andres de la Peña commented on CASSANDRA-17070:
---
I forgot to mention that I found an unrelated mistake
[here|https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/cql3/ViewComplexTest.java#L142-L148],
where two tests are supposed to run a scenario with and without flushing the
table, but both of them run the scenario with flush. It's fixed in the commit
extracting a superclass,
[here|https://github.com/adelapena/cassandra/blob/204f3bbe69aae9165c0570224bfd811c8bbd4d07/test/unit/org/apache/cassandra/cql3/ViewComplexTest.java#L55-L61].
> ViewComplexTest hardening
> -
>
> Key: CASSANDRA-17070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17070
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> I have seen a number of times already the {{ViewComplexTest}} family timeout
> on test method teardown. This leaves a dirty env behind triggering the
> following test methods to fail on it. This ticket aims at hardening them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435553#comment-17435553
]
Andres de la Peña edited comment on CASSANDRA-17070 at 10/28/21, 4:54 PM:
--
I'm not sure I understand why the MV cleanup can fail. Would the second round
of cleanup guarantee that the MV cleanups succeed, or would it reduce the
chances of having leftovers because it's unlikely to have the same error twice?
Maybe it's just giving more time for the first async cleanup to happen?
In any case, here are 500 runs of each ViewComplex test:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1090/workflows/4362eefe-a13a-4a8c-a8eb-9e543a74623a/jobs/10187]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1091/workflows/a9185c89-ceff-4ddd-90c0-960281233fe3/jobs/10189]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1092/workflows/1f74edb4-3a41-47fd-b95c-0edc1d514a20/jobs/10191]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1093/workflows/f2e69851-19ea-4ab6-9315-5af9d721ec3d/jobs/10193]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1094/workflows/0d25bf72-7c90-4287-92d6-21a751082e21/jobs/10196]
>From these runs only a test runner for {{ViewComplexTest}} is failing due to
>timeouts.
It's not strictly related to the fix, but it seems that there is some repeated
code among the five tests in which {{ViewComplexTest}} was split by
CASSANDRA-16670. Probably we should do the same thing that we did with
{{ViewTest}} in CASSANDRA-16777 and extract a superclass to avoid code
duplication. This would make changes like this one a bit easier to do. I gave
it a go in [this
commit|https://github.com/adelapena/cassandra/commit/204f3bbe69aae9165c0570224bfd811c8bbd4d07],
and it saves us some 176 lines of code, which I think is always nice.
Here are some repeated runs for the tests using a common superclass, with no
failures so far:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1096/workflows/ded0dfe1-e736-468a-babb-0c007a6bf3c9/jobs/10201]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1097/workflows/544b429d-b712-428c-b97d-edd50076992a/jobs/10203]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1098/workflows/c062d815-2e26-4a75-aa64-656bfc48c8cc/jobs/10205]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1099/workflows/1f834191-b30b-4380-87ce-da257fac731e/jobs/10207]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1095/workflows/3df8138d-1d7b-4283-be3c-a23cb2c0bc40]
An alternative/complementary approach to avoid problems with leftovers from
previous runs is making sure that all MVs are created with a new, unique name.
This way the tests don't collide with any surviving MVs if the cleanup has
failed or if it hasn't finished. {{CQLTester}} does this with keyspace, table,
function and aggregate names.
I gave a go to this generation of unique names
[here|https://github.com/adelapena/cassandra/commit/e1928447e4139f1ad4126d9ed34e3de9d3bceb10].
It is just for {{ViewComplexTest}}, but we may consider having a more generic
and reusable solution for creating/cleaning MVs in {{CQLTester}}, although I
think we don't have to do it here. wdyt?
was (Author: adelapena):
I'm not sure I understand why the MV cleanup can fail. Would the second round
of cleanup guarantee that the MV cleanups succeed, or would it reduce the
chances of having leftovers because it's unlikely to have the same error twice?
Maybe it's just giving more time for the first async cleanup to happen?
In any case, here are 500 runs of each ViewComplex test:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1090/workflows/4362eefe-a13a-4a8c-a8eb-9e543a74623a/jobs/10187]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1091/workflows/a9185c89-ceff-4ddd-90c0-960281233fe3/jobs/10189]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1092/workflows/1f74edb4-3a41-47fd-b95c-0edc1d514a20/jobs/10191]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1093/workflows/f2e69851-19ea-4ab6-9315-5af9d721ec3d/jobs/10193]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1094/workflows/0d25bf72-7c90-4287-92d6-21a751082e21/jobs/10196]
>From this only a test runner for {{ViewComplexTest}} is failing due to
>timeouts.
It's not strictly related to the fix, but it seems there is some repeated code
among the five tests in which {{ViewComplexTest}} was
[jira] [Comment Edited] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435553#comment-17435553
]
Andres de la Peña edited comment on CASSANDRA-17070 at 10/28/21, 4:49 PM:
--
I'm not sure I understand why the MV cleanup can fail. Would the second round
of cleanup guarantee that the MV cleanups succeed, or would it reduce the
chances of having leftovers because it's unlikely to have the same error twice?
Maybe it's just giving more time for the first async cleanup to happen?
In any case, here are 500 runs of each ViewComplex test:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1090/workflows/4362eefe-a13a-4a8c-a8eb-9e543a74623a/jobs/10187]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1091/workflows/a9185c89-ceff-4ddd-90c0-960281233fe3/jobs/10189]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1092/workflows/1f74edb4-3a41-47fd-b95c-0edc1d514a20/jobs/10191]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1093/workflows/f2e69851-19ea-4ab6-9315-5af9d721ec3d/jobs/10193]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1094/workflows/0d25bf72-7c90-4287-92d6-21a751082e21/jobs/10196]
>From this only a test runner for {{ViewComplexTest}} is failing due to
>timeouts.
It's not strictly related to the fix, but it seems there is some repeated code
among the five tests in which {{ViewComplexTest}} was split by CASSANDRA-16670.
Probably we should do the same thing that we did with {{ViewTest}} in
CASSANDRA-16777 and extract a superclass to avoid code duplication. This would
changes like this one a bit easier to do. I gave it a go in [this
commit|https://github.com/adelapena/cassandra/commit/204f3bbe69aae9165c0570224bfd811c8bbd4d07],
and it saves us some 176 lines of code, which I think is always nice.
Here are some repeated runs for the tests using a common superclass, with no
failures so far:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1096/workflows/ded0dfe1-e736-468a-babb-0c007a6bf3c9/jobs/10201]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1097/workflows/544b429d-b712-428c-b97d-edd50076992a/jobs/10203]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1098/workflows/c062d815-2e26-4a75-aa64-656bfc48c8cc/jobs/10205]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1099/workflows/1f834191-b30b-4380-87ce-da257fac731e/jobs/10207]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1095/workflows/3df8138d-1d7b-4283-be3c-a23cb2c0bc40]
An alternative/complementary approach to avoid problems with leftovers from
previous runs is making sure that all MVs are created with a new, unique name.
This way the tests don't collide with any surviving MVs if the cleanup has
failed or if it hasn't finished. {{CQLTester}} does this with keyspace, table,
function and aggregate names.
I gave a go to this generation of unique names
[here|https://github.com/adelapena/cassandra/commit/e1928447e4139f1ad4126d9ed34e3de9d3bceb10].
It is just for {{ViewComplexTest}}, but we may consider having a more generic
and reusable solution for creating/cleaning MVs in {{CQLTester}}, although I
think we don't have to do it here. wdyt?
was (Author: adelapena):
I'm not sure I understand why the MV cleanup can fail. Would the second round
of cleanup guarantee that the MV cleanups succeed, or would it reduce the
chances of having leftovers because it's unlikely to have the same error twice?
Maybe it's just giving more time for the first async cleanup to happen?
In any case, here are 500 runs of each ViewComplex test:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1090/workflows/4362eefe-a13a-4a8c-a8eb-9e543a74623a/jobs/10187]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1091/workflows/a9185c89-ceff-4ddd-90c0-960281233fe3/jobs/10189]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1092/workflows/1f74edb4-3a41-47fd-b95c-0edc1d514a20/jobs/10191]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1093/workflows/f2e69851-19ea-4ab6-9315-5af9d721ec3d/jobs/10193]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1094/workflows/0d25bf72-7c90-4287-92d6-21a751082e21/jobs/10196]
>From this only a test runner for {{ViewComplexTest}} is failing due to
>timeouts.
It's not strictly related to the fix, but it seems there is some repeated code
among the five tests in which {{ViewComplexTest}} was split by
[jira] [Comment Edited] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435553#comment-17435553
]
Andres de la Peña edited comment on CASSANDRA-17070 at 10/28/21, 4:43 PM:
--
I'm not sure I understand why the MV cleanup can fail. Would the second round
of cleanup guarantee that the MV cleanups succeed, or would it reduce the
chances of having leftovers because it's unlikely to have the same error twice?
Maybe it's just giving more time for the first async cleanup to happen?
In any case, here are 500 runs of each ViewComplex test:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1090/workflows/4362eefe-a13a-4a8c-a8eb-9e543a74623a/jobs/10187]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1091/workflows/a9185c89-ceff-4ddd-90c0-960281233fe3/jobs/10189]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1092/workflows/1f74edb4-3a41-47fd-b95c-0edc1d514a20/jobs/10191]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1093/workflows/f2e69851-19ea-4ab6-9315-5af9d721ec3d/jobs/10193]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1094/workflows/0d25bf72-7c90-4287-92d6-21a751082e21/jobs/10196]
>From this only a test runner for {{ViewComplexTest}} is failing due to
>timeouts.
It's not strictly related to the fix, but it seems there is some repeated code
among the five tests in which {{ViewComplexTest}} was split by CASSANDRA-16670.
Probably we should do the same thing that we did with {{ViewTest}} in
CASSANDRA-16777 and extract a superclass to avoid code duplication. This would
changes like this one a bit easier to do. I gave it a go in [this
commit|https://github.com/adelapena/cassandra/commit/204f3bbe69aae9165c0570224bfd811c8bbd4d07],
and it saves us some 176 lines of code, which I think is always nice.
Here are some repeated runs for the test using a common superclass, with no
failures so far:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1096/workflows/ded0dfe1-e736-468a-babb-0c007a6bf3c9/jobs/10201]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1097/workflows/544b429d-b712-428c-b97d-edd50076992a/jobs/10203]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1098/workflows/c062d815-2e26-4a75-aa64-656bfc48c8cc/jobs/10205]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1099/workflows/1f834191-b30b-4380-87ce-da257fac731e/jobs/10207]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1095/workflows/3df8138d-1d7b-4283-be3c-a23cb2c0bc40]
An alternative/complementary approach to avoid problems with leftovers from
previous test runs is making sure that all MVs are created with a new, unique
name. This way the tests don't collide with any surviving MVs if the cleanup
has failed or if it hasn't finished. {{CQLTester}} does this with keyspace,
table, function and aggregate names.
I gave it a go to this generation of unique names
[here|https://github.com/adelapena/cassandra/commit/e1928447e4139f1ad4126d9ed34e3de9d3bceb10].
It is just for {{ViewComplexTest}}, but we may consider having a more generic
and reusable solution for creating/cleaning MVs in {{CQLTester}}, although I
think we don't have to do it here. wdyt?
was (Author: adelapena):
I'm not sure I understand why the MV cleanup can fail. Would the second round
of cleanup guarantee that the MV cleanups succeed, or would it reduce the
chances of having leftovers because it's unlikely to have the same error twice?
Maybe it's just giving more time for the first async cleanup to happen?
In any case, here are 500 runs of each ViewComplex test:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1090/workflows/4362eefe-a13a-4a8c-a8eb-9e543a74623a/jobs/10187]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1091/workflows/a9185c89-ceff-4ddd-90c0-960281233fe3/jobs/10189]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1092/workflows/1f74edb4-3a41-47fd-b95c-0edc1d514a20/jobs/10191]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1093/workflows/f2e69851-19ea-4ab6-9315-5af9d721ec3d/jobs/10193]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1094/workflows/0d25bf72-7c90-4287-92d6-21a751082e21/jobs/10196]
>From this only a test runner for {{ViewComplexTest}} is failing due to
>timeouts.
It's not strictly related to the fix, but it seems there is some repeated code
among the five tests in which {{ViewComplexTest}} was split by
[jira] [Commented] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435553#comment-17435553
]
Andres de la Peña commented on CASSANDRA-17070:
---
I'm not sure I understand why the MV cleanup can fail. Would the second round
of cleanup guarantee that the MV cleanups succeed, or would it reduce the
chances of having leftovers because it's unlikely to have the same error twice?
Maybe it's just giving more time for the first async cleanup to happen?
In any case, here are 500 runs of each ViewComplex test:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1090/workflows/4362eefe-a13a-4a8c-a8eb-9e543a74623a/jobs/10187]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1091/workflows/a9185c89-ceff-4ddd-90c0-960281233fe3/jobs/10189]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1092/workflows/1f74edb4-3a41-47fd-b95c-0edc1d514a20/jobs/10191]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1093/workflows/f2e69851-19ea-4ab6-9315-5af9d721ec3d/jobs/10193]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1094/workflows/0d25bf72-7c90-4287-92d6-21a751082e21/jobs/10196]
>From this only a test runner for {{ViewComplexTest}} is failing due to
>timeouts.
It's not strictly related to the fix, but it seems there is some repeated code
among the five tests in which {{ViewComplexTest}} was split by CASSANDRA-16670.
Probably we should do the same thing that we did with {{ViewTest}} in
CASSANDRA-16777 and extract a superclass to avoid code duplication. This would
changes like this one a bit easier to do. I gave it a go in [this
commit|https://github.com/adelapena/cassandra/commit/204f3bbe69aae9165c0570224bfd811c8bbd4d07],
and it saves us some 176 lines of code, which I think is always nice.
Here are some repeated runs for the test using a common superclass, with no
failures so far:
*
[ViewComplexDeletionsTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1096/workflows/ded0dfe1-e736-468a-babb-0c007a6bf3c9/jobs/10201]
*
[ViewComplexLivenessTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1097/workflows/544b429d-b712-428c-b97d-edd50076992a/jobs/10203]
*
[ViewComplexTTLTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1098/workflows/c062d815-2e26-4a75-aa64-656bfc48c8cc/jobs/10205]
*
[ViewComplexTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1099/workflows/1f834191-b30b-4380-87ce-da257fac731e/jobs/10207]
*
[ViewComplexUpdatesTest|https://app.circleci.com/pipelines/github/adelapena/cassandra/1095/workflows/3df8138d-1d7b-4283-be3c-a23cb2c0bc40]
An alternative/complementary approach to avoid problems with leftovers from
previous test runs is making sure that all MVs are created with a new, unique
name. This way the tests don't collide with any surviving MVs if the cleanup
has failed or if it hasn't finished. {{CQLTester}} does this with keyspace,
table, function and aggregate names.
I gave it a go to this generation of unique names
[here|https://github.com/adelapena/cassandra/commit/e1928447e4139f1ad4126d9ed34e3de9d3bceb10].
It is just for {{ViewComplexTest}}, but we may consider having a more generic
and reusable solution for creating/cleaning MVs in {{CQLTester}}, although I
think we don't have to it here. wdyt?
> ViewComplexTest hardening
> -
>
> Key: CASSANDRA-17070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17070
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> I have seen a number of times already the {{ViewComplexTest}} family timeout
> on test method teardown. This leaves a dirty env behind triggering the
> following test methods to fail on it. This ticket aims at hardening them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17050) Upgrade tests fail with InvocationTargetException
[ https://issues.apache.org/jira/browse/CASSANDRA-17050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435487#comment-17435487 ] Benedict Elliott Smith commented on CASSANDRA-17050: [dtest-api|https://github.com/belliottsmith/cassandra-in-jvm-dtest-api/tree/17050] It looks like Jenkins CI has its own issues, as I don't think CQLSHLIB tests have been affected by this or earlier work, and 2.2 has failing python dtests that are definitely unrelated. CircleCI is now showing upgrade dtests as clean, and I have the other jvm dtests for confirmation. However we will need to release a new dtest-api (preferably to include CASSANDRA-17064) before we merge this. I wonder if we should aim to reduce the friction for internal dependencies by using the apache snapshots repository for non-release builds. Not sure if somebody has a better idea, but today it is more than a little annoying to have to release the dtest-api before merging any change that uses it. The problem may become more common as we accumulate more sub-projects. > Upgrade tests fail with InvocationTargetException > - > > Key: CASSANDRA-17050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17050 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Urgent > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x, 4.x > > > Upgrade tests are currently failing due to the new dtest-api changes and > their integration with trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17050) Upgrade tests fail with InvocationTargetException
[ https://issues.apache.org/jira/browse/CASSANDRA-17050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435487#comment-17435487 ] Benedict Elliott Smith edited comment on CASSANDRA-17050 at 10/28/21, 3:15 PM: --- Additional necessary changes to [dtest-api|https://github.com/belliottsmith/cassandra-in-jvm-dtest-api/tree/17050] It looks like Jenkins CI has its own issues, as I don't think CQLSHLIB tests have been affected by this or earlier work, and 2.2 has failing python dtests that are definitely unrelated. CircleCI is now showing upgrade dtests as clean, and I have the other jvm dtests for confirmation. However we will need to release a new dtest-api (preferably to include CASSANDRA-17064) before we merge this. I wonder if we should aim to reduce the friction for internal dependencies by using the apache snapshots repository for non-release builds. Not sure if somebody has a better idea, but today it is more than a little annoying to have to release the dtest-api before merging any change that uses it. The problem may become more common as we accumulate more sub-projects. was (Author: benedict): [dtest-api|https://github.com/belliottsmith/cassandra-in-jvm-dtest-api/tree/17050] It looks like Jenkins CI has its own issues, as I don't think CQLSHLIB tests have been affected by this or earlier work, and 2.2 has failing python dtests that are definitely unrelated. CircleCI is now showing upgrade dtests as clean, and I have the other jvm dtests for confirmation. However we will need to release a new dtest-api (preferably to include CASSANDRA-17064) before we merge this. I wonder if we should aim to reduce the friction for internal dependencies by using the apache snapshots repository for non-release builds. Not sure if somebody has a better idea, but today it is more than a little annoying to have to release the dtest-api before merging any change that uses it. The problem may become more common as we accumulate more sub-projects. > Upgrade tests fail with InvocationTargetException > - > > Key: CASSANDRA-17050 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17050 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Urgent > Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0.x, 4.x > > > Upgrade tests are currently failing due to the new dtest-api changes and > their integration with trunk. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17064) dtest-api: Option to start nodes with blank gossip state
[ https://issues.apache.org/jira/browse/CASSANDRA-17064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435484#comment-17435484 ] Benedict Elliott Smith commented on CASSANDRA-17064: [dtest-api|https://github.com/belliottsmith/cassandra-in-jvm-dtest-api/tree/17064] > dtest-api: Option to start nodes with blank gossip state > > > Key: CASSANDRA-17064 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17064 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > dtest-api should permit starting nodes with neither regular gossip nor > artificially injected gossip, so that the Gossip state may be modified by the > tests from a blank slate -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17056) Pluggable SSTable format (SSTable format API)
[ https://issues.apache.org/jira/browse/CASSANDRA-17056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-17056: -- Mentor: Branimir Lambov > Pluggable SSTable format (SSTable format API) > - > > Key: CASSANDRA-17056 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17056 > Project: Cassandra > Issue Type: New Feature > Components: Local/SSTable >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-17%3A+SSTable+format+API -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17052) Queries performed with NODE_LOCAL consistency level do not update request metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-17052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-17052:
--
Reviewers: Alex Petrov
> Queries performed with NODE_LOCAL consistency level do not update request
> metrics
> -
>
> Key: CASSANDRA-17052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17052
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> Currently, queries performed with {{NODE_LOCAL}} consistency level are not
> reflected in request metrics. The suggested patch addresses that, and also
> allows modification and batch statements to be used with {{NODE_LOCAL}}
> consistency level.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17052) Queries performed with NODE_LOCAL consistency level do not update request metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-17052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-17052:
--
Status: Patch Available (was: In Progress)
Code: https://github.com/iamaleksey/cassandra/commits/17052-4.0
CI:
https://app.circleci.com/pipelines/github/iamaleksey/cassandra?branch=17052-4.0
> Queries performed with NODE_LOCAL consistency level do not update request
> metrics
> -
>
> Key: CASSANDRA-17052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17052
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> Currently, queries performed with {{NODE_LOCAL}} consistency level are not
> reflected in request metrics. The suggested patch addresses that, and also
> allows modification and batch statements to be used with {{NODE_LOCAL}}
> consistency level.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17052) Queries performed with NODE_LOCAL consistency level do not update request metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-17052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-17052:
--
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Normal
Discovered By: Code Inspection
Severity: Low
Status: Open (was: Triage Needed)
> Queries performed with NODE_LOCAL consistency level do not update request
> metrics
> -
>
> Key: CASSANDRA-17052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17052
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> Currently, queries performed with {{NODE_LOCAL}} consistency level are not
> reflected in request metrics. The suggested patch addresses that, and also
> allows modification and batch statements to be used with {{NODE_LOCAL}}
> consistency level.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17052) Queries performed with NODE_LOCAL consistency level do not update request metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-17052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-17052:
--
Fix Version/s: 4.x
4.0.x
Source Control Link:
https://github.com/iamaleksey/cassandra/commits/17052-4.0
Test and Documentation Plan: Unit tests included
Description: Currently, queries performed with
{{NODE_LOCAL}} consistency level are not reflected in request metrics. The
suggested patch addresses that, and also allows modification and batch
statements to be used with {{NODE_LOCAL}} consistency level. (was: TBD)
Summary: Queries performed with NODE_LOCAL consistency
level do not update request metrics (was: TBD)
> Queries performed with NODE_LOCAL consistency level do not update request
> metrics
> -
>
> Key: CASSANDRA-17052
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17052
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> Currently, queries performed with {{NODE_LOCAL}} consistency level are not
> reflected in request metrics. The suggested patch addresses that, and also
> allows modification and batch statements to be used with {{NODE_LOCAL}}
> consistency level.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[
https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Branimir Lambov updated CASSANDRA-6936:
---
Authors: Branimir Lambov, Dimitar Dimitrov, Jacek
Lewandowski (was: Branimir Lambov)
Test and Documentation Plan: Unit tests and in-tree documentation are
included.
Status: Patch Available (was: In Progress)
Patch uploaded here:
[branch|https://github.com/blambov/cassandra/tree/CASSANDRA-6936] [pull
request|https://github.com/apache/cassandra/pull/1294]
Provides an API for converting keys to and from byte-comparable
representations. Documentation of the translation is given in the included
{{ByteComparable.md}}. The translation is versioned and the patch provides two
versions: forward-only translation corresponding to DSE 6's trie-indexed
sstable format, and a current bi-directional version that can be freely
modified.
The byte-ordered translation is required for trie memtables, trie-based primary
indexes and SAI secondary indexes.
> Make all byte representations of types comparable by their unsigned byte
> representation only
>
>
> Key: CASSANDRA-6936
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6936
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Benedict Elliott Smith
>Assignee: Branimir Lambov
>Priority: Normal
> Labels: compaction, performance
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This could be a painful change, but is necessary for implementing a
> trie-based index, and settling for less would be suboptimal; it also should
> make comparisons cheaper all-round, and since comparison operations are
> pretty much the majority of C*'s business, this should be easily felt (see
> CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with
> major performance impacts). No copying/special casing/slicing should mean
> fewer opportunities to introduce performance regressions as well.
> Since I have slated for 3.0 a lot of non-backwards-compatible sstable
> changes, hopefully this shouldn't be too much more of a burden.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435382#comment-17435382 ] Benedict Elliott Smith commented on CASSANDRA-6936: --- Another blast from the past. I'm looking forward to seeing this land. > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Branimir Lambov >Priority: Normal > Labels: compaction, performance > Fix For: 4.x > > Time Spent: 10m > Remaining Estimate: 0h > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17074) Remove custom Duration object and refactor its usages to use Duration from Java
[ https://issues.apache.org/jira/browse/CASSANDRA-17074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-17074: -- Change Category: Code Clarity Complexity: Low Hanging Fruit Component/s: Local/Other Fix Version/s: 4.1 Status: Open (was: Triage Needed) > Remove custom Duration object and refactor its usages to use Duration from > Java > --- > > Key: CASSANDRA-17074 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17074 > Project: Cassandra > Issue Type: Task > Components: Local/Other >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > Extracted this as a separate ticket per discussion on CASSANDRA-17044 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-6936) Make all byte representations of types comparable by their unsigned byte representation only
[ https://issues.apache.org/jira/browse/CASSANDRA-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov reassigned CASSANDRA-6936: -- Assignee: Branimir Lambov > Make all byte representations of types comparable by their unsigned byte > representation only > > > Key: CASSANDRA-6936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6936 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Branimir Lambov >Priority: Normal > Labels: compaction, performance > Fix For: 4.x > > > This could be a painful change, but is necessary for implementing a > trie-based index, and settling for less would be suboptimal; it also should > make comparisons cheaper all-round, and since comparison operations are > pretty much the majority of C*'s business, this should be easily felt (see > CASSANDRA-6553 and CASSANDRA-6934 for an example of some minor changes with > major performance impacts). No copying/special casing/slicing should mean > fewer opportunities to introduce performance regressions as well. > Since I have slated for 3.0 a lot of non-backwards-compatible sstable > changes, hopefully this shouldn't be too much more of a burden. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-17070:
--
Reviewers: Andres de la Peña
> ViewComplexTest hardening
> -
>
> Key: CASSANDRA-17070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17070
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> I have seen a number of times already the {{ViewComplexTest}} family timeout
> on test method teardown. This leaves a dirty env behind triggering the
> following test methods to fail on it. This ticket aims at hardening them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-17070:
--
Reviewers: Andres de la Peña, Andres de la Peña (was: Andres de la Peña)
Andres de la Peña, Andres de la Peña (was: Andres de la Peña)
Status: Review In Progress (was: Patch Available)
> ViewComplexTest hardening
> -
>
> Key: CASSANDRA-17070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17070
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> I have seen a number of times already the {{ViewComplexTest}} family timeout
> on test method teardown. This leaves a dirty env behind triggering the
> following test methods to fail on it. This ticket aims at hardening them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435282#comment-17435282 ] Jacek Lewandowski commented on CASSANDRA-17044: --- For things which were mentioned as unrelated and suggested to be addressed in separate tickets, I've created: - CASSANDRA-17071 - CASSANDRA-17072 - CASSANDRA-17073 - CASSANDRA-17074 > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17074) Remove custom Duration object and refactor its usages to use Duration from Java
Jacek Lewandowski created CASSANDRA-17074: - Summary: Remove custom Duration object and refactor its usages to use Duration from Java Key: CASSANDRA-17074 URL: https://issues.apache.org/jira/browse/CASSANDRA-17074 Project: Cassandra Issue Type: Task Reporter: Jacek Lewandowski Assignee: Jacek Lewandowski Extracted this as a separate ticket per discussion on CASSANDRA-17044 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17073) The initial endpoint state is not propagated to all registered listeners in Gossiper
Jacek Lewandowski created CASSANDRA-17073:
-
Summary: The initial endpoint state is not propagated to all
registered listeners in Gossiper
Key: CASSANDRA-17073
URL: https://issues.apache.org/jira/browse/CASSANDRA-17073
Project: Cassandra
Issue Type: Bug
Components: Local/Other
Reporter: Jacek Lewandowski
Assignee: Jacek Lewandowski
Extracted this as a separate ticket per discussion on CASSANDRA-17044
Once joined the ring, {{StorageService}} artificially executes {{onChange}}
(endpoint event subscriber method) with all the updated values. However, what
we want probably that all registered endpoint event subscribers get updated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17072) Fix client warning in schema change related statements
Jacek Lewandowski created CASSANDRA-17072:
-
Summary: Fix client warning in schema change related statements
Key: CASSANDRA-17072
URL: https://issues.apache.org/jira/browse/CASSANDRA-17072
Project: Cassandra
Issue Type: Bug
Components: Local/Other
Reporter: Jacek Lewandowski
Assignee: Jacek Lewandowski
Extracted this as a separate ticket per discussion on CASSANDRA-17044
This seemed to be screwed a bit. In just two schema alteration statements we
collect client warnings which are captured during the transformation into a
local collection.
I guess it is done that way because the transformation is being executed in a
different stage (migration) and client warnings collected in that stage are not
present in the stage where the query is executed.
Then, the client warnings are retrieved using {{clientWarnings}} method and
added to the captured client warnings in the stage which is executing the
query.
This mechanism was implemented only in two schema alteration statements. It is
possible that for other ones the client warnings can simply get lost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17071) Relax schema synchronization when opening a keyspace
Jacek Lewandowski created CASSANDRA-17071: - Summary: Relax schema synchronization when opening a keyspace Key: CASSANDRA-17071 URL: https://issues.apache.org/jira/browse/CASSANDRA-17071 Project: Cassandra Issue Type: Improvement Components: Cluster/Schema Reporter: Jacek Lewandowski Assignee: Jacek Lewandowski Extracted this as a separate ticket per discussion on CASSANDRA-17044 In short, there are two purposes of this change: # Move the code around to the more appropriate places (for example, CFS specific code for dropping table was moved from SM class to CFS class) # Relax the synchronization when adding/removing keyspace instances in SM - instead of synchronizing the whole collection of keyspace instances, we only synchronize the related item (the original idea authored by [~blambov]) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17044) Refactor schema management to allow for schema source pluggability
[ https://issues.apache.org/jira/browse/CASSANDRA-17044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-17044: -- Change Category: Code Clarity Complexity: Normal Fix Version/s: 4.1 Status: Open (was: Triage Needed) > Refactor schema management to allow for schema source pluggability > -- > > Key: CASSANDRA-17044 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17044 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Schema >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 4.1 > > > The idea is decompose `Schema` into separate entities responsible for > different things. In particular extract what is related to schema storage and > synchronization into a separate class so that it is possible to create an > extension point there and store schema in a different way than > `system_schema` keyspace, for example in etcd. > This would also simplify the logic and reduce the number of special cases, > make all the things more testable and the logic of internal classes > encapsulated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16801) PasswordObfuscator should not assume PASSWORD is the last item in the WITH clause
[
https://issues.apache.org/jira/browse/CASSANDRA-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16801:
Test and Documentation Plan: See PR
Status: Patch Available (was: In Progress)
> PasswordObfuscator should not assume PASSWORD is the last item in the WITH
> clause
> -
>
> Key: CASSANDRA-16801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16801
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Caleb Rackliffe
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> CASSANDRA-16669 introduced support for obfuscating passwords for audit log
> statements, but there are a few cases where the obfuscation logic can destroy
> some of the contents of the original/provided string.
> ex. This is perfectly valid...
> {noformat}
> WITH LOGIN = false AND PASSWORD = 'bar' AND SUPERUSER = false
> {noformat}
> ...but calling obfuscate() on it will produce...
> {noformat}
> WITH LOGIN = false AND PASSWORD ***
> {noformat}
> -We should be able to create a reasonable RegEx and use String#replaceAll()
> to both simplify and correct PasswordObfuscator#obfuscate().-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17009) Sstableverify unit test operate on SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-17009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435244#comment-17435244 ] Berenguer Blasi commented on CASSANDRA-17009: - Hi [~bhouser] I noticed you pushed. Is this ready for review again? Feel free to ask for help if you're too busy etc > Sstableverify unit test operate on SSTables > --- > > Key: CASSANDRA-17009 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17009 > Project: Cassandra > Issue Type: Sub-task > Components: Tool/sstable >Reporter: Brian Houser >Assignee: Brian Houser >Priority: Normal > Fix For: 4.0.x, 4.x > > Time Spent: 3.5h > Remaining Estimate: 0h > > as part of https://issues.apache.org/jira/browse/CASSANDRA-16009, unit > coverage is a bit lax and doesn't run through the verifier (based on my > coverage results). > There should be a unit test that exercises the internal verifier both for a > corrupt example and a working example. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16801) PasswordObfuscator should not assume PASSWORD is the last item in the WITH clause
[
https://issues.apache.org/jira/browse/CASSANDRA-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435236#comment-17435236
]
Berenguer Blasi edited comment on CASSANDRA-16801 at 10/28/21, 8:07 AM:
Info for the reviewer:
Luckily antlr was already moving the password around in a DTO {{RoleOptions}}.
So the new logic is applied whenever possible and uses the previous logic as a
fallback.
Interesting corner case I found where {{testp}} is being revealed #justfyi
{noformat}
Type: audit
LogMessage:
user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:41278|timestamp:1635328638577|type:REQUEST_FAILURE|category:ERROR|operation:create
user 'test' with password ***; line 1:33 mismatched input 'testp'
expecting STRING_LITERAL (create user 'test' with password ***
{noformat}
was (Author: bereng):
Info for the reviewer:
Luckily antlr was already moving the password around in a DTO {{RoleOptions}}.
So the new logic is applied whenever possible and uses the previous logic as a
fallback.
Interesting corner case I found where {{testp}} is being revealed #jus
{noformat}
tfyi
Type: audit
LogMessage:
user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:41278|timestamp:1635328638577|type:REQUEST_FAILURE|category:ERROR|operation:create
user 'test' with password ***; line 1:33 mismatched input 'testp'
expecting STRING_LITERAL (create user 'test' with password ***
{noformat}
> PasswordObfuscator should not assume PASSWORD is the last item in the WITH
> clause
> -
>
> Key: CASSANDRA-16801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16801
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Caleb Rackliffe
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> CASSANDRA-16669 introduced support for obfuscating passwords for audit log
> statements, but there are a few cases where the obfuscation logic can destroy
> some of the contents of the original/provided string.
> ex. This is perfectly valid...
> {noformat}
> WITH LOGIN = false AND PASSWORD = 'bar' AND SUPERUSER = false
> {noformat}
> ...but calling obfuscate() on it will produce...
> {noformat}
> WITH LOGIN = false AND PASSWORD ***
> {noformat}
> -We should be able to create a reasonable RegEx and use String#replaceAll()
> to both simplify and correct PasswordObfuscator#obfuscate().-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16801) PasswordObfuscator should not assume PASSWORD is the last item in the WITH clause
[
https://issues.apache.org/jira/browse/CASSANDRA-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435236#comment-17435236
]
Berenguer Blasi edited comment on CASSANDRA-16801 at 10/28/21, 8:07 AM:
Info for the reviewer:
Luckily antlr was already moving the password around in a DTO {{RoleOptions}}.
So the new logic is applied whenever possible and uses the previous logic as a
fallback.
Interesting corner case I found where {{testp}} is being revealed #jus
{noformat}
tfyi
Type: audit
LogMessage:
user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:41278|timestamp:1635328638577|type:REQUEST_FAILURE|category:ERROR|operation:create
user 'test' with password ***; line 1:33 mismatched input 'testp'
expecting STRING_LITERAL (create user 'test' with password ***
{noformat}
was (Author: bereng):
Info for the reviewer:
Luckily antlr was already moving the password around in a DTO {{RoleOptions}}.
So the new logic is applied whenever possible and uses the previous logic as a
fallback.
Interesting corner case I found where {{testp}} is being revealed #justfyi
{{Type: audit
LogMessage:
user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:41278|timestamp:1635328638577|type:REQUEST_FAILURE|category:ERROR|operation:create
user 'test' with password ***; line 1:33 mismatched input 'testp'
expecting STRING_LITERAL (create user 'test' with password ***
}}
> PasswordObfuscator should not assume PASSWORD is the last item in the WITH
> clause
> -
>
> Key: CASSANDRA-16801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16801
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Caleb Rackliffe
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> CASSANDRA-16669 introduced support for obfuscating passwords for audit log
> statements, but there are a few cases where the obfuscation logic can destroy
> some of the contents of the original/provided string.
> ex. This is perfectly valid...
> {noformat}
> WITH LOGIN = false AND PASSWORD = 'bar' AND SUPERUSER = false
> {noformat}
> ...but calling obfuscate() on it will produce...
> {noformat}
> WITH LOGIN = false AND PASSWORD ***
> {noformat}
> -We should be able to create a reasonable RegEx and use String#replaceAll()
> to both simplify and correct PasswordObfuscator#obfuscate().-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16801) PasswordObfuscator should not assume PASSWORD is the last item in the WITH clause
[
https://issues.apache.org/jira/browse/CASSANDRA-16801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435236#comment-17435236
]
Berenguer Blasi commented on CASSANDRA-16801:
-
Info for the reviewer:
Luckily antlr was already moving the password around in a DTO {{RoleOptions}}.
So the new logic is applied whenever possible and uses the previous logic as a
fallback.
Interesting corner case I found where {{testp}} is being revealed #justfyi
{{Type: audit
LogMessage:
user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:41278|timestamp:1635328638577|type:REQUEST_FAILURE|category:ERROR|operation:create
user 'test' with password ***; line 1:33 mismatched input 'testp'
expecting STRING_LITERAL (create user 'test' with password ***
}}
> PasswordObfuscator should not assume PASSWORD is the last item in the WITH
> clause
> -
>
> Key: CASSANDRA-16801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16801
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Caleb Rackliffe
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> CASSANDRA-16669 introduced support for obfuscating passwords for audit log
> statements, but there are a few cases where the obfuscation logic can destroy
> some of the contents of the original/provided string.
> ex. This is perfectly valid...
> {noformat}
> WITH LOGIN = false AND PASSWORD = 'bar' AND SUPERUSER = false
> {noformat}
> ...but calling obfuscate() on it will produce...
> {noformat}
> WITH LOGIN = false AND PASSWORD ***
> {noformat}
> -We should be able to create a reasonable RegEx and use String#replaceAll()
> to both simplify and correct PasswordObfuscator#obfuscate().-
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17031) Add support for PEM based key material for SSL
[
https://issues.apache.org/jira/browse/CASSANDRA-17031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435201#comment-17435201
]
Stefan Miklosovic commented on CASSANDRA-17031:
---
Sure, I am just busy. Maybe next week. Sorry, try to ping somebody else if it
is urgent.
> Add support for PEM based key material for SSL
> --
>
> Key: CASSANDRA-17031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17031
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Maulin Vasavada
>Assignee: Maulin Vasavada
>Priority: Normal
>
> h1. Scope
> Currently Cassandra supports standard keystore types for SSL
> keys/certificates. The scope of this enhancement is to add support for PEM
> based key material (keys/certificate) given that PEM is widely used common
> format for the same. We intend to add support for Unencrypted and Password
> Based Encrypted (PBE) PKCS#8 formatted Private Keys in PEM format with
> standard algorithms (RSA, DSA and EC) along with the certificate chain for
> the private key and PEM based X509 certificates. The work here is going to be
> built on top of [CEP-9: Make SSLContext creation
> pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable]
> for which the code is merged for Apache Cassandra 4.1 release.
> We intend to support the key material be configured as direct PEM values
> input OR via the file (configured with keystore and truststore configurations
> today). We are not going to model PEM as a valid 'store_type' given that
> 'store_type' has a [specific
> definition|https://docs.oracle.com/en/java/javase/11/security/java-cryptography-architecture-jca-reference-guide.html#GUID-AB51DEFD-5238-4F96-967F-082F6D34FBEA].
>
> h1. Approach
> Create an implementation for
> [ISslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/ISslContextFactory.java]
> extending
> [FileBasedSslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java]
> implementation to add PEM formatted key/certificates.
> h1. Motivation
> PEM is a widely used format for encoding Private Keys and X.509 Certificates
> and Apache Cassandra's current implementation lacks the support for
> specifying the PEM formatted key material for SSL configurations. This means
> operators have to re-create the key material to comply to the supported
> formats (using key/trust store types - jks, pkcs12 etc) and deal with an
> operational task for managing it. This is an operational overhead we can
> avoid by supporting the PEM format making Apache Cassandra even more customer
> friendly and drive more adoption.
> h1. Proposed Changes
> # A new implementation for ISslContextFactory - PEMBasedSslContextFactory
> with the following supported configuration
> {panel:title=New configurations}
> {panel}
> |{{encryption_options: }}
> {{}}{{ssl_context_factory:}}
> {{}}{{class_name:
> org.apache.cassandra.security.PEMBasedSslContextFactory}}
> {{}}{{parameters:}}
> {{ }}{{private_key: certificate chain>}}
> {{ }}{{private_key_password: }}{{private}} {{key }}{{if}} {{it is encrypted>}}
> {{ }}{{trusted_certificates: }}|
> *NOTE:* We could reuse 'keystore_password' instead of the
> 'private_key_password'. However PEM encoded private key is not a 'keystore'
> in itself hence it would be inappropriate to piggyback on that other than
> avoid duplicating similar fields.
> # The PEMBasedSslContextFactory will also support file based key material
> (and the corresponding HOT Reloading based on file timestamp updates) for the
> PEM format via existing 'keystore' and 'truststore' encryption options.
> However in that case the 'truststore_password' configuration won't be used
> since generally PEM formatted certificates for truststore don't get encrypted
> with a password.
> # The PEMBasedSslContextFactory will internally create PKCS12 keystore for
> private key and the trusted certificates. However, this doesn't impact the
> user of the implementation in anyway and it is mentioned for clarity only.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435174#comment-17435174
]
Berenguer Blasi commented on CASSANDRA-17070:
-
Hardening by trying to rerun the cleanup logic on any leftovers in case the
first attempt failed. Trunk PR will be identical.
> ViewComplexTest hardening
> -
>
> Key: CASSANDRA-17070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17070
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> I have seen a number of times already the {{ViewComplexTest}} family timeout
> on test method teardown. This leaves a dirty env behind triggering the
> following test methods to fail on it. This ticket aims at hardening them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17070) ViewComplexTest hardening
[
https://issues.apache.org/jira/browse/CASSANDRA-17070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-17070:
Test and Documentation Plan: See PR
Status: Patch Available (was: In Progress)
> ViewComplexTest hardening
> -
>
> Key: CASSANDRA-17070
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17070
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> I have seen a number of times already the {{ViewComplexTest}} family timeout
> on test method teardown. This leaves a dirty env behind triggering the
> following test methods to fail on it. This ticket aims at hardening them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
