[cassandra] branch trunk updated: Intoduce Harry to the tree
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 81922c5 Intoduce Harry to the tree 81922c5 is described below commit 81922c5a7bcbf9db7564a29922c9d8f6222c7cdc Author: Alex Petrov AuthorDate: Tue Oct 12 13:04:11 2021 +0200 Intoduce Harry to the tree Patch by Alex Petrov; reviewed by Caleb Rackliffe and Abe Ratnofsky for CASSANDRA-16262. --- .build/build-rat.xml | 1 + build.xml | 3 +- .../cql3/statements/ModificationStatement.java | 14 +- test/conf/harry-generic.yaml | 75 + .../fuzz/FixedSchemaProviderConfiguration.java | 43 +++ .../cassandra/distributed/fuzz/FuzzTestBase.java | 131 .../cassandra/distributed/fuzz/HarryHelper.java| 96 ++ .../cassandra/distributed/fuzz/InJvmSut.java | 86 + .../cassandra/distributed/fuzz/InJvmSutBase.java | 294 + .../distributed/fuzz/QueryingNoOpChecker.java | 66 .../distributed/fuzz/SSTableGenerator.java | 360 + .../distributed/fuzz/SSTableLoadingVisitor.java| 113 +++ .../fuzz/test/SSTableGeneratorTest.java| 131 13 files changed, 1410 insertions(+), 3 deletions(-) diff --git a/.build/build-rat.xml b/.build/build-rat.xml index 599d5ea..c988043 100644 --- a/.build/build-rat.xml +++ b/.build/build-rat.xml @@ -53,6 +53,7 @@ + diff --git a/build.xml b/build.xml index 07c4057..01e2061 100644 --- a/build.xml +++ b/build.xml @@ -539,6 +539,7 @@ + @@ -742,7 +743,7 @@ - + diff --git a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java index 4233d23..b6d274a 100644 --- a/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java @@ -855,9 +855,19 @@ public abstract class ModificationStatement implements CQLStatement.SingleKeyspa private Slices toSlices(SortedSet> startBounds, SortedSet> endBounds) { +return toSlices(metadata, startBounds, endBounds); +} + +public static Slices toSlices(TableMetadata metadata, SortedSet> startBounds, SortedSet> endBounds) +{ +return toSlices(metadata.comparator, startBounds, endBounds); +} + +public static Slices toSlices(ClusteringComparator comparator, SortedSet> startBounds, SortedSet> endBounds) +{ assert startBounds.size() == endBounds.size(); -Slices.Builder builder = new Slices.Builder(metadata().comparator); +Slices.Builder builder = new Slices.Builder(comparator); Iterator> starts = startBounds.iterator(); Iterator> ends = endBounds.iterator(); @@ -865,7 +875,7 @@ public abstract class ModificationStatement implements CQLStatement.SingleKeyspa while (starts.hasNext()) { Slice slice = Slice.make(starts.next(), ends.next()); -if (!slice.isEmpty(metadata().comparator)) +if (!slice.isEmpty(comparator)) { builder.add(slice); } diff --git a/test/conf/harry-generic.yaml b/test/conf/harry-generic.yaml new file mode 100644 index 000..e4bfbab --- /dev/null +++ b/test/conf/harry-generic.yaml @@ -0,0 +1,75 @@ +seed: 1 + +# Default schema provider generates random schema +schema_provider: + default: {} + +drop_schema: false +create_schema: true +truncate_table: false + +clock: + approximate_monotonic: +history_size: 7300 +epoch_length: 1 +epoch_time_unit: "SECONDS" + +system_under_test: + println: {} + +partition_descriptor_selector: + default: +window_size: 100 +slide_after_repeats: 10 + +clustering_descriptor_selector: + default: +modifications_per_lts: + type: "constant" + constant: 2 +rows_per_modification: + type: "constant" + constant: 2 +operation_kind_weights: + DELETE_RANGE: 1 + DELETE_SLICE: 1 + DELETE_ROW: 1 + DELETE_COLUMN: 1 + DELETE_PARTITION: 1 + INSERT: 50 + UPDATE: 50 + DELETE_COLUMN_WITH_STATICS: 1 + INSERT_WITH_STATICS: 1 + UPDATE_WITH_STATICS: 1 +column_mask_bitsets: null +max_partition_size: 100 + +data_tracker: + default: +max_seen_lts: -1 +max_complete_lts: -1 + +runner: + sequential: +run_time: 60 +run_time_unit: "MINUTES" +visitors: + - logging: + row_visitor: +
[jira] [Updated] (CASSANDRA-16262) 4.0 Quality: Coordination & Replication Fuzz Testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-16262: Source Control Link: https://github.com/apache/cassandra/commit/81922c5a7bcbf9db7564a29922c9d8f6222c7cdc Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed to trunk with [81922c5a7bcbf9db7564a29922c9d8f6222c7cdc|https://github.com/apache/cassandra/commit/81922c5a7bcbf9db7564a29922c9d8f6222c7cdc] > 4.0 Quality: Coordination & Replication Fuzz Testing > > > Key: CASSANDRA-16262 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16262 > Project: Cassandra > Issue Type: Task > Components: Test/fuzz >Reporter: Caleb Rackliffe >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Fix For: 4.x > > Time Spent: 10h 10m > Remaining Estimate: 0h > > CASSANDRA-16180, CASSANDRA-16181, and CASSANDRA-15977 have largely focused on > auditing the existing tests around coordination, replication, and > read-repair, respectively. We've expanded existing test cases, added coverage > around components that we've refactored along the way, and added in-JVM dtest > upgrade tests where possible. > What remains is verifying the distributed read and write paths in the face of > common operational events, namely node restarts, bootstrapping, decommission, > and cleanup. If we can find a way to simulate these events, > [Harry|https://github.com/apache/cassandra-harry] seems like a good candidate > to host the verification logic itself. > To keep things simple initially, I would propose that we start by testing > simple read-only and write-only workloads (the former without read repair). -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16262) 4.0 Quality: Coordination & Replication Fuzz Testing
[ https://issues.apache.org/jira/browse/CASSANDRA-16262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-16262: Status: Ready to Commit (was: Review In Progress) > 4.0 Quality: Coordination & Replication Fuzz Testing > > > Key: CASSANDRA-16262 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16262 > Project: Cassandra > Issue Type: Task > Components: Test/fuzz >Reporter: Caleb Rackliffe >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Fix For: 4.x > > Time Spent: 10h 10m > Remaining Estimate: 0h > > CASSANDRA-16180, CASSANDRA-16181, and CASSANDRA-15977 have largely focused on > auditing the existing tests around coordination, replication, and > read-repair, respectively. We've expanded existing test cases, added coverage > around components that we've refactored along the way, and added in-JVM dtest > upgrade tests where possible. > What remains is verifying the distributed read and write paths in the face of > common operational events, namely node restarts, bootstrapping, decommission, > and cleanup. If we can find a way to simulate these events, > [Harry|https://github.com/apache/cassandra-harry] seems like a good candidate > to host the verification logic itself. > To keep things simple initially, I would propose that we start by testing > simple read-only and write-only workloads (the former without read repair). -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch trunk updated (a0fb0de -> fbb5383)
This is an automated email from the ASF dual-hosted git repository. blerer pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git. omit a0fb0de CASSANDRA-17373: February 2022 blog "Tightening Security for Apache Cassandra Part: 3" new fbb5383 February 2022 blog "Tightening Security for Apache Cassandra Part: 3" This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a0fb0de) \ N -- N -- N refs/heads/trunk (fbb5383) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] 01/01: February 2022 blog "Tightening Security for Apache Cassandra Part: 3"
This is an automated email from the ASF dual-hosted git repository. blerer pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git commit fbb538356c533feba1847fb5c51fe64c58907918 Author: Diogenese Topper AuthorDate: Thu Feb 10 19:00:05 2022 -0800 February 2022 blog "Tightening Security for Apache Cassandra Part: 3" patch by Maulin Vasavada, Diogenese Topper; reviewed by Erick Ramirez for CASSANDRA-17373 Add blog post titled "Tightening Security for Apache Cassandra Part: 3" update blog index add 2 images for blog: "Cassandra-SslContextFactory.png" and "tightening-security-for-apache-cassandra-p3-unsplash-jennefer-zacarias.jpg" --- .../images/blog/Cassandra-SslContextFactory.png| Bin 0 -> 33750 bytes ...che-cassandra-p3-unsplash-jennefer-zacarias.jpg | Bin 0 -> 1658871 bytes site-content/source/modules/ROOT/pages/blog.adoc | 24 + ...ening-Security-for-Apache-Cassandra-Part-3.adoc | 97 + 4 files changed, 121 insertions(+) diff --git a/site-content/source/modules/ROOT/images/blog/Cassandra-SslContextFactory.png b/site-content/source/modules/ROOT/images/blog/Cassandra-SslContextFactory.png new file mode 100644 index 000..872f0a4 Binary files /dev/null and b/site-content/source/modules/ROOT/images/blog/Cassandra-SslContextFactory.png differ diff --git a/site-content/source/modules/ROOT/images/blog/tightening-security-for-apache-cassandra-p3-unsplash-jennefer-zacarias.jpg b/site-content/source/modules/ROOT/images/blog/tightening-security-for-apache-cassandra-p3-unsplash-jennefer-zacarias.jpg new file mode 100644 index 000..b708d58 Binary files /dev/null and b/site-content/source/modules/ROOT/images/blog/tightening-security-for-apache-cassandra-p3-unsplash-jennefer-zacarias.jpg differ diff --git a/site-content/source/modules/ROOT/pages/blog.adoc b/site-content/source/modules/ROOT/pages/blog.adoc index d039eff..2523451 100644 --- a/site-content/source/modules/ROOT/pages/blog.adoc +++ b/site-content/source/modules/ROOT/pages/blog.adoc @@ -14,6 +14,30 @@ NOTES FOR CONTENT CREATORS [openblock,card-header] -- [discrete] +=== Tightening Security for Apache Cassandra: Part 3 +[discrete] + February 14, 2022 +-- +[openblock,card-content] +-- +In Part 3 of Maulin Vasavada’s mini-series on improving security, we detail how Cassandra 4.0 delivers ways to customize mTLS/TLS configuration while retaining the hot-reload functionality. + +[openblock,card-btn card-btn--blog] + +[.btn.btn--alt] +xref:blog/Tightening-Security-for-Apache-Cassandra-Part-3.adoc[Read More] + + +-- + +//end card + +//start card +[openblock,card shadow relative test] + +[openblock,card-header] +-- +[discrete] === Apache Cassandra Changelog #12 [discrete] February 10, 2022 diff --git a/site-content/source/modules/ROOT/pages/blog/Tightening-Security-for-Apache-Cassandra-Part-3.adoc b/site-content/source/modules/ROOT/pages/blog/Tightening-Security-for-Apache-Cassandra-Part-3.adoc new file mode 100644 index 000..c8ac3fb --- /dev/null +++ b/site-content/source/modules/ROOT/pages/blog/Tightening-Security-for-Apache-Cassandra-Part-3.adoc @@ -0,0 +1,97 @@ += Tightening security for Apache Cassandra: Part 3 +:page-layout: single-post +:page-role: blog-post +:page-post-date: February, 14 2022 +:page-post-author: Maulin Vasavada +:description: The Apache Cassandra Community +:keywords: + +:!figure-caption: + +.Image credit: https://unsplash.com/@zenchic[Jennefer Zacarias^] +image::blog/tightening-security-for-apache-cassandra-p3-unsplash-jennefer-zacarias.jpg[3-4] + +In xref:blog/Tightening-Security-for-Apache-Cassandra-Part-2.adoc[Part-2] of this series, we explored avenues for securing data in transit and described how to configure TLS/mTLS with Apache Cassandra 4.0. In Part 3, we’ll look at how you can customize TLS/mTLS for Apache Cassandra 4.0+ to overcome the challenges with a TLS configuration. + +=== How We Made TLS Configuration Better With 4.0 + +With Apache Cassandra 4.0, https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable[we enhanced^] the TLS/mTLS configuration to allow for specifying custom ways to build SSLContext and we provided a default implementation for backward compatibility. We introduced a new configuration, `ssl_context_factory`, where you can specify your custom class to build SSLContext objects required by Java/Netty SSL libraries. You can also add custom properties to [...] + +To demonstrate this customization, let’s use the example of Kubernetes, the popular cloud-native solution. Kubernetes allows configuring https://kubernetes.io/docs/concepts/configuration/secret/[Secrets^] to store sensitive data. We could potentially use K8s Secrets to store the keystore and truststore artifacts along with their respective passwords. We will assume Apache Cassandra is already runni
[jira] [Commented] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493057#comment-17493057 ] Aleksei Zotov commented on CASSANDRA-17386: --- Thanks [~e.dimitrova]! I'm trying to find out the root cause for the second day and I'm not even close :D I'm fighting with my local environment in order to run these tests and get the same failures as we have in Jenkins... The only thing I can say so far is that tests do not work with 2.7.18 because of [https://github.com/pypa/pip/commit/ca832b2836e0bffa7cf95589acdcd71230f5834e#diff-e58ec0adb9172e63c4f31031fb5a5b364fe9471e332bff30114233a002ee9d1eR142.] In order to run smth the tests you need Python <= 2.7.17. I feel at some point our CI may break because of this (I hope no-one will update Python to the latest version). I'll keep this ticket updated with the results of my research. > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17198) Allow to filter using LIKE predicates
[ https://issues.apache.org/jira/browse/CASSANDRA-17198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-17198: Reviewers: Benjamin Lerer, Berenguer Blasi (was: Berenguer Blasi) > Allow to filter using LIKE predicates > - > > Key: CASSANDRA-17198 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17198 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Syntax >Reporter: Benjamin Lerer >Assignee: Kanthi Subramanian >Priority: Normal > Labels: AdventCalendar2021, lhf > Fix For: 4.x > > > {{LIKE}} predicates can only be used with the SASI indices. In several > usecases (e.g. querying the {{settings}} virtual table) it makes sense to > support them for filtering. > +Additional information for newcomers:+ > There are some checks in the {{StatementRestrictions}} constructor and on > {{LikeRestriction}} that need to be removed for allowing filtering using LIKE > on clustering and regular columns. > For filtering on partition columns the {{needFiltering}} methods in > {{PartitionKeySingleRestrictionSet}} will need to be modified to return true > when LIKE predicate are used. > The unit tests should go in {{SelectTest}}. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17283) Failing test: UNIT org.apache.cassandra.tools.BulkLoaderTest.testBulkLoader_WithArgs1
[ https://issues.apache.org/jira/browse/CASSANDRA-17283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-17283: Test and Documentation Plan: See PR Status: Patch Available (was: In Progress) > Failing test: UNIT > org.apache.cassandra.tools.BulkLoaderTest.testBulkLoader_WithArgs1 > - > > Key: CASSANDRA-17283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17283 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Josh McKenzie >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.x > > > Intermittent failure: > [https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/org.apache.cassandra.tools/BulkLoaderTest/testBulkLoader_WithArgs1] > h3. Error Message > Wrong thread status, active threads unaccounted for: > [cluster2-connection-reaper-0] > h3. Stacktrace > junit.framework.AssertionFailedError: Wrong thread status, active threads > unaccounted for: [cluster2-connection-reaper-0] at > org.apache.cassandra.tools.OfflineToolUtils.assertNoUnexpectedThreadsStarted(OfflineToolUtils.java:106) > at > org.apache.cassandra.tools.BulkLoaderTest.testBulkLoader_WithArgs1(BulkLoaderTest.java:85) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > h3. Standard Output > DEBUG [main] 2022-01-21 19:11:46,606 InternalLoggerFactory.java:63 - Using > SLF4J as the default logging framework DEBUG [main] 2022-01-21 19:11:46,634 > PlatformDependent0.java:417 - -Dio.netty.noUnsafe: false DEBUG [main] > 2022-01-21 19:11:46,634 PlatformDependent0.java:897 - Java version: 8 DEBUG > [main] 2022-01-21 19:11:46,636 PlatformDependent0.java:130 - > sun.misc.Unsafe.theUnsafe: available DEBUG [main] 2022-01-21 19:11:46,636 > PlatformDependent0.java:154 - sun.misc.Unsafe.copyMemory: available -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (7525e6c -> b94530b)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git. omit 7525e6c generate docs for a0fb0de1 omit a0fb0de CASSANDRA-17373: February 2022 blog "Tightening Security for Apache Cassandra Part: 3" add fbb5383 February 2022 blog "Tightening Security for Apache Cassandra Part: 3" new b94530b generate docs for fbb53835 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (7525e6c) \ N -- N -- N refs/heads/asf-staging (b94530b) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4740084 -> 4740084 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17387) Test failure: dtest-offheap.counter_test.TestCounters.test_13691
[ https://issues.apache.org/jira/browse/CASSANDRA-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493086#comment-17493086 ] Berenguer Blasi commented on CASSANDRA-17387: - We have history of this test in jenkins going back to January last year and it has [always|https://ci-cassandra.apache.org/job/Cassandra-3.11/320/testReport/junit/dtest-offheap.counter_test/TestCounters/test_13691_2/history/?start=25] failed. Hence +1 to Brandon's assessment assuming [this|https://ci-cassandra.apache.org/job/Cassandra-devbranch/1428/] devbranch run succeeds. > Test failure: dtest-offheap.counter_test.TestCounters.test_13691 > > > Key: CASSANDRA-17387 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17387 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Urgent > Fix For: 3.11.x > > > This test fails only off heap on 3.11, hard to say when it started as it was > always failing since we have Butler. Marking it critical as it shows breaking > config change. > h3. Stacktrace > Unexpected error found in node logs (see stdout for full details). Errors: > [ERROR [main] 2022-02-10 05:06:27,462 DatabaseDescriptor.java (line 117) > Fatal configuration error > org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml. Please > remove properties [memtable_allocation_type] from your cassandra.yaml at > org.apache.cassandra.config.YamlConfigurationLoader$MissingPropertiesChecker.check(YamlConfigurationLoader.java:137) > at > org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:100) > at -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17387) Test failure: dtest-offheap.counter_test.TestCounters.test_13691
[ https://issues.apache.org/jira/browse/CASSANDRA-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-17387: Reviewers: Berenguer Blasi, Berenguer Blasi Berenguer Blasi, Berenguer Blasi (was: Berenguer Blasi) Status: Review In Progress (was: Patch Available) > Test failure: dtest-offheap.counter_test.TestCounters.test_13691 > > > Key: CASSANDRA-17387 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17387 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Urgent > Fix For: 3.11.x > > > This test fails only off heap on 3.11, hard to say when it started as it was > always failing since we have Butler. Marking it critical as it shows breaking > config change. > h3. Stacktrace > Unexpected error found in node logs (see stdout for full details). Errors: > [ERROR [main] 2022-02-10 05:06:27,462 DatabaseDescriptor.java (line 117) > Fatal configuration error > org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml. Please > remove properties [memtable_allocation_type] from your cassandra.yaml at > org.apache.cassandra.config.YamlConfigurationLoader$MissingPropertiesChecker.check(YamlConfigurationLoader.java:137) > at > org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:100) > at -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.0 by this push: new 85c202d Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction 85c202d is described below commit 85c202d8a8b037791db08c531f4f3c8336c82696 Author: Marcus Eriksson AuthorDate: Fri Jan 21 09:02:10 2022 +0100 Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction Patch by marcuse; reviewed by Caleb Rackliffe for CASSANDRA-17273 --- CHANGES.txt| 2 +- .../org/apache/cassandra/db/lifecycle/LogFile.java | 23 + .../cassandra/db/lifecycle/LogTransactionTest.java | 58 ++ 3 files changed, 64 insertions(+), 19 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index f0fefb2..527450d 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,5 +1,5 @@ 3.0.27 - + * Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction (CASSANDRA-17273) 3.0.26 * Fix conversion from megabits to bytes in streaming rate limiter (CASSANDRA-17243) diff --git a/src/java/org/apache/cassandra/db/lifecycle/LogFile.java b/src/java/org/apache/cassandra/db/lifecycle/LogFile.java index ac64f13..42d81ca 100644 --- a/src/java/org/apache/cassandra/db/lifecycle/LogFile.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogFile.java @@ -67,6 +67,9 @@ final class LogFile implements AutoCloseable // The transaction records, this set must be ORDER PRESERVING private final LinkedHashSet records = new LinkedHashSet<>(); +// the transaction records we have written to disk - used to guarantee that the +// on-disk log files become identical when creating a new replica +private final LinkedHashSet onDiskRecords = new LinkedHashSet<>(); // The type of the transaction private final OperationType type; @@ -320,17 +323,13 @@ final class LogFile implements AutoCloseable assert type == Type.ADD || type == Type.REMOVE; for (SSTableReader sstable : tables) -{ -File folder = sstable.descriptor.directory; -replicas.maybeCreateReplica(folder, getFileName(folder), records); -} +maybeCreateReplica(sstable); return LogRecord.make(type, tables); } private LogRecord makeAddRecord(SSTable table) { -File folder = table.descriptor.directory; -replicas.maybeCreateReplica(folder, getFileName(folder), records); +maybeCreateReplica(table); return LogRecord.make(Type.ADD, table); } @@ -342,12 +341,17 @@ final class LogFile implements AutoCloseable private LogRecord makeRecord(Type type, SSTable table, LogRecord record) { assert type == Type.ADD || type == Type.REMOVE; - -File folder = table.descriptor.directory; -replicas.maybeCreateReplica(folder, getFileName(folder), records); +maybeCreateReplica(table); return record.asType(type); } +private void maybeCreateReplica(SSTable sstable) +{ +File folder = sstable.descriptor.directory; +String fileName = getFileName(folder); +replicas.maybeCreateReplica(folder, fileName, onDiskRecords); +} + void addRecord(LogRecord record) { if (completed()) @@ -359,6 +363,7 @@ final class LogFile implements AutoCloseable replicas.append(record); if (!records.add(record)) throw new IllegalStateException("Failed to add record"); +onDiskRecords.add(record); } void remove(SSTable table) diff --git a/test/unit/org/apache/cassandra/db/lifecycle/LogTransactionTest.java b/test/unit/org/apache/cassandra/db/lifecycle/LogTransactionTest.java index 7ba1c39..6fb2334 100644 --- a/test/unit/org/apache/cassandra/db/lifecycle/LogTransactionTest.java +++ b/test/unit/org/apache/cassandra/db/lifecycle/LogTransactionTest.java @@ -22,7 +22,13 @@ import java.io.IOError; import java.io.IOException; import java.io.RandomAccessFile; import java.nio.file.Files; -import java.util.*; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; import java.util.function.BiConsumer; import java.util.function.Consumer; import java.util.stream.Collectors; @@ -33,19 +39,14 @@ import com.google.common.collect.Sets; import org.junit.BeforeClass; import org.junit.Test; -import static junit.framework.Assert.assertNotNull; -import static junit.framework.Assert.assertNull; -import static junit.framework.Assert.fail; -import static org.junit.Assert.assertEquals; -import static org.junit.Assert.assertFalse; -import static org.ju
[cassandra] branch cassandra-3.11 updated (0218d1f -> eeec360)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 0218d1f Merge branch 'cassandra-3.0' into cassandra-3.11 new 85c202d Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction new eeec360 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 + .../org/apache/cassandra/db/lifecycle/LogFile.java | 24 + .../cassandra/db/lifecycle/LogTransactionTest.java | 60 ++ 3 files changed, 65 insertions(+), 21 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (a87055d -> 2687cf1)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from a87055d Prepare debian changelog for 4.0.3 new 85c202d Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction new eeec360 Merge branch 'cassandra-3.0' into cassandra-3.11 new 2687cf1 Merge branch 'cassandra-3.11' into cassandra-4.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 3 + .../org/apache/cassandra/db/lifecycle/LogFile.java | 22 +++--- .../cassandra/db/lifecycle/LogTransactionTest.java | 86 -- 3 files changed, 76 insertions(+), 35 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 2687cf1edf5d61379475f5a4e65a112239649b8a Merge: a87055d eeec360 Author: Marcus Eriksson AuthorDate: Wed Feb 16 10:17:14 2022 +0100 Merge branch 'cassandra-3.11' into cassandra-4.0 CHANGES.txt| 3 + .../org/apache/cassandra/db/lifecycle/LogFile.java | 22 +++--- .../cassandra/db/lifecycle/LogTransactionTest.java | 86 -- 3 files changed, 76 insertions(+), 35 deletions(-) diff --cc CHANGES.txt index 937bedf,402048f..c64b602 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,43 -1,20 +1,46 @@@ -3.11.13 -Merged from 3.0: ++4.0.4 + * Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction (CASSANDRA-17273) + +4.0.3 + * Deprecate otc_coalescing_strategy, otc_coalescing_window_us, otc_coalescing_enough_coalesced_messages, + otc_backlog_expiration_interval_ms (CASSANDRA-17377) + * Improve start up processing of Incremental Repair information read from system.repairs (CASSANDRA-17342) -3.11.12 - * Upgrade snakeyaml to 1.26 in 3.11 (CASSANDRA=17028) +4.0.2 + * Full Java 11 support (CASSANDRA-16894) + * Remove unused 'geomet' package from cqlsh path (CASSANDRA-17271) + * Removed unused 'cql' dependency (CASSANDRA-17247) + * Don't block gossip when clearing repair snapshots (CASSANDRA-17168) + * Deduplicate warnings for deprecated parameters (changed names) (CASSANDRA-17160) + * Update ant-junit to version 1.10.12 (CASSANDRA-17218) + * Add droppable tombstone metrics to nodetool tablestats (CASSANDRA-16308) + * Fix disk failure triggered when enabling FQL on an unclean directory (CASSANDRA-17136) + * Fixed broken classpath when multiple jars in build directory (CASSANDRA-17129) + * DebuggableThreadPoolExecutor does not propagate client warnings (CASSANDRA-17072) + * internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes have new names. Backward compatibility with the old names added (CASSANDRA-17141) + * Remove unused configuration parameters from cassandra.yaml (CASSANDRA-17132) + * Queries performed with NODE_LOCAL consistency level do not update request metrics (CASSANDRA-17052) + * Fix multiple full sources can be select unexpectedly for bootstrap streaming (CASSANDRA-16945) + * Fix cassandra.yaml formatting of parameters (CASSANDRA-17131) + * Add backward compatibility for CQLSSTableWriter Date fields (CASSANDRA-17117) + * Push initial client connection messages to trace (CASSANDRA-17038) + * Correct the internode message timestamp if sending node has wrapped (CASSANDRA-16997) + * Avoid race causing us to return null in RangesAtEndpoint (CASSANDRA-16965) + * Avoid rewriting all sstables during cleanup when transient replication is enabled (CASSANDRA-16966) + * Prevent CQLSH from failure on Python 3.10 (CASSANDRA-16987) + * Avoid trying to acquire 0 permits from the rate limiter when taking snapshot (CASSANDRA-16872) + * Upgrade Caffeine to 2.5.6 (CASSANDRA-15153) + * Include SASI components to snapshots (CASSANDRA-15134) + * Fix missed wait latencies in the output of `nodetool tpstats -F` (CASSANDRA-16938) + * Remove all the state pollution between tests in SSTableReaderTest (CASSANDRA-16888) + * Delay auth setup until after gossip has settled to avoid unavailables on startup (CASSANDRA-16783) + * Fix clustering order logic in CREATE MATERIALIZED VIEW (CASSANDRA-16898) + * org.apache.cassandra.db.rows.ArrayCell#unsharedHeapSizeExcludingData includes data twice (CASSANDRA-16900) + * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) +Merged from 3.11: * Add key validation to ssstablescrub (CASSANDRA-16969) * Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851) - * Include SASI components to snapshots (CASSANDRA-15134) * Make assassinate more resilient to missing tokens (CASSANDRA-16847) - * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) - * Validate SASI tokenizer options before adding index to schema (CASSANDRA-15135) - * Fixup scrub output when no data post-scrub and clear up old use of row, which really means partition (CASSANDRA-16835) - * Fix ant-junit dependency issue (CASSANDRA-16827) - * Reduce thread contention in CommitLogSegment and HintsBuffer (CASSANDRA-16072) - * Avoid sending CDC column if not enabled (CASSANDRA-16770) Merged from 3.0: * Fix conversion from megabits to bytes in streaming rate limiter (CASSANDRA-17243) * Upgrade logback to 1.2.9 (CASSANDRA-17204) diff --cc src/java/org/apache/cassandra/db/lifecycle/LogFile.java index a91af73,3550d66..9053034 --- a/src/java/org/apache/cassandra/db/lifecycle/LogFile.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogFile.java @@@ -66,7 -66,10 +66,8 @@@
[cassandra] branch trunk updated (81922c5 -> e56bf10)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 81922c5 Intoduce Harry to the tree new 85c202d Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction new eeec360 Merge branch 'cassandra-3.0' into cassandra-3.11 new 2687cf1 Merge branch 'cassandra-3.11' into cassandra-4.0 new e56bf10 Merge branch 'cassandra-4.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 8 +- .../org/apache/cassandra/db/lifecycle/LogFile.java | 22 +++--- .../cassandra/db/lifecycle/LogTransactionTest.java | 91 +++--- 3 files changed, 82 insertions(+), 39 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit eeec360e757c191c517ddb51154c7c4f8d283d88 Merge: 0218d1f 85c202d Author: Marcus Eriksson AuthorDate: Wed Feb 16 10:14:41 2022 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 CHANGES.txt| 2 + .../org/apache/cassandra/db/lifecycle/LogFile.java | 24 + .../cassandra/db/lifecycle/LogTransactionTest.java | 60 ++ 3 files changed, 65 insertions(+), 21 deletions(-) diff --cc CHANGES.txt index c166202,527450d..402048f --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,19 -1,7 +1,21 @@@ -3.0.27 +3.11.13 ++Merged from 3.0: + * Lazy transaction log replica creation allows incorrect replica content divergence during anticompaction (CASSANDRA-17273) -3.0.26 + +3.11.12 + * Upgrade snakeyaml to 1.26 in 3.11 (CASSANDRA=17028) + * Add key validation to ssstablescrub (CASSANDRA-16969) + * Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851) + * Include SASI components to snapshots (CASSANDRA-15134) + * Make assassinate more resilient to missing tokens (CASSANDRA-16847) + * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) + * Validate SASI tokenizer options before adding index to schema (CASSANDRA-15135) + * Fixup scrub output when no data post-scrub and clear up old use of row, which really means partition (CASSANDRA-16835) + * Fix ant-junit dependency issue (CASSANDRA-16827) + * Reduce thread contention in CommitLogSegment and HintsBuffer (CASSANDRA-16072) + * Avoid sending CDC column if not enabled (CASSANDRA-16770) +Merged from 3.0: * Fix conversion from megabits to bytes in streaming rate limiter (CASSANDRA-17243) * Upgrade logback to 1.2.9 (CASSANDRA-17204) * Avoid race in AbstractReplicationStrategy endpoint caching (CASSANDRA-16673) diff --cc src/java/org/apache/cassandra/db/lifecycle/LogFile.java index 39bfaa9,42d81ca..3550d66 --- a/src/java/org/apache/cassandra/db/lifecycle/LogFile.java +++ b/src/java/org/apache/cassandra/db/lifecycle/LogFile.java @@@ -345,11 -341,15 +342,15 @@@ final class LogFile implements AutoClos private LogRecord makeRecord(Type type, SSTable table, LogRecord record) { assert type == Type.ADD || type == Type.REMOVE; + maybeCreateReplica(table); + return record.asType(type); + } - File directory = table.descriptor.directory; + private void maybeCreateReplica(SSTable sstable) + { -File folder = sstable.descriptor.directory; -String fileName = getFileName(folder); -replicas.maybeCreateReplica(folder, fileName, onDiskRecords); ++File directory = sstable.descriptor.directory; +String fileName = StringUtils.join(directory, File.separator, getFileName()); - replicas.maybeCreateReplica(directory, fileName, records); - return record.asType(type); ++replicas.maybeCreateReplica(directory, fileName, onDiskRecords); } void addRecord(LogRecord record) diff --cc test/unit/org/apache/cassandra/db/lifecycle/LogTransactionTest.java index c727241,6fb2334..09c75e1 --- a/test/unit/org/apache/cassandra/db/lifecycle/LogTransactionTest.java +++ b/test/unit/org/apache/cassandra/db/lifecycle/LogTransactionTest.java @@@ -44,15 -44,18 +44,16 @@@ import org.apache.cassandra.MockSchema import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.SerializationHeader; - import org.apache.cassandra.db.compaction.*; - import org.apache.cassandra.io.sstable.*; + import org.apache.cassandra.db.compaction.OperationType; + import org.apache.cassandra.io.sstable.Component; + import org.apache.cassandra.io.sstable.Descriptor; +import org.apache.cassandra.io.sstable.format.SSTableFormat; import org.apache.cassandra.io.sstable.format.SSTableReader; import org.apache.cassandra.io.sstable.metadata.MetadataCollector; import org.apache.cassandra.io.sstable.metadata.MetadataType; import org.apache.cassandra.io.sstable.metadata.StatsMetadata; - import org.apache.cassandra.io.util.FileUtils; -import org.apache.cassandra.io.util.BufferedSegmentedFile; -import org.apache.cassandra.io.util.ChannelProxy; +import org.apache.cassandra.io.util.FileHandle; + import org.apache.cassandra.io.util.FileUtils; -import org.apache.cassandra.io.util.RandomAccessReader; -import org.apache.cassandra.io.util.SegmentedFile; import org.apache.cassandra.utils.AlwaysPresentFilter; import org.apache.cassandra.utils.concurrent.AbstractTransactionalTest; import org.apache.cassandra.utils.concurrent.Transactional; - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@c
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e56bf105a161b1d9677ea4c9c70436f3169edb9c Merge: 81922c5 2687cf1 Author: Marcus Eriksson AuthorDate: Wed Feb 16 11:09:55 2022 +0100 Merge branch 'cassandra-4.0' into trunk CHANGES.txt| 8 +- .../org/apache/cassandra/db/lifecycle/LogFile.java | 22 +++--- .../cassandra/db/lifecycle/LogTransactionTest.java | 91 +++--- 3 files changed, 82 insertions(+), 39 deletions(-) diff --cc CHANGES.txt index 51e39bf,c64b602..9043898 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,95 -1,9 +1,101 @@@ +4.1 + * Add support for string concatenations through the + operator (CASSANDRA-17190) + * Limit the maximum hints size per host (CASSANDRA-17142) + * Add a virtual table for exposing batch metrics (CASSANDRA-17225) + * Flatten guardrails config (CASSANDRA-17353) + * Instance failed to start up due to NPE in StartupClusterConnectivityChecker (CASSANDRA-17347) + * add the shorter version of version flag (-v) in cqlsh (CASSANDRA-17236) + * Make vtables accessible via internode messaging (CASSANDRA-17295) + * Add support for PEM based key material for SSL (CASSANDRA-17031) + * Standardize storage configuration parameters' names. Support unit suffixes. (CASSANDRA-15234) + * Remove support for Windows (CASSANDRA-16956) + * Runtime-configurable YAML option to prohibit USE statements (CASSANDRA-17318) + * When streaming sees a ClosedChannelException this triggers the disk failure policy (CASSANDRA-17116) + * Add a virtual table for exposing prepared statements metrics (CASSANDRA-17224) + * Remove python 2.x support from cqlsh (CASSANDRA-17242) + * Prewarm role and credential caches to avoid timeouts at startup (CASSANDRA-16958) + * Make capacity/validity/updateinterval/activeupdate for Auth Caches configurable via nodetool (CASSANDRA-17063) + * Added startup check for read_ahead_kb setting (CASSANDRA-16436) + * Avoid unecessary array allocations and initializations when performing query checks (CASSANDRA-17209) + * Add guardrail for list operations that require read before write (CASSANDRA-17154) + * Migrate thresholds for number of keyspaces and tables to guardrails (CASSANDRA-17195) + * Remove self-reference in SSTableTidier (CASSANDRA-17205) + * Add guardrail for query page size (CASSANDRA-17189) + * Allow column_index_size_in_kb to be configurable through nodetool (CASSANDRA-17121) + * Emit a metric for number of local read and write calls + * Add non-blocking mode for CDC writes (CASSANDRA-17001) + * Add guardrails framework (CASSANDRA-17147) + * Harden resource management on SSTable components to prevent future leaks (CASSANDRA-17174) + * Make nodes more resilient to local unrelated files during startup (CASSANDRA-17082) + * repair prepare message would produce a wrong error message if network timeout happened rather than reply wait timeout (CASSANDRA-16992) + * Log queries that fail on timeout or unavailable errors up to once per minute by default (CASSANDRA-17159) + * Refactor normal/preview/IR repair to standardize repair cleanup and error handling of failed RepairJobs (CASSANDRA-17069) + * Log missing peers in StartupClusterConnectivityChecker (CASSANDRA-17130) + * Introduce separate rate limiting settings for entire SSTable streaming (CASSANDRA-17065) + * Implement Virtual Tables for Auth Caches (CASSANDRA-16914) + * Actively update auth cache in the background (CASSANDRA-16957) + * Add unix time conversion functions (CASSANDRA-17029) + * JVMStabilityInspector.forceHeapSpaceOomMaybe should handle all non-heap OOMs rather than only supporting direct only (CASSANDRA-17128) + * Forbid other Future implementations with checkstyle (CASSANDRA-17055) + * commit log was switched from non-daemon to daemon threads, which causes the JVM to exit in some case as no non-daemon threads are active (CASSANDRA-17085) + * Add a Denylist to block reads and writes on specific partition keys (CASSANDRA-12106) + * v4+ protocol did not clean up client warnings, which caused leaking the state (CASSANDRA-17054) + * Remove duplicate toCQLString in ReadCommand (CASSANDRA-17023) + * Ensure hint window is persistent across restarts of a node (CASSANDRA-14309) + * Allow to GRANT or REVOKE multiple permissions in a single statement (CASSANDRA-17030) + * Allow to grant permission for all tables in a keyspace (CASSANDRA-17027) + * Log time spent writing keys during compaction (CASSANDRA-17037) + * Make nodetool compactionstats and sstable_tasks consistent (CASSANDRA-16976) + * Add metrics and logging around index summary redistribution (CASSANDRA-17036) + * Add configuration options for minimum allowable replication factor and default replication factor (CASSANDRA-14557) + * Expose information about stored hints via a nodetool command and a
[jira] [Updated] (CASSANDRA-17273) Lazy transaction log replica creation allows incorrect replica content divergence during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-17273: Status: Ready to Commit (was: Review In Progress) > Lazy transaction log replica creation allows incorrect replica content > divergence during compaction > --- > > Key: CASSANDRA-17273 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17273 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair, Local/Compaction >Reporter: Caleb Rackliffe >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x > > Time Spent: 20m > Remaining Estimate: 0h > > Recently encountered this around compaction/anticompaction: > {noformat} > 2022-01-13 10:18:24,325 ERROR [main] > org.apache.cassandra.db.lifecycle.LogTransaction - Unexpected disk state: > failed to read transaction log > [mf_txn_anticompactionafterrepair_2f826324-742c-11ec-b293-65cae21e111c.log in > .../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f, > .../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f] > Files and contents follow: > .../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f/mf_txn_anticompactionafterrepair_2f826324-742c-11ec-b293-65cae21e111c.log > > ADD:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350438-big,0,8][2380834168] > > REMOVE:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350435-big,1642049328006,8][2338829485] > > REMOVE:[.../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350436-big,1642049366291,8][4248366924] > COMMIT:[,0,0][2613697770] > .../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/mf_txn_anticompactionafterrepair_2f826324-742c-11ec-b293-65cae21e111c.log > > ADD:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350437-big,0,8][4051162457] > ***Does not match > > in first replica file > > ADD:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350438-big,0,8][2380834168] > > REMOVE:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350435-big,1642049328006,8][2338829485] > > REMOVE:[.../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350436-big,1642049366291,8][4248366924] > COMMIT:[,0,0][2613697770] > {noformat} > We have two data directories and two transaction log files, but one is > missing an ADD entry when the contents of the two log replicas should be > identical. One scenario that can cause this is the following: > 1. Start anticompaction on a single file, in directory {{/tmp/d0}}. > 2. Call {{trackNew()}} with 2 new files, both in a single directory, but in > directory {{/tmp/d1}}. This initializes the log file in {{/tmp/d1}}, but > there is still no log file in {{/tmp/d0}}. > 3. Anticompaction only writes to one of the files in {{/tmp/d1}} (say all > other keys were outside the repaired range). > 4. When anticompaction is done, the empty writer is aborted and we call > {{untrackNew()}}, which removes the added file from the registered log > “records" (BUT NOT FROM DISK in {{/tmp/d1}}). > 5. The REMOVE record is added. This references {{/tmp/d0}}. We lazily create > the log file there by dumping all the records we have in memory to that file, > which does not include the aborted SSTable above. > 6. Now the log files contain: > {noformat} > /tmp/d1/logfile.log: > ADD:[/tmp/d1/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-2-big,0,8][3268492367] > ADD:[/tmp/d1/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-3-big,0,8][2813724425] > REMOVE:[/tmp/d0/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-1-big,1642078019000,8][2401235379] > COMMIT:[,0,0][2613697770] > ** /tmp/d0/logfile.log: > ADD:[/tmp/d1/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-3-big,0,8][2813724425] > REMOVE:[/tmp/d0/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-1-big,1642078019000,8][2401235379] > COMMIT:[,0,0][2613697770] > {noformat} -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17273) Lazy transaction log replica creation allows incorrect replica content divergence during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-17273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-17273: Since Version: 3.0.0 Source Control Link: https://github.com/apache/cassandra/commit/85c202d8a8b037791db08c531f4f3c8336c82696 Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks! test results; [3.0|https://app.circleci.com/pipelines/github/krummas/cassandra/776/workflows/1c800bdb-6b0c-4e90-be78-50c6b93be6a6] [3.11|https://app.circleci.com/pipelines/github/krummas/cassandra/778/workflows/78b6cd62-b504-4d18-aa5e-328df61c5fc1] [4.0|https://app.circleci.com/pipelines/github/krummas/cassandra/779/workflows/aa41d20c-c0b3-4fb0-a43c-ea1e52912fdf] [trunk|https://app.circleci.com/pipelines/github/krummas/cassandra/777/workflows/f36a02d3-305e-4530-a462-ba215d8f06b9] > Lazy transaction log replica creation allows incorrect replica content > divergence during compaction > --- > > Key: CASSANDRA-17273 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17273 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair, Local/Compaction >Reporter: Caleb Rackliffe >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x > > Time Spent: 20m > Remaining Estimate: 0h > > Recently encountered this around compaction/anticompaction: > {noformat} > 2022-01-13 10:18:24,325 ERROR [main] > org.apache.cassandra.db.lifecycle.LogTransaction - Unexpected disk state: > failed to read transaction log > [mf_txn_anticompactionafterrepair_2f826324-742c-11ec-b293-65cae21e111c.log in > .../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f, > .../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f] > Files and contents follow: > .../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f/mf_txn_anticompactionafterrepair_2f826324-742c-11ec-b293-65cae21e111c.log > > ADD:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350438-big,0,8][2380834168] > > REMOVE:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350435-big,1642049328006,8][2338829485] > > REMOVE:[.../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350436-big,1642049366291,8][4248366924] > COMMIT:[,0,0][2613697770] > .../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/mf_txn_anticompactionafterrepair_2f826324-742c-11ec-b293-65cae21e111c.log > > ADD:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350437-big,0,8][4051162457] > ***Does not match > > in first replica file > > ADD:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350438-big,0,8][2380834168] > > REMOVE:[.../d2/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350435-big,1642049328006,8][2338829485] > > REMOVE:[.../d1/data/.../files-c351f12917af3a5cbc57791cdf178a1f/prod_p203-files-mf-350436-big,1642049366291,8][4248366924] > COMMIT:[,0,0][2613697770] > {noformat} > We have two data directories and two transaction log files, but one is > missing an ADD entry when the contents of the two log replicas should be > identical. One scenario that can cause this is the following: > 1. Start anticompaction on a single file, in directory {{/tmp/d0}}. > 2. Call {{trackNew()}} with 2 new files, both in a single directory, but in > directory {{/tmp/d1}}. This initializes the log file in {{/tmp/d1}}, but > there is still no log file in {{/tmp/d0}}. > 3. Anticompaction only writes to one of the files in {{/tmp/d1}} (say all > other keys were outside the repaired range). > 4. When anticompaction is done, the empty writer is aborted and we call > {{untrackNew()}}, which removes the added file from the registered log > “records" (BUT NOT FROM DISK in {{/tmp/d1}}). > 5. The REMOVE record is added. This references {{/tmp/d0}}. We lazily create > the log file there by dumping all the records we have in memory to that file, > which does not include the aborted SSTable above. > 6. Now the log files contain: > {noformat} > /tmp/d1/logfile.log: > ADD:[/tmp/d1/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-2-big,0,8][3268492367] > ADD:[/tmp/d1/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-3-big,0,8][2813724425] > REMOVE:[/tmp/d0/AntiCompactionTest/AntiCompactionTest-e4fdddf0746e11ecb73ad5a997381615/AntiCompactionTest-AntiCompactionTest-mf-1-big,1642078019000,8][2401235379] > COMMIT:[,0,0][2613697770] > ** /tmp/d0/logfile.log: > ADD:[/tmp/d1/AntiCompactionTest
[jira] [Commented] (CASSANDRA-17186) Guardrail for number of partition keys on IN queries
[ https://issues.apache.org/jira/browse/CASSANDRA-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493147#comment-17493147 ] Andres de la Peña commented on CASSANDRA-17186: --- [~bereng] thanks for reviewing :). The two first comments on this ticket point to [the PR|https://github.com/apache/cassandra/pull/1442] kindly created by [~tejavadali], it contains the current review comments. The branch mentioned in the CI results above is just a copy on my repo with the CircleCI file on top, do you want me to create another PR? > Guardrail for number of partition keys on IN queries > > > Key: CASSANDRA-17186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17186 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Krishna Vadali >Priority: Normal > Labels: AdventCalendar2021, lhf > Fix For: 4.x > > Time Spent: 2.5h > Remaining Estimate: 0h > > Add a guardrail for limiting the number of partitions restricted with an > {{IN}} clause in a {{SELECT}} query, for example: > {code:java} > # Guardrail to warn or abort when querying with an IN restriction selecting > # more partition keys than threshold. > # The two thresholds default to -1 to disable. > partition_keys_in_select: > warn_threshold: -1 > abort_threshold: -1 > {code} > +Additional information for newcomers:+ > * Add the configuration for the new guardrail on the number of partitions on > IN queries in the guardrails section of cassandra.yaml. > * Add a getPartitionKeysInSelect method in GuardrailsConfig returning a > Threshold.Config object > * Implement that method in GuardrailsOptions, which is the default > yaml-based implementation of GuardrailsConfig > * Add a Threshold guardrail named partitionKeysInSelect in Guardrails, using > the previously created config > * Define JMX-friendly getters and setters for the previously created config > in GuardrailsMBean > * Implement the JMX-friendly getters and setters in Guardrails > * Now that we have the guardrail ready, it’s time to use it. We should > search for a place to invoke the Guardrails.partitionKeysInSelect#guard > method with the number of keys specified in select query. The > SelectStatement#getSliceCommands methods look like good candidates for this. > * Finally, add some tests for the new guardrail. Given that the new > guardrail is a Threshold, our new test should probably extend ThresholdTester. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17186) Guardrail for number of partition keys on IN queries
[ https://issues.apache.org/jira/browse/CASSANDRA-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493157#comment-17493157 ] Berenguer Blasi commented on CASSANDRA-17186: - Oh sorry I missed that. I was looking at your 'trunk' branch. I'll jump on that PR thx. > Guardrail for number of partition keys on IN queries > > > Key: CASSANDRA-17186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17186 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Krishna Vadali >Priority: Normal > Labels: AdventCalendar2021, lhf > Fix For: 4.x > > Time Spent: 2.5h > Remaining Estimate: 0h > > Add a guardrail for limiting the number of partitions restricted with an > {{IN}} clause in a {{SELECT}} query, for example: > {code:java} > # Guardrail to warn or abort when querying with an IN restriction selecting > # more partition keys than threshold. > # The two thresholds default to -1 to disable. > partition_keys_in_select: > warn_threshold: -1 > abort_threshold: -1 > {code} > +Additional information for newcomers:+ > * Add the configuration for the new guardrail on the number of partitions on > IN queries in the guardrails section of cassandra.yaml. > * Add a getPartitionKeysInSelect method in GuardrailsConfig returning a > Threshold.Config object > * Implement that method in GuardrailsOptions, which is the default > yaml-based implementation of GuardrailsConfig > * Add a Threshold guardrail named partitionKeysInSelect in Guardrails, using > the previously created config > * Define JMX-friendly getters and setters for the previously created config > in GuardrailsMBean > * Implement the JMX-friendly getters and setters in Guardrails > * Now that we have the guardrail ready, it’s time to use it. We should > search for a place to invoke the Guardrails.partitionKeysInSelect#guard > method with the number of keys specified in select query. The > SelectStatement#getSliceCommands methods look like good candidates for this. > * Finally, add some tests for the new guardrail. Given that the new > guardrail is a Threshold, our new test should probably extend ThresholdTester. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksei Zotov reassigned CASSANDRA-17386: - Assignee: Aleksei Zotov > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17186) Guardrail for number of partition keys on IN queries
[ https://issues.apache.org/jira/browse/CASSANDRA-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493175#comment-17493175 ] Berenguer Blasi commented on CASSANDRA-17186: - So I dropped some comments on the original PR. One of my concerns was the naming between abort/fail. And now I am confused bc I think you intend to use fail but on Andres' branch it says [abort|https://github.com/adelapena/cassandra/blob/17186-trunk/conf/cassandra.yaml#L1623] and there's also a comment/non-commented mismatch. So could we resolve those to a final version and squash so it's easier to comment? > Guardrail for number of partition keys on IN queries > > > Key: CASSANDRA-17186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17186 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Krishna Vadali >Priority: Normal > Labels: AdventCalendar2021, lhf > Fix For: 4.x > > Time Spent: 3h > Remaining Estimate: 0h > > Add a guardrail for limiting the number of partitions restricted with an > {{IN}} clause in a {{SELECT}} query, for example: > {code:java} > # Guardrail to warn or abort when querying with an IN restriction selecting > # more partition keys than threshold. > # The two thresholds default to -1 to disable. > partition_keys_in_select: > warn_threshold: -1 > abort_threshold: -1 > {code} > +Additional information for newcomers:+ > * Add the configuration for the new guardrail on the number of partitions on > IN queries in the guardrails section of cassandra.yaml. > * Add a getPartitionKeysInSelect method in GuardrailsConfig returning a > Threshold.Config object > * Implement that method in GuardrailsOptions, which is the default > yaml-based implementation of GuardrailsConfig > * Add a Threshold guardrail named partitionKeysInSelect in Guardrails, using > the previously created config > * Define JMX-friendly getters and setters for the previously created config > in GuardrailsMBean > * Implement the JMX-friendly getters and setters in Guardrails > * Now that we have the guardrail ready, it’s time to use it. We should > search for a place to invoke the Guardrails.partitionKeysInSelect#guard > method with the number of keys specified in select query. The > SelectStatement#getSliceCommands methods look like good candidates for this. > * Finally, add some tests for the new guardrail. Given that the new > guardrail is a Threshold, our new test should probably extend ThresholdTester. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493179#comment-17493179 ] Aleksei Zotov commented on CASSANDRA-17386: --- Looks like I found the issue in tests configuration. Raised a PR and triggered CI, let's wait and see. ||Item||Link|| |PR|https://github.com/apache/cassandra/pull/1450| |CI|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/| > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493179#comment-17493179 ] Aleksei Zotov edited comment on CASSANDRA-17386 at 2/16/22, 11:19 AM: -- Looks like I found the issue in tests configuration. Raised a PR and triggered CI, let's wait and see. ||Item||Link|| |PR|[https://github.com/apache/cassandra/pull/1450]| |CI|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/]| was (Author: azotcsit): Looks like I found the issue in tests configuration. Raised a PR and triggered CI, let's wait and see. ||Item||Link|| |PR|https://github.com/apache/cassandra/pull/1450| |CI|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/| > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17186) Guardrail for number of partition keys on IN queries
[ https://issues.apache.org/jira/browse/CASSANDRA-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493203#comment-17493203 ] Andres de la Peña commented on CASSANDRA-17186: --- I think I run CI over the branch not rebased on top of CASSANDRA-17353. That's the ticket that introduced the abort/fail renaming. Here is a new run of a squashed+rebased version of the patch, including [this trivial fix|https://github.com/apache/cassandra/pull/1442#discussion_r807878834]: ||Branch||CI|| |[17186-trunk-review|https://github.com/adelapena/cassandra/commits/17186-trunk-review]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1299/workflows/c858b3ac-3d37-4049-a104-3944fd3ade2c] [j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1299/workflows/583cb738-00cc-4f29-ab9c-40461095645f]| > Guardrail for number of partition keys on IN queries > > > Key: CASSANDRA-17186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17186 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Krishna Vadali >Priority: Normal > Labels: AdventCalendar2021, lhf > Fix For: 4.x > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Add a guardrail for limiting the number of partitions restricted with an > {{IN}} clause in a {{SELECT}} query, for example: > {code:java} > # Guardrail to warn or abort when querying with an IN restriction selecting > # more partition keys than threshold. > # The two thresholds default to -1 to disable. > partition_keys_in_select: > warn_threshold: -1 > abort_threshold: -1 > {code} > +Additional information for newcomers:+ > * Add the configuration for the new guardrail on the number of partitions on > IN queries in the guardrails section of cassandra.yaml. > * Add a getPartitionKeysInSelect method in GuardrailsConfig returning a > Threshold.Config object > * Implement that method in GuardrailsOptions, which is the default > yaml-based implementation of GuardrailsConfig > * Add a Threshold guardrail named partitionKeysInSelect in Guardrails, using > the previously created config > * Define JMX-friendly getters and setters for the previously created config > in GuardrailsMBean > * Implement the JMX-friendly getters and setters in Guardrails > * Now that we have the guardrail ready, it’s time to use it. We should > search for a place to invoke the Guardrails.partitionKeysInSelect#guard > method with the number of keys specified in select query. The > SelectStatement#getSliceCommands methods look like good candidates for this. > * Finally, add some tests for the new guardrail. Given that the new > guardrail is a Threshold, our new test should probably extend ThresholdTester. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17186) Guardrail for number of partition keys on IN queries
[ https://issues.apache.org/jira/browse/CASSANDRA-17186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493203#comment-17493203 ] Andres de la Peña edited comment on CASSANDRA-17186 at 2/16/22, 12:24 PM: -- I think I run CI over the branch not rebased on top of CASSANDRA-17353. That's the ticket that introduced the abort/fail renaming. Here is a new run of a squashed+rebased version of the patch, including [this trivial fix|https://github.com/apache/cassandra/pull/1442#discussion_r807878834]: ||Branch||CI|| |[adelapena:17186-trunk-review|https://github.com/adelapena/cassandra/commits/17186-trunk-review]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1299/workflows/c858b3ac-3d37-4049-a104-3944fd3ade2c] [j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1299/workflows/583cb738-00cc-4f29-ab9c-40461095645f]| was (Author: adelapena): I think I run CI over the branch not rebased on top of CASSANDRA-17353. That's the ticket that introduced the abort/fail renaming. Here is a new run of a squashed+rebased version of the patch, including [this trivial fix|https://github.com/apache/cassandra/pull/1442#discussion_r807878834]: ||Branch||CI|| |[17186-trunk-review|https://github.com/adelapena/cassandra/commits/17186-trunk-review]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1299/workflows/c858b3ac-3d37-4049-a104-3944fd3ade2c] [j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1299/workflows/583cb738-00cc-4f29-ab9c-40461095645f]| > Guardrail for number of partition keys on IN queries > > > Key: CASSANDRA-17186 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17186 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Andres de la Peña >Assignee: Krishna Vadali >Priority: Normal > Labels: AdventCalendar2021, lhf > Fix For: 4.x > > Time Spent: 3h 40m > Remaining Estimate: 0h > > Add a guardrail for limiting the number of partitions restricted with an > {{IN}} clause in a {{SELECT}} query, for example: > {code:java} > # Guardrail to warn or abort when querying with an IN restriction selecting > # more partition keys than threshold. > # The two thresholds default to -1 to disable. > partition_keys_in_select: > warn_threshold: -1 > abort_threshold: -1 > {code} > +Additional information for newcomers:+ > * Add the configuration for the new guardrail on the number of partitions on > IN queries in the guardrails section of cassandra.yaml. > * Add a getPartitionKeysInSelect method in GuardrailsConfig returning a > Threshold.Config object > * Implement that method in GuardrailsOptions, which is the default > yaml-based implementation of GuardrailsConfig > * Add a Threshold guardrail named partitionKeysInSelect in Guardrails, using > the previously created config > * Define JMX-friendly getters and setters for the previously created config > in GuardrailsMBean > * Implement the JMX-friendly getters and setters in Guardrails > * Now that we have the guardrail ready, it’s time to use it. We should > search for a place to invoke the Guardrails.partitionKeysInSelect#guard > method with the number of keys specified in select query. The > SelectStatement#getSliceCommands methods look like good candidates for this. > * Finally, add some tests for the new guardrail. Given that the new > guardrail is a Threshold, our new test should probably extend ThresholdTester. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: Skip offheap for test_13691
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/trunk by this push: new 941772d Skip offheap for test_13691 941772d is described below commit 941772dedd8cff44c32140dd2964dd18374ad419 Author: Brandon Williams AuthorDate: Tue Feb 15 16:58:38 2022 -0600 Skip offheap for test_13691 Patch by brandonwilliams; reviewed by bereng for CASSANDRA-17387 --- counter_test.py | 1 + 1 file changed, 1 insertion(+) diff --git a/counter_test.py b/counter_test.py index b596874..a4f80ef 100644 --- a/counter_test.py +++ b/counter_test.py @@ -18,6 +18,7 @@ logger = logging.getLogger(__name__) class TestCounters(Tester): @since('3.0', max_version='3.12') +@pytest.mark.no_offheap_memtables def test_13691(self): """ 2.0 -> 2.1 -> 3.0 counters upgrade test - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17387) Test failure: dtest-offheap.counter_test.TestCounters.test_13691
[ https://issues.apache.org/jira/browse/CASSANDRA-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-17387: - Status: Ready to Commit (was: Review In Progress) > Test failure: dtest-offheap.counter_test.TestCounters.test_13691 > > > Key: CASSANDRA-17387 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17387 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Urgent > Fix For: 3.11.x > > > This test fails only off heap on 3.11, hard to say when it started as it was > always failing since we have Butler. Marking it critical as it shows breaking > config change. > h3. Stacktrace > Unexpected error found in node logs (see stdout for full details). Errors: > [ERROR [main] 2022-02-10 05:06:27,462 DatabaseDescriptor.java (line 117) > Fatal configuration error > org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml. Please > remove properties [memtable_allocation_type] from your cassandra.yaml at > org.apache.cassandra.config.YamlConfigurationLoader$MissingPropertiesChecker.check(YamlConfigurationLoader.java:137) > at > org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:100) > at -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17387) Test failure: dtest-offheap.counter_test.TestCounters.test_13691
[ https://issues.apache.org/jira/browse/CASSANDRA-17387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-17387: - Fix Version/s: 3.11.13 (was: 3.11.x) Since Version: NA Source Control Link: https://github.com/apache/cassandra-dtest/commit/941772dedd8cff44c32140dd2964dd18374ad419 Resolution: Fixed Status: Resolved (was: Ready to Commit) CI looks nominal for the current state of 3.11, committed. Thank you! > Test failure: dtest-offheap.counter_test.TestCounters.test_13691 > > > Key: CASSANDRA-17387 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17387 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Urgent > Fix For: 3.11.13 > > > This test fails only off heap on 3.11, hard to say when it started as it was > always failing since we have Butler. Marking it critical as it shows breaking > config change. > h3. Stacktrace > Unexpected error found in node logs (see stdout for full details). Errors: > [ERROR [main] 2022-02-10 05:06:27,462 DatabaseDescriptor.java (line 117) > Fatal configuration error > org.apache.cassandra.exceptions.ConfigurationException: Invalid yaml. Please > remove properties [memtable_allocation_type] from your cassandra.yaml at > org.apache.cassandra.config.YamlConfigurationLoader$MissingPropertiesChecker.check(YamlConfigurationLoader.java:137) > at > org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:100) > at -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17334) Pre hashed passwords in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493251#comment-17493251 ] Berenguer Blasi commented on CASSANDRA-17334: - [~Bowen Song] I am moving your message from the ML to here so we keep the ML clean. I hope this is ok? {quote}To me this doesn't sound very useful. Here's a few threat model I can think of that may be related to this proposal, and why is this not addressing the issues & what should be done instead. 1. passwords are send over network in plaintext allows passive packet sniffier to learn about the password When the user logging in and authenticating themselves, they will have to send both the username and password to the server in plaintext anyway. Securing the connection with TLS should address this concern. 2. malicious intermediaries (external loadbancer, middleware, etc.) are able learn about the password The admin user must login against the intermediary before creating/altering other users, this exposes the admin user's credentials to the malicious intermediary. Only use trusted intermediaries, and use TLS between the client & Cassandra server wherever possible (e.g. don't terminate TLS at the loadbalancer). 3. accidentally logging the password to an insecure log file Logging a hashed password to an insecure log file is still very bad The logger module should correctly redact the data If this proposal helps mitigating a different threat model that you have in mind, please kindly share it with us. {quote} > Pre hashed passwords in CQL > --- > > Key: CASSANDRA-17334 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17334 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.1 > > > As seen on CASSANDRA-16801 and friends we are working across the system with > plain text passwords. These can be unintentionally revealed by intermediate > systems. Allowing the use of hashed passwords should mitigate that. The idea > is to add a new option {{HASHED PASSWORD}} for {{CREATE/ALTER ROLE/USER}}. > Examples: > {noformat} > CREATE ROLE foo WITH login = true AND hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > ALTER ROLE foo WITH hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > {noformat} > To generate the password hash, there will be a new tool {{hash_password}} in > resources/cassandra/bin > Based on original works from [~snazy] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17334) Pre hashed passwords in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493254#comment-17493254 ] Berenguer Blasi commented on CASSANDRA-17334: - Hi [~Bowen Song] , first of all thanks for taking the time to look into this. I see your concerns and I agree but I think we are talking about different things. This is my fault because I have the context from CASSANDRA-16801and CASSANDRA-16669 in my head I didn't provide. I am not referring to the authentication process which iiuc is where your concerns lie? and I agree with you. If you read a bit on those 2 tickets we had the problem that we were unintentionally logging plain text passwords when, i.e., creating a user. That is because the CQL statement itself to create the user carries the password in plain text, then audit logging would log it. It could happen the same with some CQL traffic capturing/replay tool i.e.. As the passwords are embedded into the DCL CQL anything that captures the CQL command string could leak them because it would be out of our control. We have the above 2 tickets to mitigate that. As you rightly point out that data should be removed and we do now. Still that removal happens only at the places we currently know about and control. We might add other functionality in the future were we forget to add this protection, people could attach custom software that captures CQL, create their own private implementation of some class, etc. So you're right again saying that leaking a hashed password is not ideal. But it's better than leaking it in plain text. with this ticket any captured CQL would now have a hash instead of the password. That would work for current, future and any work outside of our control (3rd party software or 3rd party code) Does this help and makes sense? > Pre hashed passwords in CQL > --- > > Key: CASSANDRA-17334 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17334 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.1 > > > As seen on CASSANDRA-16801 and friends we are working across the system with > plain text passwords. These can be unintentionally revealed by intermediate > systems. Allowing the use of hashed passwords should mitigate that. The idea > is to add a new option {{HASHED PASSWORD}} for {{CREATE/ALTER ROLE/USER}}. > Examples: > {noformat} > CREATE ROLE foo WITH login = true AND hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > ALTER ROLE foo WITH hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > {noformat} > To generate the password hash, there will be a new tool {{hash_password}} in > resources/cassandra/bin > Based on original works from [~snazy] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Emit a warning when the same parameter exists more than once in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493261#comment-17493261 ] Marcus Eriksson commented on CASSANDRA-17379: - Could we refuse starting if the values for the parameters differ? > Emit a warning when the same parameter exists more than once in > cassandra.yaml > --- > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17334) Pre hashed passwords in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493254#comment-17493254 ] Berenguer Blasi edited comment on CASSANDRA-17334 at 2/16/22, 2:44 PM: --- Hi [~Bowen Song] , first of all thanks for taking the time to look into this. I see your concerns and I agree but I think we are talking about different things. This is my fault because I have the context from CASSANDRA-16801and CASSANDRA-16669 in my head I didn't provide. I am not referring to the authentication process which iiuc is where your concerns lie? and I agree with you. If you read a bit on those 2 tickets we had the problem that we were unintentionally logging plain text passwords when, i.e., creating a user. That is because the CQL statement itself to create the user carries the password in plain text, then audit logging would log it. It could happen the same with some CQL traffic capturing/replay tool i.e.. As the passwords are embedded into the DCL CQL anything that captures the CQL command string could leak them because it would be out of our control. We have the above 2 tickets to mitigate that. As you rightly point out that data should be removed and we do now. Still that removal happens only at the places we currently know about and control. We might add other functionality in the future were we forget to add this protection, people could attach custom software that captures CQL, create their own private implementation of some class, forks, etc. So you're right again saying that leaking a hashed password is not ideal. But it's better than leaking it in plain text. With this ticket any captured CQL would now have a hash instead of the password. That would work for current, future and any work outside of our control (3rd party software, 3rd party code, forks...) Does this help and makes sense? was (Author: bereng): Hi [~Bowen Song] , first of all thanks for taking the time to look into this. I see your concerns and I agree but I think we are talking about different things. This is my fault because I have the context from CASSANDRA-16801and CASSANDRA-16669 in my head I didn't provide. I am not referring to the authentication process which iiuc is where your concerns lie? and I agree with you. If you read a bit on those 2 tickets we had the problem that we were unintentionally logging plain text passwords when, i.e., creating a user. That is because the CQL statement itself to create the user carries the password in plain text, then audit logging would log it. It could happen the same with some CQL traffic capturing/replay tool i.e.. As the passwords are embedded into the DCL CQL anything that captures the CQL command string could leak them because it would be out of our control. We have the above 2 tickets to mitigate that. As you rightly point out that data should be removed and we do now. Still that removal happens only at the places we currently know about and control. We might add other functionality in the future were we forget to add this protection, people could attach custom software that captures CQL, create their own private implementation of some class, etc. So you're right again saying that leaking a hashed password is not ideal. But it's better than leaking it in plain text. with this ticket any captured CQL would now have a hash instead of the password. That would work for current, future and any work outside of our control (3rd party software or 3rd party code) Does this help and makes sense? > Pre hashed passwords in CQL > --- > > Key: CASSANDRA-17334 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17334 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.1 > > > As seen on CASSANDRA-16801 and friends we are working across the system with > plain text passwords. These can be unintentionally revealed by intermediate > systems. Allowing the use of hashed passwords should mitigate that. The idea > is to add a new option {{HASHED PASSWORD}} for {{CREATE/ALTER ROLE/USER}}. > Examples: > {noformat} > CREATE ROLE foo WITH login = true AND hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > ALTER ROLE foo WITH hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > {noformat} > To generate the password hash, there will be a new tool {{hash_password}} in > resources/cassandra/bin > Based on original works from [~snazy] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...
[jira] [Updated] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-17379: Summary: Fail starting when the same parameter exists more than once with different values in cassandra.yaml (was: Emit a warning when the same parameter exists more than once in cassandra.yaml ) > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17379) Emit a warning when the same parameter exists more than once in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson reassigned CASSANDRA-17379: --- Assignee: Marcus Eriksson > Emit a warning when the same parameter exists more than once in > cassandra.yaml > --- > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493267#comment-17493267 ] Marcus Eriksson commented on CASSANDRA-17379: - we should at least refuse starting if the values for old parameters is different from the new ones > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17233) Fix org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage
[ https://issues.apache.org/jira/browse/CASSANDRA-17233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493292#comment-17493292 ] Ekaterina Dimitrova commented on CASSANDRA-17233: - Is this ready to commit? Should we change the ticket status if it is not? > Fix > org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage > --- > > Key: CASSANDRA-17233 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17233 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Fails consistently on trunk: > https://ci-cassandra.apache.org/job/Cassandra-trunk/893/testReport/junit/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerCDCTest/testNonblockingShouldMaintainSteadyDiskUsage/history/ -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17334) Pre hashed passwords in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493304#comment-17493304 ] Bowen Song commented on CASSANDRA-17334: Thanks [~bereng] for the clarification of the reasons behind this proposal. I believe 3rd party software and tools capturing CQL is not the scope of this discussion. It's the 3rd party software/tool's responsibility to redact the sensitive data they capture. That is not limited to the Cassandra authentication credentials, also the sensitive data stored in user created tables. With the 3rd party stuff out of our way, I believe adding tests to ensure password is never logged by Cassandra is sufficient to prevent password leakages from existing features. Future newly added features that logs the CQL statements will of course require new tests to be added, but I don't see this kind of features has been or will be added frequently. If, however, the feature must be added, the hash should ideally be done in the CQLSH and Cassandra client libraries, instead of an external tool. I personally have a distaste for relying on external tools to do basic stuff. In my opinion, it only makes sense to introduce an external tool if the functionality is rarely used and fairly complex or is intended to run independent of the software itself. In this particular case, an external password hashing tool is incontinent to install & use when the user don't have shell access to the Cassandra server. The existence of an external tool will make automation dealing with credentials more complex. The tool can also become a headache if the hash algorithm is changed in the future, because the tool will then need to know which hash algorithm to use in order to match the expectation of the Cassandra server. All the above mentioned issues will go away if the hashing is done in CQLSH or client libraries, the user can simply write the statement in CQLSH: CREATE WITH hashed password = PASSWORD_HASH('whatever') or client library: session.execute("CREATE ... WITH hashed password = ?", session.passwordHash("whatever")) and the CQLSH/library handles the password hash locally before sending the statement to the server. The advantage of this is the automation tool / user / developers don't need to deal with password hashes, and the CQLSH/library knows the Cassandra server version and can choose the correct hash algorithm. > Pre hashed passwords in CQL > --- > > Key: CASSANDRA-17334 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17334 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.1 > > > As seen on CASSANDRA-16801 and friends we are working across the system with > plain text passwords. These can be unintentionally revealed by intermediate > systems. Allowing the use of hashed passwords should mitigate that. The idea > is to add a new option {{HASHED PASSWORD}} for {{CREATE/ALTER ROLE/USER}}. > Examples: > {noformat} > CREATE ROLE foo WITH login = true AND hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > ALTER ROLE foo WITH hashed password = > '$2a$10$JSJEMFm6GeaW9XxT5JIheuEtPvat6i7uKbnTcxX3c1wshIIsGyUtG'; > {noformat} > To generate the password hash, there will be a new tool {{hash_password}} in > resources/cassandra/bin > Based on original works from [~snazy] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17265) dtest byteman errors
[ https://issues.apache.org/jira/browse/CASSANDRA-17265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493317#comment-17493317 ] Ekaterina Dimitrova commented on CASSANDRA-17265: - I see a lot Jolokia having trouble to start lately, can it be related? It happens with different tests both in Circle and Jenkins. Example: [https://ci-cassandra.apache.org/job/Cassandra-trunk/959/testReport/dtest-offheap.client_request_metrics_local_remote_test/TestClientRequestMetricsLocalRemote/test_batch_and_slice/] Thoughts? > dtest byteman errors > > > Key: CASSANDRA-17265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17265 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: NA > > > Testing on another ticket I noticed 4.0 was failing 3 dtests on circle on > missing btm files. I checked and the files _are there_. Also it doesn't repro > locally. See 4.0 vanilla runs > [here|https://app.circleci.com/pipelines/github/bereng/cassandra/555/workflows/21cfe9e4-2d17-45a2-8ea8-1593b7ed6d8b] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17265) dtest byteman errors
[ https://issues.apache.org/jira/browse/CASSANDRA-17265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493324#comment-17493324 ] Brandon Williams commented on CASSANDRA-17265: -- If the jar is found, it's not from this. It may be a bit of a red herring and as it says the port is in use from an orphan process. > dtest byteman errors > > > Key: CASSANDRA-17265 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17265 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: NA > > > Testing on another ticket I noticed 4.0 was failing 3 dtests on circle on > missing btm files. I checked and the files _are there_. Also it doesn't repro > locally. See 4.0 vanilla runs > [here|https://app.circleci.com/pipelines/github/bereng/cassandra/555/workflows/21cfe9e4-2d17-45a2-8ea8-1593b7ed6d8b] -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493328#comment-17493328 ] Brandon Williams commented on CASSANDRA-17379: -- +1 to refusing to start. I'm not sure we should even start if both are present; that seems like trouble waiting to happen too, and config hygiene is quick to fix. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493336#comment-17493336 ] Ekaterina Dimitrova edited comment on CASSANDRA-17379 at 2/16/22, 4:34 PM: --- I agree with [~brandon.williams] too but just want to mention that when I suggested that before I was told that this will mean we don't have full backward compatibility. SnakeYAML is giving the chance to add more than once the same parameter without warning or anything. It just loads whatever value is set last. So currently we keep on doing it but people can start adding the new name with the old one(which is loaded by utilizing the backward compatibility). I personally dislike this but I was told this can be an issue for people and only to add warning. It seems like a tribal knowledge and it is risky IMHO. was (Author: e.dimitrova): I agree with [~brandon.williams] too but just want to mention that when I suggested that before I was told that this will mean we don't have full backward compatibility. SnakeYAML is giving the chance to add more than once the same parameter without warning or anything. It just loads whatever value is set last. So currently we keep on doing it but people can start adding the new one with the old one(which is loaded by utilizing the backward compatibility). I personally dislike this but I was told this can be an issue for people and only to add warning. It seems like a tribal knowledge and it is risky IMHO. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493336#comment-17493336 ] Ekaterina Dimitrova commented on CASSANDRA-17379: - I agree with [~brandon.williams] too but just want to mention that when I suggested that before I was told that this will mean we don't have full backward compatibility. SnakeYAML is giving the chance to add more than once the same parameter without warning or anything. It just loads whatever value is set last. So currently we keep on doing it but people can start adding the new one with the old one(which is loaded by utilizing the backward compatibility). I personally dislike this but I was told this can be an issue for people and only to add warning. It seems like a tribal knowledge and it is risky IMHO. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493336#comment-17493336 ] Ekaterina Dimitrova edited comment on CASSANDRA-17379 at 2/16/22, 4:35 PM: --- I agree with [~brandon.williams] too but just want to mention that when I suggested that before I was told that this will mean we don't have full backward compatibility. SnakeYAML is giving the chance to add more than once the same parameter without warning or anything. It just loads whatever value is set last. So currently we keep on doing it but people can start adding the new name with the old one(which is loaded by utilizing the backward compatibility). I personally dislike this but I was told refusal on startup can be an issue for people and only to add warning. It seems like a tribal knowledge and it is risky IMHO. was (Author: e.dimitrova): I agree with [~brandon.williams] too but just want to mention that when I suggested that before I was told that this will mean we don't have full backward compatibility. SnakeYAML is giving the chance to add more than once the same parameter without warning or anything. It just loads whatever value is set last. So currently we keep on doing it but people can start adding the new name with the old one(which is loaded by utilizing the backward compatibility). I personally dislike this but I was told this can be an issue for people and only to add warning. It seems like a tribal knowledge and it is risky IMHO. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493356#comment-17493356 ] Brandon Williams commented on CASSANDRA-17379: -- I'm fine with a warning, if it says that the behavior may not be present in the next version. Then we are free to remove it later. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493358#comment-17493358 ] David Capwell commented on CASSANDRA-17379: --- what if we block by default with an escape hatch? If tools are creating the YAML they may need time to fix, so a system property to go back to the old behavior could be a solution? So fail by default, opt-out if requested? > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493360#comment-17493360 ] Ekaterina Dimitrova commented on CASSANDRA-17379: - We just had some discussion with [~brandon.williams] and [~blerer] and from everything I heard, I think I like the most the idea [~brandon.williams] has. Let's add warning and deprecate that overloading and say next major it will be probably not happening anymore as it is risky. But we give the chance to people to fix their config and to come back with feedback how much this change can affect them. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493362#comment-17493362 ] Brandon Williams commented on CASSANDRA-17379: -- I like that idea since it will ensure that anyone relying on the behavior is aware of the deprecation. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493362#comment-17493362 ] Brandon Williams edited comment on CASSANDRA-17379 at 2/16/22, 5:14 PM: bq. So fail by default, opt-out if requested? I like that idea since it will ensure that anyone relying on the behavior is aware of the deprecation. was (Author: brandon.williams): I like that idea since it will ensure that anyone relying on the behavior is aware of the deprecation. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17379: Complexity: Normal (was: Low Hanging Fruit) > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493372#comment-17493372 ] Caleb Rackliffe commented on CASSANDRA-17379: - Not sure exactly what the consensus is from reading the above, but I'd... 1.) Fail startup if the duplicate parameters have different values. 2.) WARN on startup if duplicates exist but have the same value. 3.) Provide a system property to opt out of #1. > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[ https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-17292: Description: Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new features") has made it clear we will gravitate toward appropriately nested structures for new parameters in {{cassandra.yaml}}, but from the scattered conversation across a few Guardrails tickets (see CASSANDRA-17212 and CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to eventually extend this to the rest of {{cassandra.yaml}}. The benefits of this change include those we gain by doing it for new features (single point of interest for feature documentation, typed configuration objects, logical grouping for additional parameters added over time, discoverability, etc.), but one a larger scale. This may overlap with ongoing work, including the Guardrails epic. Ideally, even a rough cut of a design here would allow that to move forward in a timely and coherent manner (with less long-term refactoring pain). While these would have to be adjusted to CASSANDRA-15234 (probably after it merges), there have been two proposals floated already for what this might look like: >From [~maedhroz] - >https://github.com/maedhroz/cassandra/commit/450b920e0ac072cec635e0ebcb63538ee7f1fc5a >From [~benedict] - >https://github.com/belliottsmith/cassandra/commits/CASSANDRA-15234-grouping-ideas was: Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new features") has made it clear we will gravitate toward appropriately nested structures for new parameters in {{cassandra.yaml}}, but from the scattered conversation across a few Guardrails tickets (see CASSANDRA-17212 and CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to eventually extend this to the rest of {{cassandra.yaml}}. The benefits of this change include those we gain by doing it for new features (single point of interest for feature documentation, typed configuration objects, logical grouping for additional parameters added over time, discoverability, etc.), but one a larger scale. This may overlap with ongoing work, including the Guardrails epic. Ideally, even a rough cut of a design here would allow that to move forward in a timely and coherent manner (with less long-term refactoring pain). While these would have to be adjusted to CASSANDRA-15234 (probably after it merges), there have been two proposals floated already for what this might look like: >From [~maedhroz] - >https://github.com/maedhroz/cassandra/commit/49e83c70eba3357978d1081ecf500bbbdee960d8 >From [~benedict] - >https://github.com/belliottsmith/cassandra/commits/CASSANDRA-15234-grouping-ideas > Move cassandra.yaml toward a nested structure around major database concepts > > > Key: CASSANDRA-17292 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17292 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 5.x > > > Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new > features") has made it clear we will gravitate toward appropriately nested > structures for new parameters in {{cassandra.yaml}}, but from the scattered > conversation across a few Guardrails tickets (see CASSANDRA-17212 and > CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to > eventually extend this to the rest of {{cassandra.yaml}}. The benefits of > this change include those we gain by doing it for new features (single point > of interest for feature documentation, typed configuration objects, logical > grouping for additional parameters added over time, discoverability, etc.), > but one a larger scale. > This may overlap with ongoing work, including the Guardrails epic. Ideally, > even a rough cut of a design here would allow that to move forward in a > timely and coherent manner (with less long-term refactoring pain). > While these would have to be adjusted to CASSANDRA-15234 (probably after it > merges), there have been two proposals floated already for what this might > look like: > From [~maedhroz] - > https://github.com/maedhroz/cassandra/commit/450b920e0ac072cec635e0ebcb63538ee7f1fc5a > From [~benedict] - > https://github.com/belliottsmith/cassandra/commits/CASSANDRA-15234-grouping-ideas -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17277) Automate updating tickets with CI results after merge
[ https://issues.apache.org/jira/browse/CASSANDRA-17277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17277: -- Change Category: Quality Assurance Complexity: Normal Component/s: Build CI Status: Open (was: Triage Needed) > Automate updating tickets with CI results after merge > - > > Key: CASSANDRA-17277 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17277 > Project: Cassandra > Issue Type: Task > Components: Build, CI >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > > From [the wiki > article|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=199530280] > {quote}ci-cassandra run bot updating ticket in JIRA w/state of test run for > the SHA (JIRA Pending; to be linked) > {quote} > We already run CI for every commit (see > [example|https://ci-cassandra.apache.org/job/Cassandra-trunk/]). The goal is > to have automation that'll tie the results of a commit to the original JIRA > ticket and update it automatically w/a comment indicating: > * The results of the CI run (duration, pass, fail, # failures) > * Any potential regressions in CI from the merge w/links to job history > ([example|https://ci-cassandra.apache.org/job/Cassandra-trunk/912/testReport/junit/dtest.cqlsh_tests.test_cqlsh_copy/TestCqlshCopy/test_bulk_round_trip_with_timeouts/history/]) > From a workflow perspective we want to optimize for having as minimal > friction as possible to see the impact of one's commit on ci-cassandra and > rapidly verify if their change appears to have destabilized any tests on that > infrastructure. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17277) Automate updating tickets with CI results after merge
[ https://issues.apache.org/jira/browse/CASSANDRA-17277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493385#comment-17493385 ] Josh McKenzie commented on CASSANDRA-17277: --- Ok, got something working locally with the following usage ergonomics: {code:java} usage: jenkins-jira-integration.py [-h] [--jenurl JENURL] --jenuser JENUSER --jenpass JENPASS --jiraurl JIRAURL --jirauser JIRAUSER --jirapass JIRAPASS --branch b [--buildnum n] [--verbose] Parse Jenkins build output and update JIRA tickets with results for a single branch required arguments: --jenurl JENURL Jenkins server url --jenuser JENUSERJIRA server url --jenpass JENPASSJIRA username --jiraurl JIRAURLJIRA server url --jirauser JIRAUSER JIRA username --jirapass JIRAPASS JIRA password --branch b Branch Versions to pull Jenkins results for --buildnum n Build number to process for CI status optional arguments: --verboseVerbose logging {code} The usage of the above can be triggered upon completion of a CI run in Jenkins that passes in the branch and number of the build; the rest of it is taken care of by the script. The current data and format of results it'd post into a JIRA comment (I ran this against build 940 on trunk; arbitrary pick): --- [CI Results] Branch: trunk, build number: 940 jenkins url: [https://ci-cassandra.apache.org/job/Cassandra-trunk/940/] JIRA: 17352 commit url: [https://git-wip-us.apache.org/repos/asf?p=cassandra.git;a=commit;h=5c9ba06dd31157cd224af2cec75521fefe2c9883] affected paths: * src/java/org/apache/cassandra/config/DatabaseDescriptor.java * src/java/org/apache/cassandra/cql3/functions/UDFunction.java * src/java/org/apache/cassandra/config/Config.java * src/java/org/apache/cassandra/cql3/functions/ThreadAwareSecurityManager.java Build Result: UNSTABLE Passing Tests: 42908 Failing Tests: 11 ||Test|Failures|JIRA| |dtest.cqlsh_tests.test_cqlsh_copy.TestCqlshCopy.test_explicit_column_order_reading|1 of 17|[No JIRA found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]| |org.apache.cassandra.distributed.upgrade.CompactStorageUpgradeTest.compactStorageUpgradeTest|4 of 17|[No JIRA found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]| |dtest.replica_side_filtering_test.TestSecondaryIndexes.test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions|1 of 17|[No JIRA found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]| |dtest.snapshot_test.TestArchiveCommitlog.test_archive_commitlog_point_in_time_with_active_commitlog|1 of 17|[No JIRA found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]| |dtest.repair_tests.incremental_repair_test.TestIncRepair.test_repaired_tracking_with_partition_deletes|1 of 17|[No JIRA found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]| |dtest.paging_test.TestPagingDatasetChanges.test_data_change_impacting_later_page|1 of 17|[No JIRA found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]| |dtest.user_functions_test.TestUserFunctions.test_udf_with_udt|1 of 17|[No JIRA found|https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=496&quickFilter=2252]| |org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage|13 of 17|CASSANDRA-17233| For an engineer that worked on a ticket (in this case CASSANDRA-17352), the implication is that special attention should be paid to any test failure with only 1 failure in the last N builds as this is the first time we've seen it so it could have been caused by this change. Notably, [the butler run for 940|https://butler.cassandra.apache.org/#/ci/upstream/compare/Cassandra-trunk/trunk] doesn't show the {{TestArchiveCommitlog}} failure, however in Jenkins if you go to all test failure results for the build [you see it there|https://ci-cassandra.apache.org/job/Cassandra-trunk/940/testReport/] The above output is created by taking the passed in build number and walking back (and locally caching) the results of all valid previous builds. This local data is kept in a simple JSON file per branch which we then use for subsequent calculation of the # of runs we've seen a given test in and the # of failures; this is significantly faster (and less offensive) than repeatedly downloading 50+M of text data per build. The inference of the Jira column is based on a query with summary ~ the last 2 tokens of the test name . delimited. If it finds more than 1 it should link to the same open test failures kanban board as linked when no Jira ticket is found. This functionality is of
[jira] [Updated] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksei Zotov updated CASSANDRA-17386: -- Test and Documentation Plan: I fixed test issue. No documentation updates are required. Status: Patch Available (was: In Progress) > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493179#comment-17493179 ] Aleksei Zotov edited comment on CASSANDRA-17386 at 2/16/22, 6:00 PM: - Looks like I found the issue in tests configuration. CI looks good. ||Item||Link|| |PR|[https://github.com/apache/cassandra/pull/1450]| |CI|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/]| [~e.dimitrova] would you mind reviewing the changes. was (Author: azotcsit): Looks like I found the issue in tests configuration. Raised a PR and triggered CI, let's wait and see. ||Item||Link|| |PR|[https://github.com/apache/cassandra/pull/1450]| |CI|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/]| > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksei Zotov updated CASSANDRA-17386: -- Status: Needs Committer (was: Patch Available) > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493393#comment-17493393 ] Brandon Williams commented on CASSANDRA-17386: -- Do we know why this isn't failing on other branches? > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17283) Failing test: UNIT org.apache.cassandra.tools.BulkLoaderTest.testBulkLoader_WithArgs1
[ https://issues.apache.org/jira/browse/CASSANDRA-17283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493396#comment-17493396 ] Brandon Williams commented on CASSANDRA-17283: -- Looks good and the multiplexer doesn't lie, +1. > Failing test: UNIT > org.apache.cassandra.tools.BulkLoaderTest.testBulkLoader_WithArgs1 > - > > Key: CASSANDRA-17283 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17283 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Josh McKenzie >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.x > > > Intermittent failure: > [https://butler.cassandra.apache.org/#/ci/upstream/workflow/Cassandra-trunk/failure/org.apache.cassandra.tools/BulkLoaderTest/testBulkLoader_WithArgs1] > h3. Error Message > Wrong thread status, active threads unaccounted for: > [cluster2-connection-reaper-0] > h3. Stacktrace > junit.framework.AssertionFailedError: Wrong thread status, active threads > unaccounted for: [cluster2-connection-reaper-0] at > org.apache.cassandra.tools.OfflineToolUtils.assertNoUnexpectedThreadsStarted(OfflineToolUtils.java:106) > at > org.apache.cassandra.tools.BulkLoaderTest.testBulkLoader_WithArgs1(BulkLoaderTest.java:85) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > h3. Standard Output > DEBUG [main] 2022-01-21 19:11:46,606 InternalLoggerFactory.java:63 - Using > SLF4J as the default logging framework DEBUG [main] 2022-01-21 19:11:46,634 > PlatformDependent0.java:417 - -Dio.netty.noUnsafe: false DEBUG [main] > 2022-01-21 19:11:46,634 PlatformDependent0.java:897 - Java version: 8 DEBUG > [main] 2022-01-21 19:11:46,636 PlatformDependent0.java:130 - > sun.misc.Unsafe.theUnsafe: available DEBUG [main] 2022-01-21 19:11:46,636 > PlatformDependent0.java:154 - sun.misc.Unsafe.copyMemory: available -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493393#comment-17493393 ] Brandon Williams edited comment on CASSANDRA-17386 at 2/16/22, 6:20 PM: Do we know why this isn't failing on 3.0? was (Author: brandon.williams): Do we know why this isn't failing on other branches? > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[ https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493407#comment-17493407 ] Benedict Elliott Smith commented on CASSANDRA-17292: This iteration looks much better, FWIW. There remain some inconsistencies to address, that I'll comment on in due course, but at first glance they appear manageable. > Move cassandra.yaml toward a nested structure around major database concepts > > > Key: CASSANDRA-17292 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17292 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 5.x > > > Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new > features") has made it clear we will gravitate toward appropriately nested > structures for new parameters in {{cassandra.yaml}}, but from the scattered > conversation across a few Guardrails tickets (see CASSANDRA-17212 and > CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to > eventually extend this to the rest of {{cassandra.yaml}}. The benefits of > this change include those we gain by doing it for new features (single point > of interest for feature documentation, typed configuration objects, logical > grouping for additional parameters added over time, discoverability, etc.), > but one a larger scale. > This may overlap with ongoing work, including the Guardrails epic. Ideally, > even a rough cut of a design here would allow that to move forward in a > timely and coherent manner (with less long-term refactoring pain). > While these would have to be adjusted to CASSANDRA-15234 (probably after it > merges), there have been two proposals floated already for what this might > look like: > From [~maedhroz] - > https://github.com/maedhroz/cassandra/commit/450b920e0ac072cec635e0ebcb63538ee7f1fc5a > From [~benedict] - > https://github.com/belliottsmith/cassandra/commits/CASSANDRA-15234-grouping-ideas -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493408#comment-17493408 ] Aleksei Zotov commented on CASSANDRA-17386: --- Good point [~brandon.williams]! For 4.0 and 4.x the issue has been fixed in [this PR|https://github.com/apache/cassandra/commit/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79#diff-4f28d77309cd96fadc18b2e74fb7a0c2c040ec6e68e551e1198eb2361c8a56a7R56]. I additionally checked 3.0 and looks like it has the same issue, however, it is not being reported. Here are the details: [https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0/248/consoleFull] (latest 3.0): {code:java} 14:42:08 Starting building: Cassandra-3.0-cqlsh-tests #305 {code} and in [https://ci-cassandra.apache.org/job/Cassandra-3.0-cqlsh-tests/305/#showFailuresLink] there are _TestCqlshOutput_ failures. Any idea why these failures are not being reported as failures at [https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0/248/#showFailuresLink]? I was able to reproduce the issue for 3.0 locally. Hence I ported the same fix to 3.0 branch, tested locally and triggered CI. ||Item||Link|| |PR 3.11|[https://github.com/apache/cassandra/pull/1450]| |CI 3.11|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/] (looks good)| |PR 3.0|[https://github.com/apache/cassandra/pull/1451/]| |CI 3.0|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1431/] (running now)| > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493408#comment-17493408 ] Aleksei Zotov edited comment on CASSANDRA-17386 at 2/16/22, 6:33 PM: - Good point [~brandon.williams]! For 4.0 and 4.x the issue has been fixed in [this PR|https://github.com/apache/cassandra/commit/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79#diff-4f28d77309cd96fadc18b2e74fb7a0c2c040ec6e68e551e1198eb2361c8a56a7R56]. I additionally checked 3.0 and looks like it has the same issue, however, it is not being reported. Here are the details: [https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0/248/consoleFull] (latest 3.0): {code:java} 14:42:08 Starting building: Cassandra-3.0-cqlsh-tests #305 {code} and in [https://ci-cassandra.apache.org/job/Cassandra-3.0-cqlsh-tests/305/#showFailuresLink] there are _TestCqlshOutput_ failures. Any idea why these failures are not being reported as failures at [https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0/248/#showFailuresLink]? I was able to reproduce the issue for 3.0 locally. Hence I ported the same fix to 3.0 branch, tested locally and triggered CI. ||Item||Link|| |PR 3.11|[https://github.com/apache/cassandra/pull/1450]| |CI 3.11|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/] (looks good)| |PR 3.0|[https://github.com/apache/cassandra/pull/1451/]| |CI 3.0|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1431/] (running now)| was (Author: azotcsit): Good point [~brandon.williams]! For 4.0 and 4.x the issue has been fixed in [this PR|https://github.com/apache/cassandra/commit/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79#diff-4f28d77309cd96fadc18b2e74fb7a0c2c040ec6e68e551e1198eb2361c8a56a7R56]. I additionally checked 3.0 and looks like it has the same issue, however, it is not being reported. Here are the details: [https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0/248/consoleFull] (latest 3.0): {code:java} 14:42:08 Starting building: Cassandra-3.0-cqlsh-tests #305 {code} and in [https://ci-cassandra.apache.org/job/Cassandra-3.0-cqlsh-tests/305/#showFailuresLink] there are _TestCqlshOutput_ failures. Any idea why these failures are not being reported as failures at [https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0/248/#showFailuresLink]? I was able to reproduce the issue for 3.0 locally. Hence I ported the same fix to 3.0 branch, tested locally and triggered CI. ||Item||Link|| |PR 3.11|[https://github.com/apache/cassandra/pull/1450]| |CI 3.11|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1429/] (looks good)| |PR 3.0|[https://github.com/apache/cassandra/pull/1451/]| |CI 3.0|[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1431/] (running now)| > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493432#comment-17493432 ] Brandon Williams commented on CASSANDRA-17386: -- Hmm, those jobs are generated from a template, so I'm not sure why 3.11 would report a failure but 3.0 would not... maybe [~mck] has more insight. > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17233) Fix org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage
[ https://issues.apache.org/jira/browse/CASSANDRA-17233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493454#comment-17493454 ] Yifan Cai commented on CASSANDRA-17233: --- [~e.dimitrova], thanks for checking. The patch is ready to commit. Before that, I need to trigger a run in Jenkins to verify there is no more failures. Looks like it is so from the [cdc test suite|https://ci-cassandra.apache.org/job/Cassandra-devbranch/1421/testReport/], so we are good. I was waiting for the build to start. On the day that I scheduled the build, it was not started and continued as not started in the second day, and I did not check afterwards until your reminder :D I am going to commit it today. > Fix > org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage > --- > > Key: CASSANDRA-17233 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17233 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Fails consistently on trunk: > https://ci-cassandra.apache.org/job/Cassandra-trunk/893/testReport/junit/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerCDCTest/testNonblockingShouldMaintainSteadyDiskUsage/history/ -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493455#comment-17493455 ] Michael Semb Wever commented on CASSANDRA-17386: bq. Hmm, those jobs are generated from a template, so I'm not sure why 3.11 would report a failure but 3.0 would not... maybe Michael Semb Wever has more insight. Very odd. I'll need to look into that… > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17166) Enhance SnakeYAML properties to be reusable outside of YAML parsing, support camel case conversion to snake case, and add support to ignore properties
[ https://issues.apache.org/jira/browse/CASSANDRA-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493458#comment-17493458 ] Jeremiah Jordan commented on CASSANDRA-17166: - This property support sounds great. One nitpick, I would use something different than just: -Dcassandra.[Config name]. Maybe -Dcassandra.config.[Config Name] or -Dcassandra.settings.[Config Name] or -Dcassandraconfig.[Config Name]. > Enhance SnakeYAML properties to be reusable outside of YAML parsing, support > camel case conversion to snake case, and add support to ignore properties > -- > > Key: CASSANDRA-17166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17166 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.x > > Time Spent: 3.5h > Remaining Estimate: 0h > > SnakeYaml is rather limited in the “object mapping” layer, which forces our > internal code to match specific patterns (all fields public and camel case); > we can remove this restriction by leveraging Jackson for property lookup, and > leaving the YAML handling to SnakeYAML -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17166) Enhance SnakeYAML properties to be reusable outside of YAML parsing, support camel case conversion to snake case, and add support to ignore properties
[ https://issues.apache.org/jira/browse/CASSANDRA-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493458#comment-17493458 ] Jeremiah Jordan edited comment on CASSANDRA-17166 at 2/16/22, 7:29 PM: --- This property support sounds great. One nitpick, I would use something different than just: -Dcassandra.[Config name]. Maybe -Dcassandra.config.[Config Name] or -Dcassandra.settings.[Config Name] or -Dcassandraconfig.[Config Name]. Otherwise there could be confusion between config properties and non config properties. was (Author: jjordan): This property support sounds great. One nitpick, I would use something different than just: -Dcassandra.[Config name]. Maybe -Dcassandra.config.[Config Name] or -Dcassandra.settings.[Config Name] or -Dcassandraconfig.[Config Name]. > Enhance SnakeYAML properties to be reusable outside of YAML parsing, support > camel case conversion to snake case, and add support to ignore properties > -- > > Key: CASSANDRA-17166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17166 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.x > > Time Spent: 3.5h > Remaining Estimate: 0h > > SnakeYaml is rather limited in the “object mapping” layer, which forces our > internal code to match specific patterns (all fields public and camel case); > we can remove this restriction by leveraging Jackson for property lookup, and > leaving the YAML handling to SnakeYAML -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493460#comment-17493460 ] Marcus Eriksson commented on CASSANDRA-17379: - {quote}1.) Fail startup if the duplicate parameters have different values. 2.) WARN on startup if duplicates exist but have the same value. 3.) Provide a system property to opt out of #1.{quote} +1 > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17233) Fix org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage
[ https://issues.apache.org/jira/browse/CASSANDRA-17233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493465#comment-17493465 ] Ekaterina Dimitrova commented on CASSANDRA-17233: - Thanks[~yifanc] and no worries, I am just a Build Lead this week so that’s how I noticed it. With all fires lately, it is easy to forget 😁 > Fix > org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDCTest.testNonblockingShouldMaintainSteadyDiskUsage > --- > > Key: CASSANDRA-17233 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17233 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Fails consistently on trunk: > https://ci-cassandra.apache.org/job/Cassandra-trunk/893/testReport/junit/org.apache.cassandra.db.commitlog/CommitLogSegmentManagerCDCTest/testNonblockingShouldMaintainSteadyDiskUsage/history/ -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493467#comment-17493467 ] David Capwell commented on CASSANDRA-17379: --- +1 > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17166) Enhance SnakeYAML properties to be reusable outside of YAML parsing, support camel case conversion to snake case, and add support to ignore properties
[ https://issues.apache.org/jira/browse/CASSANDRA-17166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493468#comment-17493468 ] David Capwell commented on CASSANDRA-17166: --- bq. One nitpick, I would use something different than just: -Dcassandra.[Config name]. Maybe -Dcassandra.config.[Config Name] or -Dcassandra.settings.[Config Name] or -Dcassandraconfig.[Config Name]. Good point, ill poke others to see how they feel. I prefer -Dcassandra.[Config name] but totally understand your point, and we currently do -Dcassandra.[Config name] for non-config settings, so could cause an overlap (not checked) > Enhance SnakeYAML properties to be reusable outside of YAML parsing, support > camel case conversion to snake case, and add support to ignore properties > -- > > Key: CASSANDRA-17166 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17166 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.x > > Time Spent: 3.5h > Remaining Estimate: 0h > > SnakeYaml is rather limited in the “object mapping” layer, which forces our > internal code to match specific patterns (all fields public and camel case); > we can remove this restriction by leveraging Jackson for property lookup, and > leaving the YAML handling to SnakeYAML -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17379) Fail starting when the same parameter exists more than once with different values in cassandra.yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-17379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493472#comment-17493472 ] Ekaterina Dimitrova commented on CASSANDRA-17379: - +1 > Fail starting when the same parameter exists more than once with different > values in cassandra.yaml > > > Key: CASSANDRA-17379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Marcus Eriksson >Priority: Low > Fix For: 4.x > > > The way that SnakeYAML works, if someone has added the same parameter more > than once - the last occurrence will be the one that will take precedence. > Now after CASSANDRA-15234 we can even add the parameter with the old name and > with the new name and the occurrences will replace each other. Again, > whatever is last in cassandra.yaml will take precedence. Example: > If you add the following in cassandra.yaml > {code:java} > hinted_handoff_enabled: true > enabled_hinted_handolff: false > {code} > you will get loaded in Config - > {code:java} > hinted_handoff_enabled: false{code} > //there is backward compatibility from the old name to load into the new one > Currently Cassandra prints in the logs what config was loaded but it is good > also to detect the case mentioned and emit a warning for the user so they can > verify that the value they wanted was loaded in config. > To do that you might want to look at the PropertiesChecker and the way we > emit other warnings in > [check()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/YamlConfigurationLoader.java#L376] > in YamlConfigurationLoader. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17293) Update python test framework from nose to pytest
[ https://issues.apache.org/jira/browse/CASSANDRA-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493475#comment-17493475 ] Brandon Williams commented on CASSANDRA-17293: -- [~bereng] I think your problem locally is you need to clean up '.ccm' from a previous run, the script doesn't do it. The jenkins run you linked is broken in some other way, it says no tests were run so there's nothing to troubleshoot. This passes on circle and it passes for me locally and I've got another jenkins attempt going: [!https://ci-cassandra.apache.org/job/Cassandra-devbranch/1430/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1430/pipeline] > Update python test framework from nose to pytest > > > Key: CASSANDRA-17293 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17293 > Project: Cassandra > Issue Type: Task > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Brad Schoening >Priority: Normal > Fix For: 4.x > > > I had trouble trying to install and run the python nose test from pip > (nosetest not found). > According to the homepage of nose at [https://nose.readthedocs.io/en/latest/] > h1. _Note to Users_ > _Nose has been in maintenance mode for the past several years and will likely > cease without a new person/team to take over maintainership. New projects > should consider using [Nose2|https://github.com/nose-devs/nose2], > [py.test|http://pytest.org/], or just plain unittest/unittest2._ > > Upgrading to pytest is likely the least effort. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17293) Update python test framework from nose to pytest
[ https://issues.apache.org/jira/browse/CASSANDRA-17293?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482252#comment-17482252 ] Brad Schoening edited comment on CASSANDRA-17293 at 2/16/22, 8:32 PM: -- Please do. It looks pretty straightforward... * change assert_raises -> self.assertRaises * since @notest isn't supported, change the name of testrun_cqlsh() * update connect to conn = Cluster((TEST_HOST,), TEST_PORT, cql_version=cql_version, load_balancing_policy=RoundRobinPolicy()) * add from cassandra.metadata import maybe_escape_name as quote_name was (Author: bschoeni): Please do. It looks pretty straightforward... * change assert_raises -> self.assertRaises * since @notest isn't supported, change the name of testrun_cqlsh() * update connect to conn = Cluster((TEST_HOST,), TEST_PORT, cql_version=cql_version, load_balancing_policy=RoundRobinPolicy()) * add from cassandra.metadata import maybe_escape_name as quote_name > Update python test framework from nose to pytest > > > Key: CASSANDRA-17293 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17293 > Project: Cassandra > Issue Type: Task > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Brad Schoening >Priority: Normal > Fix For: 4.x > > > I had trouble trying to install and run the python nose test from pip > (nosetest not found). > According to the homepage of nose at [https://nose.readthedocs.io/en/latest/] > h1. _Note to Users_ > _Nose has been in maintenance mode for the past several years and will likely > cease without a new person/team to take over maintainership. New projects > should consider using [Nose2|https://github.com/nose-devs/nose2], > [py.test|http://pytest.org/], or just plain unittest/unittest2._ > > Upgrading to pytest is likely the least effort. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17334) Pre hashed passwords in CQL
[ https://issues.apache.org/jira/browse/CASSANDRA-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493304#comment-17493304 ] Bowen Song edited comment on CASSANDRA-17334 at 2/16/22, 9:46 PM: -- Thanks [~bereng] for the clarification of the reasons behind this proposal. I believe 3rd party software and tools capturing CQL is not the scope of this discussion. It's the 3rd party software/tool's responsibility to redact the sensitive data they capture. That is not limited to the Cassandra authentication credentials, also the sensitive data stored in user created tables. With the 3rd party stuff out of our way, I believe adding tests to ensure password is never logged by Cassandra is sufficient to prevent password leakages from existing features. Future newly added features that logs the CQL statements will of course require new tests to be added, but I don't see this kind of features has been or will be added frequently. If, however, the feature must be added, the hash should ideally be done in the CQLSH and Cassandra client libraries, instead of an external tool. I personally have a distaste for relying on external tools to do basic stuff. In my opinion, it only makes sense to introduce an external tool if the functionality is rarely used and fairly complex or is intended to run independent of the software itself. In this particular case, an external password hashing tool is inconvenient to install & use when the user don't have shell access to the Cassandra server. The existence of an external tool will make automation dealing with credentials more complex. The tool can also become a headache if the hash algorithm is changed in the future, because the tool will then need to know which hash algorithm to use in order to match the expectation of the Cassandra server. All the above mentioned issues will go away if the hashing is done in CQLSH or client libraries, the user can simply write the statement in CQLSH: CREATE WITH hashed password = PASSWORD_HASH('whatever') or client library: session.execute("CREATE ... WITH hashed password = ?", session.passwordHash("whatever")) and the CQLSH/library handles the password hash locally before sending the statement to the server. The advantage of this is the automation tool / user / developers don't need to deal with password hashes, and the CQLSH/library knows the Cassandra server version and can choose the correct hash algorithm. was (Author: bowen song): Thanks [~bereng] for the clarification of the reasons behind this proposal. I believe 3rd party software and tools capturing CQL is not the scope of this discussion. It's the 3rd party software/tool's responsibility to redact the sensitive data they capture. That is not limited to the Cassandra authentication credentials, also the sensitive data stored in user created tables. With the 3rd party stuff out of our way, I believe adding tests to ensure password is never logged by Cassandra is sufficient to prevent password leakages from existing features. Future newly added features that logs the CQL statements will of course require new tests to be added, but I don't see this kind of features has been or will be added frequently. If, however, the feature must be added, the hash should ideally be done in the CQLSH and Cassandra client libraries, instead of an external tool. I personally have a distaste for relying on external tools to do basic stuff. In my opinion, it only makes sense to introduce an external tool if the functionality is rarely used and fairly complex or is intended to run independent of the software itself. In this particular case, an external password hashing tool is incontinent to install & use when the user don't have shell access to the Cassandra server. The existence of an external tool will make automation dealing with credentials more complex. The tool can also become a headache if the hash algorithm is changed in the future, because the tool will then need to know which hash algorithm to use in order to match the expectation of the Cassandra server. All the above mentioned issues will go away if the hashing is done in CQLSH or client libraries, the user can simply write the statement in CQLSH: CREATE WITH hashed password = PASSWORD_HASH('whatever') or client library: session.execute("CREATE ... WITH hashed password = ?", session.passwordHash("whatever")) and the CQLSH/library handles the password hash locally before sending the statement to the server. The advantage of this is the automation tool / user / developers don't need to deal with password hashes, and the CQLSH/library knows the Cassandra server version and can choose the correct hash algorithm. > Pre hashed passwords in CQL > --- > > Key: CASSANDRA-17334 > URL: https://issues.apache.org/
[jira] [Commented] (CASSANDRA-17386) Test failure: TestCqlshOutput failing tests
[ https://issues.apache.org/jira/browse/CASSANDRA-17386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493520#comment-17493520 ] Michael Semb Wever commented on CASSANDRA-17386: bq. so I'm not sure why 3.11 would report a failure but 3.0 would not In https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0/248/consoleFull there is the following… {noformat} 11:55:51 ERROR: missing test xml files 11:55:51 ERROR: Unable to find project for artifact copy: Cassandra-3.0-cqlsh-test {noformat} But the cqlshlib.xml is there… e.g. and archived in https://nightlies.apache.org/cassandra/cassandra-3.0/Cassandra-3.0-cqlsh-tests/305/Cassandra-3.0-cqlsh-tests/cython=no,jdk=jdk_1.8_latest,label=cassandra/ Boils down to a typo here: https://github.com/apache/cassandra/blob/cassandra-3.0/.jenkins/Jenkinsfile#L161 (the first argument to copyTestResults should be 'cqlsh-tests') > Test failure: TestCqlshOutput failing tests > > > Key: CASSANDRA-17386 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17386 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Ekaterina Dimitrova >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 3.11.x > > > The following TestCqlshOutput tests are failing consistently on 3.11 and as > far as Butler goes, since the very beginning it started collecting data so I > am not able to say at this point what exactly broke them: > Test_static_cf_output, test_boolean_output, test_count_output, > test_string_output_ascii, test_null_output, test_columness_key_output, > test_numeric_output -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] annotated tag cassandra-4.0.3 created (now 381e915)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to annotated tag cassandra-4.0.3 in repository https://gitbox.apache.org/repos/asf/cassandra.git. at 381e915 (tag) tagging a87055d56a33a9b17606f14535f48eb461965b82 (commit) replaces cassandra-4.0.2 by Mick Semb Wever on Wed Feb 16 23:03:31 2022 +0100 - Log - Apache Cassandra 4.0.3 release --- No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] tag 4.0.3-tentative deleted (was a87055d)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to tag 4.0.3-tentative in repository https://gitbox.apache.org/repos/asf/cassandra.git. *** WARNING: tag 4.0.3-tentative was deleted! *** was a87055d Prepare debian changelog for 4.0.3 The revisions that were on this tag are still contained in other references; therefore, this change does not discard any commits from the repository. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52584 - /dev/cassandra/4.0.3/ /release/cassandra/4.0.3/
Author: mck Date: Wed Feb 16 22:03:55 2022 New Revision: 52584 Log: Apache Cassandra 4.0.3 release Added: release/cassandra/4.0.3/ - copied from r52583, dev/cassandra/4.0.3/ Removed: dev/cassandra/4.0.3/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52586 - in /release/cassandra: 4.0.3/debian/pool/main/c/cassandra/cassandra-tools_4.0.3_all.deb debian/pool/main/c/cassandra/cassandra-tools_4.0.3_all.deb
Author: mck Date: Wed Feb 16 22:09:27 2022 New Revision: 52586 Log: Apache Cassandra 4.0.3 debian artifact cassandra-tools_4.0.3_all.deb Added: release/cassandra/debian/pool/main/c/cassandra/cassandra-tools_4.0.3_all.deb - copied unchanged from r52585, release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra-tools_4.0.3_all.deb Removed: release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra-tools_4.0.3_all.deb - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52587 - in /release/cassandra: 4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3.dsc debian/pool/main/c/cassandra/cassandra_4.0.3.dsc
Author: mck Date: Wed Feb 16 22:09:30 2022 New Revision: 52587 Log: Apache Cassandra 4.0.3 debian artifact cassandra_4.0.3.dsc Added: release/cassandra/debian/pool/main/c/cassandra/cassandra_4.0.3.dsc - copied unchanged from r52586, release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3.dsc Removed: release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3.dsc - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52588 - in /release/cassandra: 4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3.tar.gz debian/pool/main/c/cassandra/cassandra_4.0.3.tar.gz
Author: mck Date: Wed Feb 16 22:09:33 2022 New Revision: 52588 Log: Apache Cassandra 4.0.3 debian artifact cassandra_4.0.3.tar.gz Added: release/cassandra/debian/pool/main/c/cassandra/cassandra_4.0.3.tar.gz - copied unchanged from r52587, release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3.tar.gz Removed: release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3.tar.gz - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52589 - in /release/cassandra: 4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3_all.deb debian/pool/main/c/cassandra/cassandra_4.0.3_all.deb
Author: mck Date: Wed Feb 16 22:09:36 2022 New Revision: 52589 Log: Apache Cassandra 4.0.3 debian artifact cassandra_4.0.3_all.deb Added: release/cassandra/debian/pool/main/c/cassandra/cassandra_4.0.3_all.deb - copied unchanged from r52588, release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3_all.deb Removed: release/cassandra/4.0.3/debian/pool/main/c/cassandra/cassandra_4.0.3_all.deb - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52590 - /release/cassandra/debian/dists/40x/
Author: mck Date: Wed Feb 16 22:09:37 2022 New Revision: 52590 Log: Apache Cassandra 4.0.3 debian artifacts Removed: release/cassandra/debian/dists/40x/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52592 - /release/cassandra/4.0.3/debian/dists/
Author: mck Date: Wed Feb 16 22:09:41 2022 New Revision: 52592 Log: Apache Cassandra 4.0.3 debian artifacts Removed: release/cassandra/4.0.3/debian/dists/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52591 - in /release/cassandra: 4.0.3/debian/dists/40x/ debian/dists/40x/
Author: mck Date: Wed Feb 16 22:09:40 2022 New Revision: 52591 Log: Apache Cassandra 4.0.3 debian artifacts Added: release/cassandra/debian/dists/40x/ - copied from r52590, release/cassandra/4.0.3/debian/dists/40x/ Removed: release/cassandra/4.0.3/debian/dists/40x/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52593 - /release/cassandra/4.0.3/debian/pool/
Author: mck Date: Wed Feb 16 22:09:42 2022 New Revision: 52593 Log: Apache Cassandra 4.0.3 debian artifacts Removed: release/cassandra/4.0.3/debian/pool/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52594 - /release/cassandra/redhat/40x/
Author: mck Date: Wed Feb 16 22:10:28 2022 New Revision: 52594 Log: Apache Cassandra 4.0.3 redhat artifacts Removed: release/cassandra/redhat/40x/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
svn commit: r52595 - in /release/cassandra: 4.0.3/redhat/ redhat/40x/
Author: mck Date: Wed Feb 16 22:10:29 2022 New Revision: 52595 Log: Apache Cassandra 4.0.3 redhat artifacts Added: release/cassandra/redhat/40x/ - copied from r52594, release/cassandra/4.0.3/redhat/ Removed: release/cassandra/4.0.3/redhat/ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17327) Fix flaky test - dtest.offline_tools_test.TestOfflineTools.test_sstableverify
[ https://issues.apache.org/jira/browse/CASSANDRA-17327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17493539#comment-17493539 ] Brandon Williams commented on CASSANDRA-17327: -- Apparently 3.0 is just on the cusp of sometimes creating 4 sstables, which causes autocompaction to kick in, resulting in a single sstable. I have a [branch that disables autocompaction|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-17327], and to finally be done with this test here are repeated runs passing for everything: [3.0|https://app.circleci.com/pipelines/github/driftx/cassandra/367/workflows/c9107c4c-e921-4a25-a0f1-a470bccc4434/jobs/4172], [3.11|https://app.circleci.com/pipelines/github/driftx/cassandra/369/workflows/28af769d-05fb-4dae-81c2-084353dd0e2f/jobs/4192], [4.0|https://app.circleci.com/pipelines/github/driftx/cassandra/366/workflows/2a0547f6-f5b7-4261-ab32-5f95ccf57390/jobs/4151], and [trunk|https://app.circleci.com/pipelines/github/driftx/cassandra/368/workflows/e114c12f-8b94-4e54-bdcb-944c4f3c00cb/jobs/4178]. > Fix flaky test - dtest.offline_tools_test.TestOfflineTools.test_sstableverify > - > > Key: CASSANDRA-17327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17327 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x > > > Failed 15 times in the last 16 runs. Flakiness: 13%, Stability: 6% > Error Message > IndexError: list index out of range -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17327) Fix flaky test - dtest.offline_tools_test.TestOfflineTools.test_sstableverify
[ https://issues.apache.org/jira/browse/CASSANDRA-17327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-17327: - Test and Documentation Plan: run test repeatedly Status: Patch Available (was: Open) > Fix flaky test - dtest.offline_tools_test.TestOfflineTools.test_sstableverify > - > > Key: CASSANDRA-17327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17327 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x > > > Failed 15 times in the last 16 runs. Flakiness: 13%, Stability: 6% > Error Message > IndexError: list index out of range -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit f0bb2996dd7712c78548475ef01b2e03b59707a3 Merge: eeec360 c586cca Author: Brandon Williams AuthorDate: Wed Feb 16 17:27:45 2022 -0600 Merge branch 'cassandra-3.0' into cassandra-3.11 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (eeec360 -> f0bb299)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from eeec360 Merge branch 'cassandra-3.0' into cassandra-3.11 new c586cca ninja fix typo in Jenkinsfile new f0bb299 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: ninja fix typo in Jenkinsfile
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-3.0 by this push: new c586cca ninja fix typo in Jenkinsfile c586cca is described below commit c586cca2b3cac996881671bfc771f3c7c37d62e2 Author: Brandon Williams AuthorDate: Wed Feb 16 17:27:24 2022 -0600 ninja fix typo in Jenkinsfile --- .jenkins/Jenkinsfile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.jenkins/Jenkinsfile b/.jenkins/Jenkinsfile index f213045..8df18ec 100644 --- a/.jenkins/Jenkinsfile +++ b/.jenkins/Jenkinsfile @@ -158,7 +158,7 @@ pipeline { always { warnError('missing test xml files') { script { -copyTestResults('cqlsh-test', cqlsh.getNumber()) +copyTestResults('cqlsh-tests', cqlsh.getNumber()) } } } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit d70e004975e47de750449557a714b1db799c10a1 Merge: e56bf10 b837a58 Author: Brandon Williams AuthorDate: Wed Feb 16 17:28:09 2022 -0600 Merge branch 'cassandra-4.0' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (2687cf1 -> b837a58)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 2687cf1 Merge branch 'cassandra-3.11' into cassandra-4.0 new c586cca ninja fix typo in Jenkinsfile new f0bb299 Merge branch 'cassandra-3.0' into cassandra-3.11 new b837a58 Merge branch 'cassandra-3.11' into cassandra-4.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b837a586f1df4813ea9026feaf95acdadf57b56b Merge: 2687cf1 f0bb299 Author: Brandon Williams AuthorDate: Wed Feb 16 17:27:58 2022 -0600 Merge branch 'cassandra-3.11' into cassandra-4.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org