[jira] [Commented] (CASSANDRA-19448) CommitlogArchiver only has granularity to seconds for restore_point_in_time
[ https://issues.apache.org/jira/browse/CASSANDRA-19448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824281#comment-17824281 ] Maxwell Guo commented on CASSANDRA-19448: - [~jeromatron] I don't think this is a bug after I read the relevant code and configuration files, [restore pint in time config |https://github.com/apache/cassandra/blob/trunk/conf/commitlog_archiving.properties#L40] shows that the restore_point_in_time format is " :MM:dd HH:mm:ss " , that is to say we only support secondary level restore point. But I think we can make the level to millisecond. [~brandon.williams]WDYT ? > CommitlogArchiver only has granularity to seconds for restore_point_in_time > --- > > Key: CASSANDRA-19448 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19448 > Project: Cassandra > Issue Type: Bug > Components: Local/Commit Log >Reporter: Jeremy Hanna >Assignee: Maxwell Guo >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x > > > Commitlog archiver allows users to backup commitlog files for the purpose of > doing point in time restores. The [configuration > file|https://github.com/apache/cassandra/blob/trunk/conf/commitlog_archiving.properties] > gives an example of down to the seconds granularity but then asks what > whether the timestamps are microseconds or milliseconds - defaulting to > microseconds. Because the [CommitLogArchiver uses a second based date > format|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/commitlog/CommitLogArchiver.java#L52], > if a user specifies to restore at something at a lower granularity like > milliseconds or microseconds, that means that the it will truncate everything > after the second and restore to that second. So say you specify a > restore_point_in_time like this: > restore_point_in_time=2024:01:18 17:01:01.623392 > it will silently truncate everything after the 01 seconds. So effectively to > the user, it is missing updates between 01 and 01.623392. > This appears to be a bug in the intent. We should allow users to specify > down to the millisecond or even microsecond level. If we allow them to > specify down to microseconds for the restore point in time, then it may > internally need to change from a long. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (7fdb92ecb -> 618ca889f)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 7fdb92ecb generate docs for fd550e9c new 618ca889f generate docs for fd550e9c This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (7fdb92ecb) \ N -- N -- N refs/heads/asf-staging (618ca889f) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4883646 -> 4883646 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19417) LIST SUPERUSERS cql command
[ https://issues.apache.org/jira/browse/CASSANDRA-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824274#comment-17824274 ] Maxwell Guo commented on CASSANDRA-19417: - [~blerer] would you mind take a look at this issue and the ML about the grammar's discussion ? > LIST SUPERUSERS cql command > --- > > Key: CASSANDRA-19417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19417 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Shailaja Koppu >Assignee: Shailaja Koppu >Priority: Normal > Labels: CQL > Fix For: 5.x > > Time Spent: 0.5h > Remaining Estimate: 0h > > Developing a new CQL command LIST SUPERUSERS to return list of roles with > superuser privilege. This includes roles who acquired superuser privilege in > the hierarchy. > Context: LIST ROLES cql command lists roles, their membership details and > displays super=true for immediate superusers. But there can be roles who > acquired superuser privilege due to a grant. LIST ROLES command won't display > super=true for such roles and the only way to recognize such roles is to look > for atleast one row with super=true in the output of LIST ROLES OF name> command. While this works to check is a given role has superuser > privilege, there may be services (for example, Sidecar) working with C* and > may need to maintain list of roles with superuser privilege. There is no > existing command/tool to retrieve such roles details. Hence developing this > command which returns all roles having superuser privilege. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (0d849e285 -> 7fdb92ecb)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 0d849e285 generate docs for fd550e9c new 7fdb92ecb generate docs for fd550e9c This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (0d849e285) \ N -- N -- N refs/heads/asf-staging (7fdb92ecb) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4883646 -> 4883646 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19428) Clean up KeyRangeIterator classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19428: - Summary: Clean up KeyRangeIterator classes (was: Clean up KeyRangeIteraror classes) > Clean up KeyRangeIterator classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.x > > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19428) Clean up KeyRangeIteraror classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19428: - Summary: Clean up KeyRangeIteraror classes (was: Clean up KeyRangeItearor classes) > Clean up KeyRangeIteraror classes > - > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.x > > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19428) Clean up KeyRangeItearor classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824215#comment-17824215 ] Ekaterina Dimitrova edited comment on CASSANDRA-19428 at 3/6/24 11:28 PM: -- I created some trunk [patch|https://github.com/apache/cassandra/pull/3162] based on a few works from Jonathan Ellis, Michael Marshall, and a PR in progress by [~pkolaczk](with the idea to help him make rebase easier), but now CI fails consistently for one test which times out: testFailedRangeIteratorOnMultiIndexesQuery - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4/jobs/58263/tests] The rest of the CI matches what we currently see on trunk - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4] I can reproduce the failure of testFailedRangeIteratorOnMultiIndexesQuery locally. I haven't investigated it yet, but I have a few immediate observations so far: - despite that, the test is testFailedRangeIteratorOn*Multi*IndexesQuery, with stress on Multi, I see only one index created. ([~mikea] ? You are the author of this test, so checking here if you have any comments) - The moment we disable the injection, we start timing out after that. If I swap to check first the assertions that do not throw exceptions and then the ones that will throw after the injected failure - the test succeeds. I will have to check that I am not sure whether I will be able to get back to this before next week and how much time I will have, but I wanted to show what I am doing and check whether it makes sense so far. CC [~maedhroz] was (Author: e.dimitrova): I created some trunk [patch|https://github.com/apache/cassandra/pull/3162] based on a few works from Jonathan Ellis, Michael Marchall, and a PR in progress by [~pkolaczk](with the idea to help him make rebase easier), but now CI fails consistently for one test which times out: testFailedRangeIteratorOnMultiIndexesQuery - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4/jobs/58263/tests] The rest of the CI matches what we currently see on trunk - https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4 I can reproduce the failure of testFailedRangeIteratorOnMultiIndexesQuery locally. I haven't investigated it yet, but I have a few immediate observations so far: - despite that, the test is testFailedRangeIteratorOn*Multi*IndexesQuery, with stress on Multi, I see only one index created. ([~mikea] ? You are the author of this test, so checking here if you have any comments) - The moment we disable the injection, we start timing out after that. If I swap to check first the assertions that do not throw exceptions and then the ones that will throw after the injected failure - the test succeeds. I will have to check that I am not sure whether I will be able to get back to this before next week and how much time I will have, but I wanted to show what I am doing and check whether it makes sense so far. CC [~maedhroz] > Clean up KeyRangeItearor classes > > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.x > > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19428) Clean up KeyRangeItearor classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824215#comment-17824215 ] Ekaterina Dimitrova edited comment on CASSANDRA-19428 at 3/6/24 11:27 PM: -- I created some trunk [patch|https://github.com/apache/cassandra/pull/3162] based on a few works from Jonathan Ellis, Michael Marchall, and a PR in progress by [~pkolaczk](with the idea to help him make rebase easier), but now CI fails consistently for one test which times out: testFailedRangeIteratorOnMultiIndexesQuery - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4/jobs/58263/tests] The rest of the CI matches what we currently see on trunk - https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4 I can reproduce the failure of testFailedRangeIteratorOnMultiIndexesQuery locally. I haven't investigated it yet, but I have a few immediate observations so far: - despite that, the test is testFailedRangeIteratorOn*Multi*IndexesQuery, with stress on Multi, I see only one index created. ([~mikea] ? You are the author of this test, so checking here if you have any comments) - The moment we disable the injection, we start timing out after that. If I swap to check first the assertions that do not throw exceptions and then the ones that will throw after the injected failure - the test succeeds. I will have to check that I am not sure whether I will be able to get back to this before next week and how much time I will have, but I wanted to show what I am doing and check whether it makes sense so far. CC [~maedhroz] was (Author: e.dimitrova): I created some trunk [patch|https://github.com/apache/cassandra/pull/3162] based on a few works from Jonathan Ellis, Michael Marchall, and a PR in progress by [~pkolaczk](with the idea to help him make rebase easier), but now CI fails consistently for one test which times out: testFailedRangeIteratorOnMultiIndexesQuery - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4/jobs/58263/tests] I can reproduce it locally. I haven't investigated it yet, but I have a few immediate observations so far: - despite that, the test is testFailedRangeIteratorOn*Multi*IndexesQuery, with stress on Multi, I see only one index created. ([~mikea] ? You are the author of this test, so checking here if you have any comments) - The moment we disable the injection, we start timing out after that. If I swap to check first the assertions that do not throw exceptions and then the ones that will throw after the injected failure - the test succeeds. I will have to check that I am not sure whether I will be able to get back to this before next week and how much time I will have, but I wanted to show what I am doing and check whether it makes sense so far. CC [~maedhroz] > Clean up KeyRangeItearor classes > > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.x > > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19428) Clean up KeyRangeItearor classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824215#comment-17824215 ] Ekaterina Dimitrova edited comment on CASSANDRA-19428 at 3/6/24 11:26 PM: -- I created some trunk [patch|https://github.com/apache/cassandra/pull/3162] based on a few works from Jonathan Ellis, Michael Marchall, and a PR in progress by [~pkolaczk](with the idea to help him make rebase easier), but now CI fails consistently for one test which times out: testFailedRangeIteratorOnMultiIndexesQuery - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4/jobs/58263/tests] I can reproduce it locally. I haven't investigated it yet, but I have a few immediate observations so far: - despite that, the test is testFailedRangeIteratorOn*Multi*IndexesQuery, with stress on Multi, I see only one index created. ([~mikea] ? You are the author of this test, so checking here if you have any comments) - The moment we disable the injection, we start timing out after that. If I swap to check first the assertions that do not throw exceptions and then the ones that will throw after the injected failure - the test succeeds. I will have to check that I am not sure whether I will be able to get back to this before next week and how much time I will have, but I wanted to show what I am doing and check whether it makes sense so far. CC [~maedhroz] was (Author: e.dimitrova): I created some trunk [patch|https://github.com/apache/cassandra/pull/3162] based on a few works from Jonathan Ellis, Michael Marchall, and a PR in progress by [~pkolaczk](with the idea to help him make rebase easier), but now CI fails consistently for one test which times out: testFailedRangeIteratorOnMultiIndexesQuery - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4/jobs/58263/tests] I can reproduce it locally. I haven't investigated it yet, but I have a few immediate observations so far: - despite that, the test is testFailedRangeIteratorOn*Multi*IndexesQuery, with stress on Multi, I see only one index created. ([~mikea] ? You are the author of this test, so checking here if you have any comments) - The moment we disable the injection, we start timing out after that. If I swap to check first the assertions that do not throw exceptions and then the ones that will throw after the injected failure - the test succeeds. I will have to check that I am not sure whether I will be able to get back to this before next week and how much time I will have, but I wanted to show what I am doing and check whether it makes sense so far. > Clean up KeyRangeItearor classes > > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.x > > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19428) Clean up KeyRangeItearor classes
[ https://issues.apache.org/jira/browse/CASSANDRA-19428?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824215#comment-17824215 ] Ekaterina Dimitrova commented on CASSANDRA-19428: - I created some trunk [patch|https://github.com/apache/cassandra/pull/3162] based on a few works from Jonathan Ellis, Michael Marchall, and a PR in progress by [~pkolaczk](with the idea to help him make rebase easier), but now CI fails consistently for one test which times out: testFailedRangeIteratorOnMultiIndexesQuery - [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2666/workflows/fdd384e4-931b-408c-8496-dc22e4757ec4/jobs/58263/tests] I can reproduce it locally. I haven't investigated it yet, but I have a few immediate observations so far: - despite that, the test is testFailedRangeIteratorOn*Multi*IndexesQuery, with stress on Multi, I see only one index created. ([~mikea] ? You are the author of this test, so checking here if you have any comments) - The moment we disable the injection, we start timing out after that. If I swap to check first the assertions that do not throw exceptions and then the ones that will throw after the injected failure - the test succeeds. I will have to check that I am not sure whether I will be able to get back to this before next week and how much time I will have, but I wanted to show what I am doing and check whether it makes sense so far. > Clean up KeyRangeItearor classes > > > Key: CASSANDRA-19428 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19428 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Low > Fix For: 5.x > > > Remove KeyRangeIterator.current and simplify -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19424) Add certificate expiry check to start up validations done in Cassandra Analytics library
[ https://issues.apache.org/jira/browse/CASSANDRA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-19424: -- Fix Version/s: NA Source Control Link: https://github.com/apache/cassandra-analytics/commit/6ce33604bbd9acbee092ab3c4f7f11c0d434f730 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Add certificate expiry check to start up validations done in Cassandra > Analytics library > > > Key: CASSANDRA-19424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19424 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Fix For: NA > > Time Spent: 50m > Remaining Estimate: 0h > > Want to add certificate expiry check to startup Keystore validations done in > Cassandra Analytics library. This is to fail fast if user certificates are > expired. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-analytics) branch trunk updated: CASSANDRA-19424 Check for expired certificate during start up validation (#43)
This is an automated email from the ASF dual-hosted git repository. ycai pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-analytics.git The following commit(s) were added to refs/heads/trunk by this push: new 6ce3360 CASSANDRA-19424 Check for expired certificate during start up validation (#43) 6ce3360 is described below commit 6ce33604bbd9acbee092ab3c4f7f11c0d434f730 Author: Saranya Krishnakumar AuthorDate: Wed Mar 6 14:32:22 2024 -0800 CASSANDRA-19424 Check for expired certificate during start up validation (#43) patch by Saranya Krishnakumar; reviewed by Francisco Guerrero, Yifan Cai for CASSANDRA-19424 --- CHANGES.txt | 1 + .../spark/validation/KeyStoreValidation.java | 18 ++ .../spark/validation/KeyStoreValidationTests.java | 12 .../test/resources/validation/keystore-expired.p12| Bin 0 -> 2421 bytes 4 files changed, 31 insertions(+) diff --git a/CHANGES.txt b/CHANGES.txt index 92620a9..6004ee3 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 1.0.0 + * Add certificate expiry check to start up validations done in Cassandra Analytics library (CASSANDRA-19424) * Use constant reference time during bulk read process (CASSANDRA-19452) * Update access of ClearSnapshotStrategy (CASSANDRA-19442) * Bulk reader fails to produce a row when regular column values are null (CASSANDRA-19411) diff --git a/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/validation/KeyStoreValidation.java b/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/validation/KeyStoreValidation.java index febb0c8..6926eb8 100644 --- a/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/validation/KeyStoreValidation.java +++ b/cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/validation/KeyStoreValidation.java @@ -25,6 +25,9 @@ import java.security.GeneralSecurityException; import java.security.Key; import java.security.KeyStore; import java.security.PrivateKey; +import java.security.cert.Certificate; +import java.security.cert.CertificateExpiredException; +import java.security.cert.X509Certificate; import java.util.Enumeration; import java.util.function.Supplier; @@ -62,6 +65,7 @@ public class KeyStoreValidation implements StartupValidation @Override public void validate() { +String latestAlias = null; try { if (!configured) @@ -81,6 +85,16 @@ public class KeyStoreValidation implements StartupValidation throw new RuntimeException("KeyStore is empty"); } +for (Enumeration aliases = keyStore.aliases(); aliases.hasMoreElements();) +{ +latestAlias = aliases.nextElement(); +Certificate cert = keyStore.getCertificate(latestAlias); +if (cert instanceof X509Certificate) +{ +((X509Certificate) cert).checkValidity(); +} +} + for (Enumeration aliases = keyStore.aliases(); aliases.hasMoreElements();) { Key key = keyStore.getKey(aliases.nextElement(), password); @@ -91,6 +105,10 @@ public class KeyStoreValidation implements StartupValidation } throw new RuntimeException("KeyStore contains no private keys"); } +catch (CertificateExpiredException exception) +{ +throw new RuntimeException(String.format("Certificate with alias '%s' is expired.", latestAlias), exception); +} catch (IOException | GeneralSecurityException exception) { throw new RuntimeException("KeyStore is misconfigured", exception); diff --git a/cassandra-analytics-core/src/test/java/org/apache/cassandra/spark/validation/KeyStoreValidationTests.java b/cassandra-analytics-core/src/test/java/org/apache/cassandra/spark/validation/KeyStoreValidationTests.java index 75cf826..f6acb39 100644 --- a/cassandra-analytics-core/src/test/java/org/apache/cassandra/spark/validation/KeyStoreValidationTests.java +++ b/cassandra-analytics-core/src/test/java/org/apache/cassandra/spark/validation/KeyStoreValidationTests.java @@ -24,6 +24,7 @@ import org.junit.jupiter.api.Test; import org.apache.cassandra.secrets.SecretsProvider; import org.apache.cassandra.secrets.TestSecretsProvider; +import static org.assertj.core.api.AssertionsForClassTypes.assertThat; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertInstanceOf; import static org.junit.jupiter.api.Assertions.assertNull; @@ -97,4 +98,15 @@ public class KeyStoreValidationTests Throwable throwable = validation.perform(); assertNull(throwable); } + +@Test +public void testExpiredKeyStore() +{ +SecretsProvider secrets = TestSecret
Re: [PR] CASSANDRA-19424 Check for expired certificate during start up validation [cassandra-analytics]
yifan-c merged PR #43: URL: https://github.com/apache/cassandra-analytics/pull/43 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19424) Add certificate expiry check to start up validations done in Cassandra Analytics library
[ https://issues.apache.org/jira/browse/CASSANDRA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824200#comment-17824200 ] Yifan Cai commented on CASSANDRA-19424: --- +1 > Add certificate expiry check to start up validations done in Cassandra > Analytics library > > > Key: CASSANDRA-19424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19424 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > Want to add certificate expiry check to startup Keystore validations done in > Cassandra Analytics library. This is to fail fast if user certificates are > expired. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19424) Add certificate expiry check to start up validations done in Cassandra Analytics library
[ https://issues.apache.org/jira/browse/CASSANDRA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-19424: -- Status: Ready to Commit (was: Review In Progress) > Add certificate expiry check to start up validations done in Cassandra > Analytics library > > > Key: CASSANDRA-19424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19424 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > Want to add certificate expiry check to startup Keystore validations done in > Cassandra Analytics library. This is to fail fast if user certificates are > expired. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19457) Memory Leak of `DefaultSession`
[ https://issues.apache.org/jira/browse/CASSANDRA-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824198#comment-17824198 ] Jane He commented on CASSANDRA-19457: - I reproduced this. My program keeps opening and closing connections, when there are 901 connections created, there are 901 `DefaultSession` in the heap. However, my investigation shows that the leak comes from `NodeStateEvent` and `DistanceEvent`. You have to turn on the `MicrometerMetricsFactory` to reproduce this. {code:java} datastax-java-driver.advanced.metrics { session.enabled = [ connected-nodes, cql-requests ] node.enabled = [ pool.open-connections, pool.in-flight ] factory.class = MicrometerMetricsFactory }{code} > Memory Leak of `DefaultSession` > --- > > Key: CASSANDRA-19457 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19457 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Jane He >Assignee: Jane He >Priority: Normal > Attachments: Screenshot 2024-03-06 at 2.07.01 PM.png, Screenshot > 2024-03-06 at 2.07.13 PM.png > > > There is a memory leak of previous closed {{{}DefaultSession{}}}s. It can be > reproduced by this: > {code:java} > public static void main(String[] args) throws InterruptedException { > Semaphore sema = new Semaphore(20); > for (int i = 0; i < 1; i++) { > new Thread(() -> { > try { > sema.acquire(); > try(CqlSession session = CqlSession.builder() > > .withCloudSecureConnectBundle(Paths.get("bundle.zip")) > .withAuthCredentials("token", "") > .build()) { > // Do stuff > } > } catch (Exception e) { > System.out.println(e); > } finally { > sema.release(); > } > }).start(); > } > }{code} > On initial investigation, it seems like > {{MicrometerMetricUpdater.initializeGauge()}} uses > {{{}Gauge.{}}}{{{}_builder()_{}}} _using_ {{_Supplier_}} _._ This creates a > strong reference that is causing the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19457) Memory Leak of `DefaultSession`
[ https://issues.apache.org/jira/browse/CASSANDRA-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jane He updated CASSANDRA-19457: Attachment: Screenshot 2024-03-06 at 2.07.13 PM.png > Memory Leak of `DefaultSession` > --- > > Key: CASSANDRA-19457 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19457 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Jane He >Assignee: Jane He >Priority: Normal > Attachments: Screenshot 2024-03-06 at 2.07.01 PM.png, Screenshot > 2024-03-06 at 2.07.13 PM.png > > > There is a memory leak of previous closed {{{}DefaultSession{}}}s. It can be > reproduced by this: > {code:java} > public static void main(String[] args) throws InterruptedException { > Semaphore sema = new Semaphore(20); > for (int i = 0; i < 1; i++) { > new Thread(() -> { > try { > sema.acquire(); > try(CqlSession session = CqlSession.builder() > > .withCloudSecureConnectBundle(Paths.get("bundle.zip")) > .withAuthCredentials("token", "") > .build()) { > // Do stuff > } > } catch (Exception e) { > System.out.println(e); > } finally { > sema.release(); > } > }).start(); > } > }{code} > On initial investigation, it seems like > {{MicrometerMetricUpdater.initializeGauge()}} uses > {{{}Gauge.{}}}{{{}_builder()_{}}} _using_ {{_Supplier_}} _._ This creates a > strong reference that is causing the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19457) Memory Leak of `DefaultSession`
[ https://issues.apache.org/jira/browse/CASSANDRA-19457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jane He updated CASSANDRA-19457: Attachment: Screenshot 2024-03-06 at 2.07.01 PM.png > Memory Leak of `DefaultSession` > --- > > Key: CASSANDRA-19457 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19457 > Project: Cassandra > Issue Type: Bug > Components: Client/java-driver >Reporter: Jane He >Assignee: Jane He >Priority: Normal > Attachments: Screenshot 2024-03-06 at 2.07.01 PM.png, Screenshot > 2024-03-06 at 2.07.13 PM.png > > > There is a memory leak of previous closed {{{}DefaultSession{}}}s. It can be > reproduced by this: > {code:java} > public static void main(String[] args) throws InterruptedException { > Semaphore sema = new Semaphore(20); > for (int i = 0; i < 1; i++) { > new Thread(() -> { > try { > sema.acquire(); > try(CqlSession session = CqlSession.builder() > > .withCloudSecureConnectBundle(Paths.get("bundle.zip")) > .withAuthCredentials("token", "") > .build()) { > // Do stuff > } > } catch (Exception e) { > System.out.println(e); > } finally { > sema.release(); > } > }).start(); > } > }{code} > On initial investigation, it seems like > {{MicrometerMetricUpdater.initializeGauge()}} uses > {{{}Gauge.{}}}{{{}_builder()_{}}} _using_ {{_Supplier_}} _._ This creates a > strong reference that is causing the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-112) ClosedChannelException when downloading from S3
[ https://issues.apache.org/jira/browse/CASSANDRASC-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRASC-112: -- Status: Ready to Commit (was: Review In Progress) > ClosedChannelException when downloading from S3 > --- > > Key: CASSANDRASC-112 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-112 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > {code:java} > org.apache.cassandra.sidecar.exceptions.RestoreJobFatalException: > Unrecoverable error when downloading object. > Caused by: java.nio.channels.ClosedChannelException > at > java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) > at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:266) > at > org.apache.cassandra.sidecar.restore.StorageClient.lambda$subscribeRateLimitedWrite$6(StorageClient.java:271) > ... 22 more > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-112) ClosedChannelException when downloading from S3
[ https://issues.apache.org/jira/browse/CASSANDRASC-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRASC-112: -- Fix Version/s: 1.0 Source Control Link: https://github.com/apache/cassandra-sidecar/commit/d19a1b9a112614d378b42bb136cc84ae9cc6213a Resolution: Fixed Status: Resolved (was: Ready to Commit) > ClosedChannelException when downloading from S3 > --- > > Key: CASSANDRASC-112 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-112 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > Fix For: 1.0 > > > {code:java} > org.apache.cassandra.sidecar.exceptions.RestoreJobFatalException: > Unrecoverable error when downloading object. > Caused by: java.nio.channels.ClosedChannelException > at > java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) > at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:266) > at > org.apache.cassandra.sidecar.restore.StorageClient.lambda$subscribeRateLimitedWrite$6(StorageClient.java:271) > ... 22 more > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-112) ClosedChannelException when downloading from S3
[ https://issues.apache.org/jira/browse/CASSANDRASC-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824190#comment-17824190 ] ASF subversion and git services commented on CASSANDRASC-112: - Commit d19a1b9a112614d378b42bb136cc84ae9cc6213a in cassandra-sidecar's branch refs/heads/trunk from Yifan Cai [ https://gitbox.apache.org/repos/asf?p=cassandra-sidecar.git;h=d19a1b9 ] CASSANDRASC-112 Fix ClosedChannelException when downloading from S3 (#103) patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRASC-112 > ClosedChannelException when downloading from S3 > --- > > Key: CASSANDRASC-112 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-112 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > {code:java} > org.apache.cassandra.sidecar.exceptions.RestoreJobFatalException: > Unrecoverable error when downloading object. > Caused by: java.nio.channels.ClosedChannelException > at > java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) > at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:266) > at > org.apache.cassandra.sidecar.restore.StorageClient.lambda$subscribeRateLimitedWrite$6(StorageClient.java:271) > ... 22 more > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-sidecar) branch trunk updated: CASSANDRASC-112 Fix ClosedChannelException when downloading from S3 (#103)
This is an automated email from the ASF dual-hosted git repository. ycai pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-sidecar.git The following commit(s) were added to refs/heads/trunk by this push: new d19a1b9 CASSANDRASC-112 Fix ClosedChannelException when downloading from S3 (#103) d19a1b9 is described below commit d19a1b9a112614d378b42bb136cc84ae9cc6213a Author: Yifan Cai <52585731+yifa...@users.noreply.github.com> AuthorDate: Wed Mar 6 13:57:00 2024 -0800 CASSANDRASC-112 Fix ClosedChannelException when downloading from S3 (#103) patch by Yifan Cai; reviewed by Francisco Guerrero for CASSANDRASC-112 --- .circleci/config.yml | 2 + CHANGES.txt| 1 + build.gradle | 76 -- spotbugs-exclude.xml | 20 ++ .../config/yaml/RestoreJobConfigurationImpl.java | 4 +- .../cassandra/sidecar/restore/StorageClient.java | 4 - .../sidecar/restore/StorageClientTest.java | 277 + .../cassandra/sidecar/db/RestoreJobTest.java | 8 + .../sidecar/restore/RestoreSliceTaskTest.java | 147 ++- 9 files changed, 457 insertions(+), 82 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index f027cae..2f0acb1 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -65,6 +65,7 @@ jobs: environment: skipIntegrationTest: true steps: + - setup_remote_docker - checkout - run: ./gradlew --info check -x integrationTest --stacktrace @@ -133,6 +134,7 @@ jobs: environment: skipIntegrationTest: true steps: + - setup_remote_docker - checkout - run: ./gradlew --info check -x integrationTest --stacktrace diff --git a/CHANGES.txt b/CHANGES.txt index b46c455..351ca44 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,5 +1,6 @@ 1.0.0 - + * Fix ClosedChannelException when downloading from S3 (CASSANDRASC-112) * Fix NPE thrown when getting StorageClient from pool (CASSANDRASC-110) * Relocate Sidecar common classes in vertx-client-shaded (CASSANDRASC-104) * Automated yaml type binding for deserialization (CASSANDRASC-103) diff --git a/build.gradle b/build.gradle index 5acb5b2..5aa49d5 100644 --- a/build.gradle +++ b/build.gradle @@ -135,7 +135,7 @@ run { "-javaagent:$projectDir/agents/jolokia-jvm-1.6.0-agent.jar=port=,host=localhost"] } -// add integration test +// add additional test source set sourceSets { integrationTest { java { @@ -144,12 +144,21 @@ sourceSets { srcDir file('src/test/integration') } } + +containerTest { +java { +compileClasspath += main.output + test.output +runtimeClasspath += main.output + test.output +srcDir file('src/test/containerTest') +} +} } // and mark it as test root idea { module { testSourceDirs += sourceSets.integrationTest.java.srcDirs +testSourceDirs += sourceSets.containerTest.java.srcDirs } } @@ -157,6 +166,7 @@ configurations { jolokia integrationTestImplementation.extendsFrom testImplementation +containerTestImplementation.extendsFrom testImplementation runtime.exclude(group: "com.google.code.findbugs", module: "jsr305") runtime.exclude(group: "org.codehaus.mojo", module: "animal-sniffer-annotations") @@ -238,6 +248,7 @@ dependencies { // Needed for snapshot manifest validation integrationTestImplementation(group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-jsr310', version: "${project.jacksonVersion}") + containerTestImplementation('com.adobe.testing:s3mock-testcontainers:2.17.0') // 3.x version do not support java 11 } jar { @@ -313,24 +324,25 @@ test { } } +def JDK11_OPTIONS = ['-Djdk.attach.allowAttachSelf=true', + '--add-exports', 'java.base/jdk.internal.misc=ALL-UNNAMED', + '--add-exports', 'java.base/jdk.internal.ref=ALL-UNNAMED', + '--add-exports', 'java.base/sun.nio.ch=ALL-UNNAMED', + '--add-exports', 'java.management.rmi/com.sun.jmx.remote.internal.rmi=ALL-UNNAMED', + '--add-exports', 'java.rmi/sun.rmi.registry=ALL-UNNAMED', + '--add-exports', 'java.rmi/sun.rmi.server=ALL-UNNAMED', + '--add-exports', 'java.sql/java.sql=ALL-UNNAMED', + '--add-opens', 'java.base/java.lang.module=ALL-UNNAMED', + '--add-opens', 'java.base/jdk.internal.loader=ALL-UNNAMED', + '--add-opens', 'java.base/jdk.internal.ref=ALL-UNNAMED', + '--add-opens', 'java.base/jdk.internal.reflect=ALL-UNNAMED', + '--add-opens', 'java.base/jdk.internal.math=ALL-UNNAMED', +
[jira] [Commented] (CASSANDRA-19424) Add certificate expiry check to start up validations done in Cassandra Analytics library
[ https://issues.apache.org/jira/browse/CASSANDRA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824182#comment-17824182 ] Francisco Guerrero commented on CASSANDRA-19424: +1 Thanks for the patch > Add certificate expiry check to start up validations done in Cassandra > Analytics library > > > Key: CASSANDRA-19424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19424 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > Want to add certificate expiry check to startup Keystore validations done in > Cassandra Analytics library. This is to fail fast if user certificates are > expired. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19424) Add certificate expiry check to start up validations done in Cassandra Analytics library
[ https://issues.apache.org/jira/browse/CASSANDRA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRA-19424: --- Status: Review In Progress (was: Patch Available) > Add certificate expiry check to start up validations done in Cassandra > Analytics library > > > Key: CASSANDRA-19424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19424 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > Want to add certificate expiry check to startup Keystore validations done in > Cassandra Analytics library. This is to fail fast if user certificates are > expired. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19424) Add certificate expiry check to start up validations done in Cassandra Analytics library
[ https://issues.apache.org/jira/browse/CASSANDRA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saranya Krishnakumar updated CASSANDRA-19424: - Test and Documentation Plan: unit tests Status: Patch Available (was: In Progress) > Add certificate expiry check to start up validations done in Cassandra > Analytics library > > > Key: CASSANDRA-19424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19424 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > Want to add certificate expiry check to startup Keystore validations done in > Cassandra Analytics library. This is to fail fast if user certificates are > expired. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19424 Check for expired certificate during start up validation [cassandra-analytics]
frankgh commented on code in PR #43: URL: https://github.com/apache/cassandra-analytics/pull/43#discussion_r1515113234 ## cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/validation/KeyStoreValidation.java: ## @@ -81,6 +84,15 @@ public void validate() throw new RuntimeException("KeyStore is empty"); } +for (Enumeration aliases = keyStore.aliases(); aliases.hasMoreElements();) +{ +Certificate cert = keyStore.getCertificate(aliases.nextElement()); +if (cert instanceof X509Certificate) +{ +((X509Certificate) cert).checkValidity(); +} +} Review Comment: ah, I just realized that the second loop looks for a PK< and it if finds it, it exits the validation as successful. Please disregard this comment. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19424 Check for expired certificate during start up validation [cassandra-analytics]
frankgh commented on code in PR #43: URL: https://github.com/apache/cassandra-analytics/pull/43#discussion_r1515102552 ## cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/validation/KeyStoreValidation.java: ## @@ -81,6 +84,15 @@ public void validate() throw new RuntimeException("KeyStore is empty"); } +for (Enumeration aliases = keyStore.aliases(); aliases.hasMoreElements();) +{ +Certificate cert = keyStore.getCertificate(aliases.nextElement()); +if (cert instanceof X509Certificate) +{ +((X509Certificate) cert).checkValidity(); +} +} Review Comment: Can we collapse the for lops in lines 87 and 96: ```suggestion for (Enumeration aliases = keyStore.aliases(); aliases.hasMoreElements();) { String alias = aliases.nextElement(); Certificate cert = keyStore.getCertificate(alias); if (cert instanceof X509Certificate) { ((X509Certificate) cert).checkValidity(); } Key key = keyStore.getKey(alias, password); if (key != null && key instanceof PrivateKey) { return; // KeyStore contains a private key } } ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19454) Revert switch to approximate time in Dispatcher to avoid mixing with nanoTime() in downstream timeout calculations
[ https://issues.apache.org/jira/browse/CASSANDRA-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824134#comment-17824134 ] Arun Ganesh edited comment on CASSANDRA-19454 at 3/6/24 8:06 PM: - Done. I had to fix the commit message on the trunk PR and add a newline for one of the imports. I force pushed. Hope that is fine. was (Author: JIRAUSER303038): Done. I had to fix the commit message on the trunk PR and add a newline on one of the imports. I force pushed. Hope that is fine. > Revert switch to approximate time in Dispatcher to avoid mixing with > nanoTime() in downstream timeout calculations > -- > > Key: CASSANDRA-19454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19454 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Caleb Rackliffe >Assignee: Arun Ganesh >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: ci_summary.html > > Time Spent: 20m > Remaining Estimate: 0h > > CASSANDRA-15241 changed {{Dispatcher}} to use the {{approxTime}} > implementation of {{MonotonicClock}} rather than {{nanoTime()}}, but clock > drift between the two, can potentially cause queries to time out more > quickly. We should be able to revert the {{Dispatcher}} to use {{nanoTime()}} > again and similarly change {{QueriesTable} to {{nanoTime()}} as well for > consistency. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19424) Add certificate expiry check to start up validations done in Cassandra Analytics library
[ https://issues.apache.org/jira/browse/CASSANDRA-19424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824160#comment-17824160 ] Saranya Krishnakumar commented on CASSANDRA-19424: -- Green CI after addressing comments [https://app.circleci.com/pipelines/github/sarankk/cassandra-analytics/53] > Add certificate expiry check to start up validations done in Cassandra > Analytics library > > > Key: CASSANDRA-19424 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19424 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Saranya Krishnakumar >Assignee: Saranya Krishnakumar >Priority: Normal > Time Spent: 20m > Remaining Estimate: 0h > > Want to add certificate expiry check to startup Keystore validations done in > Cassandra Analytics library. This is to fail fast if user certificates are > expired. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19424 Check for expired certificate during start up validation [cassandra-analytics]
sarankk commented on code in PR #43: URL: https://github.com/apache/cassandra-analytics/pull/43#discussion_r1515057459 ## cassandra-analytics-core/src/test/java/org/apache/cassandra/spark/validation/KeyStoreValidationTests.java: ## @@ -97,4 +98,15 @@ public void testValidKeyStore() Throwable throwable = validation.perform(); assertNull(throwable); } + +@Test +public void testExpiredKeyStore() +{ +SecretsProvider secrets = TestSecretsProvider.forKeyStore("PKCS12", "keystore-expired.p12", "qwerty"); +KeyStoreValidation validation = new KeyStoreValidation(secrets); + +Throwable throwable = validation.perform(); +assertInstanceOf(RuntimeException.class, throwable); +assertThat(throwable.getMessage()).startsWith("Certificate expired, valid NotAfter: "); Review Comment: Updated to `Certificate expired. ` instead -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18762) Repair triggers OOM with direct buffer memory
[ https://issues.apache.org/jira/browse/CASSANDRA-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824144#comment-17824144 ] Brad Schoening commented on CASSANDRA-18762: It seems like if allocate fails in deserializeOffHeap due to lack of off heap memory, it could fall back to deserializeOnHeap {{ private static ByteBuffer allocate(int innerNodeCount, IPartitioner partitioner)}} {{ {}} {{ int size = offHeapBufferSize(innerNodeCount, partitioner);}} {{ logger.debug("Allocating direct buffer of size {} for an off-heap merkle tree", size);}} {{ ByteBuffer buffer = ByteBuffer.allocateDirect(size);}} {{ if (Ref.DEBUG_ENABLED)}} {{ MemoryUtil.setAttachment(buffer, new Ref.DirectBufferRef<>(null, null));}} {{ return buffer;}} {{ }}}{{ }} {{ private static Node deserializeTree(DataInputPlus in, IPartitioner partitioner, int innerNodeCount, boolean offHeapRequested, int version) throws IOException}} {{ {}} {{ return shouldUseOffHeapTrees(partitioner, offHeapRequested)}} {{ ? deserializeOffHeap(in, partitioner, innerNodeCount, version)}} {{ : OnHeapNode.deserialize(in, partitioner, version);}} {{ }}} > Repair triggers OOM with direct buffer memory > - > > Key: CASSANDRA-18762 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18762 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Brad Schoening >Priority: Normal > Labels: OutOfMemoryError > Attachments: Cluster-dm-metrics-1.PNG, > image-2023-12-06-15-28-05-459.png, image-2023-12-06-15-29-31-491.png, > image-2023-12-06-15-58-55-007.png > > > We are seeing repeated failures of nodes with 16GB of heap on a VM with 32GB > of physical RAM due to direct memory. This seems to be related to > CASSANDRA-15202 which moved Merkel trees off-heap in 4.0. Using Cassandra > 4.0.6 with Java 11. > {noformat} > 2023-08-09 04:30:57,470 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e55a3b0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_a from > /169.102.200.241:7000 > 2023-08-09 04:30:57,567 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e0d2900-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from > /169.93.192.29:7000 > 2023-08-09 04:30:57,568 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e1dcad0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_c from > /169.104.171.134:7000 > 2023-08-09 04:30:57,591 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e69a0e0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from > /169.79.232.67:7000 > 2023-08-09 04:30:57,876 [INFO ] [Service Thread] cluster_id=101 > ip_address=169.0.0.1 GCInspector.java:294 - G1 Old Generation GC in 282ms. > Compressed Class Space: 8444560 -> 8372152; G1 Eden Space: 7809794048 -> 0; > G1 Old Gen: 1453478400 -> 820942800; G1 Survivor Space: 419430400 -> 0; > Metaspace: 80411136 -> 80176528 > 2023-08-09 04:30:58,387 [ERROR] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 JVMStabilityInspector.java:102 - OutOfMemory error > letting the JVM handle the error: > java.lang.OutOfMemoryError: Direct buffer memory > at java.base/java.nio.Bits.reserveMemory(Bits.java:175) > at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:118) > at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:318) > at org.apache.cassandra.utils.MerkleTree.allocate(MerkleTree.java:742) > at > org.apache.cassandra.utils.MerkleTree.deserializeOffHeap(MerkleTree.java:780) > at org.apache.cassandra.utils.MerkleTree.deserializeTree(MerkleTree.java:751) > at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:720) > at org.apache.cassandra.utils.MerkleTree.deserialize(MerkleTree.java:698) > at > org.apache.cassandra.utils.MerkleTrees$MerkleTreesSerializer.deserialize(MerkleTrees.java:416) > at > org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:100) > at > org.apache.cassandra.repair.messages.ValidationResponse$1.deserialize(ValidationResponse.java:84) > at > org.apache.cassandra.net.Message$Serializer.deserializePost40(Message.java:782) > at org.apache.cassandra.net.Message$Serializer.deserialize(Message.java:642) > at > org.apache.cassandra.net.InboundMessageHandler$LargeMessage.deserialize(InboundMessageHandler.java:364) > at > org.apache.cassandra.net.InboundMessageHandler$LargeMessage.access$1100(InboundMessageHandler.java:317) > at > org.apache.cassandra.net.Inboun
[jira] [Comment Edited] (CASSANDRA-18762) Repair triggers OOM with direct buffer memory
[ https://issues.apache.org/jira/browse/CASSANDRA-18762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824144#comment-17824144 ] Brad Schoening edited comment on CASSANDRA-18762 at 3/6/24 7:14 PM: It seems like if allocate fails in deserializeOffHeap due to lack of off heap memory, it could fall back to deserializeOnHeap {{ private static ByteBuffer allocate(int innerNodeCount, IPartitioner partitioner)}} {{ {}} {{ int size = offHeapBufferSize(innerNodeCount, partitioner);}} {{ logger.debug("Allocating direct buffer of size {} for an off-heap merkle tree", size);}} {{ ByteBuffer buffer = ByteBuffer.allocateDirect(size);}} {{ if (Ref.DEBUG_ENABLED)}} {{ MemoryUtil.setAttachment(buffer, new Ref.DirectBufferRef<>(null, null));}} {{ return buffer;}} } {{ private static Node deserializeTree(DataInputPlus in, IPartitioner partitioner, int innerNodeCount, boolean offHeapRequested, int version) throws IOException}} {{ {}} {{ return shouldUseOffHeapTrees(partitioner, offHeapRequested)}} {{ ? deserializeOffHeap(in, partitioner, innerNodeCount, version)}} {{ : OnHeapNode.deserialize(in, partitioner, version);}} } was (Author: bschoeni): It seems like if allocate fails in deserializeOffHeap due to lack of off heap memory, it could fall back to deserializeOnHeap {{ private static ByteBuffer allocate(int innerNodeCount, IPartitioner partitioner)}} {{ {}} {{ int size = offHeapBufferSize(innerNodeCount, partitioner);}} {{ logger.debug("Allocating direct buffer of size {} for an off-heap merkle tree", size);}} {{ ByteBuffer buffer = ByteBuffer.allocateDirect(size);}} {{ if (Ref.DEBUG_ENABLED)}} {{ MemoryUtil.setAttachment(buffer, new Ref.DirectBufferRef<>(null, null));}} {{ return buffer;}} {{ }}}{{ }} {{ private static Node deserializeTree(DataInputPlus in, IPartitioner partitioner, int innerNodeCount, boolean offHeapRequested, int version) throws IOException}} {{ {}} {{ return shouldUseOffHeapTrees(partitioner, offHeapRequested)}} {{ ? deserializeOffHeap(in, partitioner, innerNodeCount, version)}} {{ : OnHeapNode.deserialize(in, partitioner, version);}} {{ }}} > Repair triggers OOM with direct buffer memory > - > > Key: CASSANDRA-18762 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18762 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Brad Schoening >Priority: Normal > Labels: OutOfMemoryError > Attachments: Cluster-dm-metrics-1.PNG, > image-2023-12-06-15-28-05-459.png, image-2023-12-06-15-29-31-491.png, > image-2023-12-06-15-58-55-007.png > > > We are seeing repeated failures of nodes with 16GB of heap on a VM with 32GB > of physical RAM due to direct memory. This seems to be related to > CASSANDRA-15202 which moved Merkel trees off-heap in 4.0. Using Cassandra > 4.0.6 with Java 11. > {noformat} > 2023-08-09 04:30:57,470 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e55a3b0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_a from > /169.102.200.241:7000 > 2023-08-09 04:30:57,567 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e0d2900-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from > /169.93.192.29:7000 > 2023-08-09 04:30:57,568 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e1dcad0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_c from > /169.104.171.134:7000 > 2023-08-09 04:30:57,591 [INFO ] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 RepairSession.java:202 - [repair > #5e69a0e0-366d-11ee-a644-d91df26add5e] Received merkle tree for table_b from > /169.79.232.67:7000 > 2023-08-09 04:30:57,876 [INFO ] [Service Thread] cluster_id=101 > ip_address=169.0.0.1 GCInspector.java:294 - G1 Old Generation GC in 282ms. > Compressed Class Space: 8444560 -> 8372152; G1 Eden Space: 7809794048 -> 0; > G1 Old Gen: 1453478400 -> 820942800; G1 Survivor Space: 419430400 -> 0; > Metaspace: 80411136 -> 80176528 > 2023-08-09 04:30:58,387 [ERROR] [AntiEntropyStage:1] cluster_id=101 > ip_address=169.0.0.1 JVMStabilityInspector.java:102 - OutOfMemory error > letting the JVM handle the error: > java.lang.OutOfMemoryError: Direct buffer memory > at java.base/java.nio.Bits.reserveMemory(Bits.java:175) > at java.base/java.nio.DirectByteBuffer.(DirectByteBuffer.java:118) > at java.base/java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:318) > at org.apache.cassan
(cassandra-website) branch asf-staging updated (a333335a9 -> 0d849e285)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard a35a9 generate docs for fd550e9c new 0d849e285 generate docs for fd550e9c This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a35a9) \ N -- N -- N refs/heads/asf-staging (0d849e285) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4883646 -> 4883646 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19454) Revert switch to approximate time in Dispatcher to avoid mixing with nanoTime() in downstream timeout calculations
[ https://issues.apache.org/jira/browse/CASSANDRA-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun Ganesh updated CASSANDRA-19454: Test and Documentation Plan: CI passed (summary attached) Status: Patch Available (was: In Progress) > Revert switch to approximate time in Dispatcher to avoid mixing with > nanoTime() in downstream timeout calculations > -- > > Key: CASSANDRA-19454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19454 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Caleb Rackliffe >Assignee: Arun Ganesh >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: ci_summary.html > > Time Spent: 20m > Remaining Estimate: 0h > > CASSANDRA-15241 changed {{Dispatcher}} to use the {{approxTime}} > implementation of {{MonotonicClock}} rather than {{nanoTime()}}, but clock > drift between the two, can potentially cause queries to time out more > quickly. We should be able to revert the {{Dispatcher}} to use {{nanoTime()}} > again and similarly change {{QueriesTable} to {{nanoTime()}} as well for > consistency. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19454) Revert switch to approximate time in Dispatcher to avoid mixing with nanoTime() in downstream timeout calculations
[ https://issues.apache.org/jira/browse/CASSANDRA-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824134#comment-17824134 ] Arun Ganesh commented on CASSANDRA-19454: - Done. I had to fix the commit message on the trunk PR and add a newline on one of the imports. I force pushed. Hope that is fine. > Revert switch to approximate time in Dispatcher to avoid mixing with > nanoTime() in downstream timeout calculations > -- > > Key: CASSANDRA-19454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19454 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Caleb Rackliffe >Assignee: Arun Ganesh >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: ci_summary.html > > Time Spent: 20m > Remaining Estimate: 0h > > CASSANDRA-15241 changed {{Dispatcher}} to use the {{approxTime}} > implementation of {{MonotonicClock}} rather than {{nanoTime()}}, but clock > drift between the two, can potentially cause queries to time out more > quickly. We should be able to revert the {{Dispatcher}} to use {{nanoTime()}} > again and similarly change {{QueriesTable} to {{nanoTime()}} as well for > consistency. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14572) Expose all table metrics in virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824131#comment-17824131 ] Stefan Miklosovic edited comment on CASSANDRA-14572 at 3/6/24 6:50 PM: --- org.apache.cassandra.metrics.ClientRequestRowAndColumnMetricsTest org.apache.cassandra.metrics.ClientRequestMetricsTest org.apache.cassandra.metrics.CassandraMetricsRegistryTest org.apache.cassandra.metrics.CQLMetricsTest org.apache.cassandra.metrics.BatchMetricsTest org.apache.cassandra.db.virtual.CollectionVirtualTableAdapterTest [CASSANDRA-14572|https://github.com/instaclustr/cassandra/tree/CASSANDRA-14572] {noformat} java17_pre-commit_tests java17_separate_tests java11_pre-commit_tests java11_separate_tests ✓ j11_build4m 34s ✓ j11_unit_tests_repeat 43m 51s {noformat} [java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/492749f5-a21c-4de7-bb00-31a1fd7026e5] [java17_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/a7d5534c-d510-44fd-aaeb-4ff13ae6f7c8] [java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/5a649d88-d6d6-45d9-8cb3-975a20ad] [java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/89ad653d-c169-419c-9c38-3a16a6deb4bc] was (Author: smiklosovic): [CASSANDRA-14572|https://github.com/instaclustr/cassandra/tree/CASSANDRA-14572] {noformat} java17_pre-commit_tests java17_separate_tests java11_pre-commit_tests java11_separate_tests ✓ j11_build4m 34s ✓ j11_unit_tests_repeat 43m 51s {noformat} [java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/492749f5-a21c-4de7-bb00-31a1fd7026e5] [java17_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/a7d5534c-d510-44fd-aaeb-4ff13ae6f7c8] [java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/5a649d88-d6d6-45d9-8cb3-975a20ad] [java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/89ad653d-c169-419c-9c38-3a16a6deb4bc] > Expose all table metrics in virtual table > - > > Key: CASSANDRA-14572 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14572 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Observability, Observability/Metrics >Reporter: Chris Lohfink >Assignee: Maxim Muzafarov >Priority: Low > Labels: virtual-tables > Fix For: 5.x > > Attachments: flight_recording_1270017199_13.jfr, keyspayces_group > responses times.png, keyspayces_group summary.png, select keyspaces_group by > string prefix.png, select keyspaces_group compare with wo.png, select > keyspaces_group without value.png, systemv_views.metrics_dropped_message.png, > thread_pools benchmark.png > > Time Spent: 7h > Remaining Estimate: 0h > > While we want a number of virtual tables to display data in a way thats great > and intuitive like in nodetool. There is also much for being able to expose > the metrics we have for tooling via CQL instead of JMX. This is more for the > tooling and adhoc advanced users who know exactly what they are looking for. > *Schema:* > Initial idea is to expose data via {{((keyspace, table), metric)}} with a > column for each metric value. Could also use a Map or UDT instead of the > column based that can be a bit more specific to each metric type. To that end > there can be a {{metric_type}} column and then a UDT for each metric type > filled in, or a single value with more of a Map style. I am > purposing the column type though as with {{ALLOW FILTERING}} it does allow > more extensive query capabilities. > *Implementations:* > * Use reflection to grab all the metrics from TableMetrics (see: > CASSANDRA-7622 impl). This is easiest and least abrasive towards new metric > implementors... but its reflection and a kinda a bad idea. > * Add a hook in TableMetrics to register with this virtual table when > registering > * Pull from the CassandraMetrics registery (either reporter or iterate > through metrics query on read of virtual table) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-uns
[jira] [Commented] (CASSANDRA-14572) Expose all table metrics in virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824131#comment-17824131 ] Stefan Miklosovic commented on CASSANDRA-14572: --- [CASSANDRA-14572|https://github.com/instaclustr/cassandra/tree/CASSANDRA-14572] {noformat} java17_pre-commit_tests java17_separate_tests java11_pre-commit_tests java11_separate_tests ✓ j11_build4m 34s ✓ j11_unit_tests_repeat 43m 51s {noformat} [java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/492749f5-a21c-4de7-bb00-31a1fd7026e5] [java17_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/a7d5534c-d510-44fd-aaeb-4ff13ae6f7c8] [java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/5a649d88-d6d6-45d9-8cb3-975a20ad] [java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3957/workflows/89ad653d-c169-419c-9c38-3a16a6deb4bc] > Expose all table metrics in virtual table > - > > Key: CASSANDRA-14572 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14572 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Observability, Observability/Metrics >Reporter: Chris Lohfink >Assignee: Maxim Muzafarov >Priority: Low > Labels: virtual-tables > Fix For: 5.x > > Attachments: flight_recording_1270017199_13.jfr, keyspayces_group > responses times.png, keyspayces_group summary.png, select keyspaces_group by > string prefix.png, select keyspaces_group compare with wo.png, select > keyspaces_group without value.png, systemv_views.metrics_dropped_message.png, > thread_pools benchmark.png > > Time Spent: 7h > Remaining Estimate: 0h > > While we want a number of virtual tables to display data in a way thats great > and intuitive like in nodetool. There is also much for being able to expose > the metrics we have for tooling via CQL instead of JMX. This is more for the > tooling and adhoc advanced users who know exactly what they are looking for. > *Schema:* > Initial idea is to expose data via {{((keyspace, table), metric)}} with a > column for each metric value. Could also use a Map or UDT instead of the > column based that can be a bit more specific to each metric type. To that end > there can be a {{metric_type}} column and then a UDT for each metric type > filled in, or a single value with more of a Map style. I am > purposing the column type though as with {{ALLOW FILTERING}} it does allow > more extensive query capabilities. > *Implementations:* > * Use reflection to grab all the metrics from TableMetrics (see: > CASSANDRA-7622 impl). This is easiest and least abrasive towards new metric > implementors... but its reflection and a kinda a bad idea. > * Add a hook in TableMetrics to register with this virtual table when > registering > * Pull from the CassandraMetrics registery (either reporter or iterate > through metrics query on read of virtual table) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19417) LIST SUPERUSERS cql command
[ https://issues.apache.org/jira/browse/CASSANDRA-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19417: -- Test and Documentation Plan: ci Status: Patch Available (was: In Progress) > LIST SUPERUSERS cql command > --- > > Key: CASSANDRA-19417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19417 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Shailaja Koppu >Assignee: Shailaja Koppu >Priority: Normal > Labels: CQL > Fix For: 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > Developing a new CQL command LIST SUPERUSERS to return list of roles with > superuser privilege. This includes roles who acquired superuser privilege in > the hierarchy. > Context: LIST ROLES cql command lists roles, their membership details and > displays super=true for immediate superusers. But there can be roles who > acquired superuser privilege due to a grant. LIST ROLES command won't display > super=true for such roles and the only way to recognize such roles is to look > for atleast one row with super=true in the output of LIST ROLES OF name> command. While this works to check is a given role has superuser > privilege, there may be services (for example, Sidecar) working with C* and > may need to maintain list of roles with superuser privilege. There is no > existing command/tool to retrieve such roles details. Hence developing this > command which returns all roles having superuser privilege. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19417) LIST SUPERUSERS cql command
[ https://issues.apache.org/jira/browse/CASSANDRA-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19417: -- Status: Review In Progress (was: Patch Available) > LIST SUPERUSERS cql command > --- > > Key: CASSANDRA-19417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19417 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Shailaja Koppu >Assignee: Shailaja Koppu >Priority: Normal > Labels: CQL > Fix For: 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > Developing a new CQL command LIST SUPERUSERS to return list of roles with > superuser privilege. This includes roles who acquired superuser privilege in > the hierarchy. > Context: LIST ROLES cql command lists roles, their membership details and > displays super=true for immediate superusers. But there can be roles who > acquired superuser privilege due to a grant. LIST ROLES command won't display > super=true for such roles and the only way to recognize such roles is to look > for atleast one row with super=true in the output of LIST ROLES OF name> command. While this works to check is a given role has superuser > privilege, there may be services (for example, Sidecar) working with C* and > may need to maintain list of roles with superuser privilege. There is no > existing command/tool to retrieve such roles details. Hence developing this > command which returns all roles having superuser privilege. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19417) LIST SUPERUSERS cql command
[ https://issues.apache.org/jira/browse/CASSANDRA-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19417: -- Complexity: Normal Fix Version/s: 5.x Reviewers: Stefan Miklosovic Status: Open (was: Triage Needed) > LIST SUPERUSERS cql command > --- > > Key: CASSANDRA-19417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19417 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Shailaja Koppu >Assignee: Shailaja Koppu >Priority: Normal > Labels: CQL > Fix For: 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > Developing a new CQL command LIST SUPERUSERS to return list of roles with > superuser privilege. This includes roles who acquired superuser privilege in > the hierarchy. > Context: LIST ROLES cql command lists roles, their membership details and > displays super=true for immediate superusers. But there can be roles who > acquired superuser privilege due to a grant. LIST ROLES command won't display > super=true for such roles and the only way to recognize such roles is to look > for atleast one row with super=true in the output of LIST ROLES OF name> command. While this works to check is a given role has superuser > privilege, there may be services (for example, Sidecar) working with C* and > may need to maintain list of roles with superuser privilege. There is no > existing command/tool to retrieve such roles details. Hence developing this > command which returns all roles having superuser privilege. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-112) ClosedChannelException when downloading from S3
[ https://issues.apache.org/jira/browse/CASSANDRASC-112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824129#comment-17824129 ] Francisco Guerrero commented on CASSANDRASC-112: +1 conditionally on green CI > ClosedChannelException when downloading from S3 > --- > > Key: CASSANDRASC-112 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-112 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > {code:java} > org.apache.cassandra.sidecar.exceptions.RestoreJobFatalException: > Unrecoverable error when downloading object. > Caused by: java.nio.channels.ClosedChannelException > at > java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) > at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:266) > at > org.apache.cassandra.sidecar.restore.StorageClient.lambda$subscribeRateLimitedWrite$6(StorageClient.java:271) > ... 22 more > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18879) Modernize CQLSH datetime conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824128#comment-17824128 ] Arun Ganesh commented on CASSANDRA-18879: - Thanks, everyone! My first PR :) > Modernize CQLSH datetime conversions > > > Key: CASSANDRA-18879 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18879 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Arun Ganesh >Priority: Low > Fix For: 5.1-alpha1 > > Attachments: cassandra-cqlsh-stdout > > Time Spent: 2h > Remaining Estimate: 0h > > Python 3.x introduced many updates to datetime conversion which allows > simplified conversions. > 1. For example, tracing.py defines a function datetime_from_utc_to_local() > but datetime now has a native function astimezone() which will convert UTC to > local time. > Review the following users of datetime which apply conversions: > * cqlshmain.py > * formatting.py > * tracing.py > Example: > {code:java} > >>> from dateutil import tz > >>> import datetime > >>> a = datetime.datetime.now().astimezone(tz.tzutc()) > >>> a > datetime.datetime(2023, 9, 25, 11, 22, 36, 251705, tzinfo=tzutc()) > >>> b = a.astimezone() > >>> b > datetime.datetime(2023, 9, 25, 14, 22, 36, 251705, > tzinfo=datetime.timezone(datetime.timedelta(seconds=10800), 'EST')) {code} > See [[PEP 495|http://example.com]]] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-112) ClosedChannelException when downloading from S3
[ https://issues.apache.org/jira/browse/CASSANDRASC-112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Francisco Guerrero updated CASSANDRASC-112: --- Reviewers: Francisco Guerrero, Francisco Guerrero Francisco Guerrero, Francisco Guerrero (was: Francisco Guerrero) Status: Review In Progress (was: Patch Available) > ClosedChannelException when downloading from S3 > --- > > Key: CASSANDRASC-112 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-112 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Labels: pull-request-available > > {code:java} > org.apache.cassandra.sidecar.exceptions.RestoreJobFatalException: > Unrecoverable error when downloading object. > Caused by: java.nio.channels.ClosedChannelException > at > java.base/sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:150) > at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:266) > at > org.apache.cassandra.sidecar.restore.StorageClient.lambda$subscribeRateLimitedWrite$6(StorageClient.java:271) > ... 22 more > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19417) LIST SUPERUSERS cql command
[ https://issues.apache.org/jira/browse/CASSANDRA-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824104#comment-17824104 ] Shailaja Koppu commented on CASSANDRA-19417: As there are no further comments on ML thread, continuing with PR for code changes. [~smiklosovic] I have merged all commits into one. [~maxwellguo] resolved your comments, as I merged all commits into one, you won't see a separate commit just resolving your comments. I have also added doc for this new CQL command. > LIST SUPERUSERS cql command > --- > > Key: CASSANDRA-19417 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19417 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Shailaja Koppu >Assignee: Shailaja Koppu >Priority: Normal > Labels: CQL > Time Spent: 20m > Remaining Estimate: 0h > > Developing a new CQL command LIST SUPERUSERS to return list of roles with > superuser privilege. This includes roles who acquired superuser privilege in > the hierarchy. > Context: LIST ROLES cql command lists roles, their membership details and > displays super=true for immediate superusers. But there can be roles who > acquired superuser privilege due to a grant. LIST ROLES command won't display > super=true for such roles and the only way to recognize such roles is to look > for atleast one row with super=true in the output of LIST ROLES OF name> command. While this works to check is a given role has superuser > privilege, there may be services (for example, Sidecar) working with C* and > may need to maintain list of roles with superuser privilege. There is no > existing command/tool to retrieve such roles details. Hence developing this > command which returns all roles having superuser privilege. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19454) Revert switch to approximate time in Dispatcher to avoid mixing with nanoTime() in downstream timeout calculations
[ https://issues.apache.org/jira/browse/CASSANDRA-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824103#comment-17824103 ] Caleb Rackliffe commented on CASSANDRA-19454: - Alright, so CI looks good. (I've attached the summary from running on your trunk branch. Can ignore the upgrade test issues.) Do you mind attaching a 5.0 (cassandra-5.0) patch and hitting "Submit Patch" above to stay in line w/ the normal workflow? After that, I can commit this for you... > Revert switch to approximate time in Dispatcher to avoid mixing with > nanoTime() in downstream timeout calculations > -- > > Key: CASSANDRA-19454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19454 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Caleb Rackliffe >Assignee: Arun Ganesh >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: ci_summary.html > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15241 changed {{Dispatcher}} to use the {{approxTime}} > implementation of {{MonotonicClock}} rather than {{nanoTime()}}, but clock > drift between the two, can potentially cause queries to time out more > quickly. We should be able to revert the {{Dispatcher}} to use {{nanoTime()}} > again and similarly change {{QueriesTable} to {{nanoTime()}} as well for > consistency. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19454) Revert switch to approximate time in Dispatcher to avoid mixing with nanoTime() in downstream timeout calculations
[ https://issues.apache.org/jira/browse/CASSANDRA-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-19454: Attachment: ci_summary.html > Revert switch to approximate time in Dispatcher to avoid mixing with > nanoTime() in downstream timeout calculations > -- > > Key: CASSANDRA-19454 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19454 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: Caleb Rackliffe >Assignee: Arun Ganesh >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: ci_summary.html > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15241 changed {{Dispatcher}} to use the {{approxTime}} > implementation of {{MonotonicClock}} rather than {{nanoTime()}}, but clock > drift between the two, can potentially cause queries to time out more > quickly. We should be able to revert the {{Dispatcher}} to use {{nanoTime()}} > again and similarly change {{QueriesTable} to {{nanoTime()}} as well for > consistency. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18879) Modernize CQLSH datetime conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18879: - Fix Version/s: 5.1-alpha1 Source Control Link: https://github.com/apache/cassandra/commit/bfb5c593429c1bb615426ec2d35c1c27c7f81488 Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks, committed. > Modernize CQLSH datetime conversions > > > Key: CASSANDRA-18879 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18879 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Arun Ganesh >Priority: Low > Fix For: 5.1-alpha1 > > Attachments: cassandra-cqlsh-stdout > > Time Spent: 2h > Remaining Estimate: 0h > > Python 3.x introduced many updates to datetime conversion which allows > simplified conversions. > 1. For example, tracing.py defines a function datetime_from_utc_to_local() > but datetime now has a native function astimezone() which will convert UTC to > local time. > Review the following users of datetime which apply conversions: > * cqlshmain.py > * formatting.py > * tracing.py > Example: > {code:java} > >>> from dateutil import tz > >>> import datetime > >>> a = datetime.datetime.now().astimezone(tz.tzutc()) > >>> a > datetime.datetime(2023, 9, 25, 11, 22, 36, 251705, tzinfo=tzutc()) > >>> b = a.astimezone() > >>> b > datetime.datetime(2023, 9, 25, 14, 22, 36, 251705, > tzinfo=datetime.timezone(datetime.timedelta(seconds=10800), 'EST')) {code} > See [[PEP 495|http://example.com]]] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18879) Modernize CQLSH datetime conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18879: - Reviewers: Brad Schoening, Brandon Williams (was: Brandon Williams) Status: Review In Progress (was: Needs Committer) > Modernize CQLSH datetime conversions > > > Key: CASSANDRA-18879 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18879 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Arun Ganesh >Priority: Low > Attachments: cassandra-cqlsh-stdout > > Time Spent: 2h > Remaining Estimate: 0h > > Python 3.x introduced many updates to datetime conversion which allows > simplified conversions. > 1. For example, tracing.py defines a function datetime_from_utc_to_local() > but datetime now has a native function astimezone() which will convert UTC to > local time. > Review the following users of datetime which apply conversions: > * cqlshmain.py > * formatting.py > * tracing.py > Example: > {code:java} > >>> from dateutil import tz > >>> import datetime > >>> a = datetime.datetime.now().astimezone(tz.tzutc()) > >>> a > datetime.datetime(2023, 9, 25, 11, 22, 36, 251705, tzinfo=tzutc()) > >>> b = a.astimezone() > >>> b > datetime.datetime(2023, 9, 25, 14, 22, 36, 251705, > tzinfo=datetime.timezone(datetime.timedelta(seconds=10800), 'EST')) {code} > See [[PEP 495|http://example.com]]] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18879) Modernize CQLSH datetime conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18879: - Status: Ready to Commit (was: Review In Progress) > Modernize CQLSH datetime conversions > > > Key: CASSANDRA-18879 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18879 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Arun Ganesh >Priority: Low > Attachments: cassandra-cqlsh-stdout > > Time Spent: 2h > Remaining Estimate: 0h > > Python 3.x introduced many updates to datetime conversion which allows > simplified conversions. > 1. For example, tracing.py defines a function datetime_from_utc_to_local() > but datetime now has a native function astimezone() which will convert UTC to > local time. > Review the following users of datetime which apply conversions: > * cqlshmain.py > * formatting.py > * tracing.py > Example: > {code:java} > >>> from dateutil import tz > >>> import datetime > >>> a = datetime.datetime.now().astimezone(tz.tzutc()) > >>> a > datetime.datetime(2023, 9, 25, 11, 22, 36, 251705, tzinfo=tzutc()) > >>> b = a.astimezone() > >>> b > datetime.datetime(2023, 9, 25, 14, 22, 36, 251705, > tzinfo=datetime.timezone(datetime.timedelta(seconds=10800), 'EST')) {code} > See [[PEP 495|http://example.com]]] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: Fix datetime_from_utc_to_local in cqlshlib
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new bfb5c59342 Fix datetime_from_utc_to_local in cqlshlib bfb5c59342 is described below commit bfb5c593429c1bb615426ec2d35c1c27c7f81488 Author: arkn98 <20590666+ark...@users.noreply.github.com> AuthorDate: Tue Dec 19 23:12:17 2023 -0500 Fix datetime_from_utc_to_local in cqlshlib Patch by Arun Ganesh; reviewed by brads and brandonwilliams for CASSANDRA-18879 --- CHANGES.txt | 1 + pylib/cqlshlib/tracing.py | 11 ++- 2 files changed, 7 insertions(+), 5 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index ec63666e67..43b128db24 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 5.1 + * Modernize CQLSH datetime conversions (CASSANDRA-18879) * Harry model and in-JVM tests for partition-restricted 2i queries (CASSANDRA-18275) * Refactor cqlshmain global constants (CASSANDRA-19201) * Remove native_transport_port_ssl (CASSANDRA-19397) diff --git a/pylib/cqlshlib/tracing.py b/pylib/cqlshlib/tracing.py index 3db8e3491d..b7ee43c833 100644 --- a/pylib/cqlshlib/tracing.py +++ b/pylib/cqlshlib/tracing.py @@ -14,8 +14,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -from datetime import datetime -import time +from datetime import datetime, timezone from cassandra.query import QueryTrace, TraceUnavailable from cqlshlib.displaying import MAGENTA @@ -85,6 +84,8 @@ def total_micro_seconds(td): def datetime_from_utc_to_local(utc_datetime): -now_timestamp = time.time() -offset = datetime.fromtimestamp(now_timestamp) - datetime.utcfromtimestamp(now_timestamp) -return utc_datetime + offset +""" +Convert a naive UTC datetime to the local timezone. +This is necessary because the driver always returns naive datetime objects. +""" +return utc_datetime.replace(tzinfo=timezone.utc).astimezone() - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18879) Modernize CQLSH datetime conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18879: - Status: Needs Committer (was: Review In Progress) > Modernize CQLSH datetime conversions > > > Key: CASSANDRA-18879 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18879 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Arun Ganesh >Priority: Low > Attachments: cassandra-cqlsh-stdout > > Time Spent: 2h > Remaining Estimate: 0h > > Python 3.x introduced many updates to datetime conversion which allows > simplified conversions. > 1. For example, tracing.py defines a function datetime_from_utc_to_local() > but datetime now has a native function astimezone() which will convert UTC to > local time. > Review the following users of datetime which apply conversions: > * cqlshmain.py > * formatting.py > * tracing.py > Example: > {code:java} > >>> from dateutil import tz > >>> import datetime > >>> a = datetime.datetime.now().astimezone(tz.tzutc()) > >>> a > datetime.datetime(2023, 9, 25, 11, 22, 36, 251705, tzinfo=tzutc()) > >>> b = a.astimezone() > >>> b > datetime.datetime(2023, 9, 25, 14, 22, 36, 251705, > tzinfo=datetime.timezone(datetime.timedelta(seconds=10800), 'EST')) {code} > See [[PEP 495|http://example.com]]] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18879) Modernize CQLSH datetime conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824050#comment-17824050 ] Brandon Williams commented on CASSANDRA-18879: -- There was one dtest timeout in j17 that passed in j11 so I'm not concerned, and the jvm dtest/simulator failures are not caused by python changes, so I'm +1. [~bschoeni] WDYT? > Modernize CQLSH datetime conversions > > > Key: CASSANDRA-18879 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18879 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Arun Ganesh >Priority: Low > Attachments: cassandra-cqlsh-stdout > > Time Spent: 2h > Remaining Estimate: 0h > > Python 3.x introduced many updates to datetime conversion which allows > simplified conversions. > 1. For example, tracing.py defines a function datetime_from_utc_to_local() > but datetime now has a native function astimezone() which will convert UTC to > local time. > Review the following users of datetime which apply conversions: > * cqlshmain.py > * formatting.py > * tracing.py > Example: > {code:java} > >>> from dateutil import tz > >>> import datetime > >>> a = datetime.datetime.now().astimezone(tz.tzutc()) > >>> a > datetime.datetime(2023, 9, 25, 11, 22, 36, 251705, tzinfo=tzutc()) > >>> b = a.astimezone() > >>> b > datetime.datetime(2023, 9, 25, 14, 22, 36, 251705, > tzinfo=datetime.timezone(datetime.timedelta(seconds=10800), 'EST')) {code} > See [[PEP 495|http://example.com]]] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14572) Expose all table metrics in virtual table
[ https://issues.apache.org/jira/browse/CASSANDRA-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17824019#comment-17824019 ] Stefan Miklosovic commented on CASSANDRA-14572: --- first half of repeated units looks OK these tests were run org.apache.cassandra.utils.FBUtilitiesTest org.apache.cassandra.metrics.TrieMemtableMetricsTest org.apache.cassandra.metrics.TableMetricsTest org.apache.cassandra.metrics.LatencyMetricsTest org.apache.cassandra.metrics.KeyspaceMetricsTest org.apache.cassandra.metrics.JmxVirtualTableMetricsTest All tests would be run for more than 1 hour which would timeout on the plan I have. I go to schedule another half. [CASSANDRA-14572|https://github.com/instaclustr/cassandra/tree/CASSANDRA-14572] {noformat} java17_pre-commit_tests java17_separate_tests java11_pre-commit_tests java11_separate_tests ✓ j11_build4m 25s ✓ j11_unit_tests_repeat 41m 53s {noformat} [java17_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3956/workflows/705e470d-e5e0-4589-9f87-4498e036bc38] [java17_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3956/workflows/d8874b26-2bc2-42b8-bc57-32916d9685a9] [java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3956/workflows/0c567ddb-4d6c-41e8-b54f-8ac6bf9457bb] [java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/3956/workflows/0ffa8c05-fa84-4322-be6f-126ab6e9acb1] > Expose all table metrics in virtual table > - > > Key: CASSANDRA-14572 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14572 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Observability, Observability/Metrics >Reporter: Chris Lohfink >Assignee: Maxim Muzafarov >Priority: Low > Labels: virtual-tables > Fix For: 5.x > > Attachments: flight_recording_1270017199_13.jfr, keyspayces_group > responses times.png, keyspayces_group summary.png, select keyspaces_group by > string prefix.png, select keyspaces_group compare with wo.png, select > keyspaces_group without value.png, systemv_views.metrics_dropped_message.png, > thread_pools benchmark.png > > Time Spent: 7h > Remaining Estimate: 0h > > While we want a number of virtual tables to display data in a way thats great > and intuitive like in nodetool. There is also much for being able to expose > the metrics we have for tooling via CQL instead of JMX. This is more for the > tooling and adhoc advanced users who know exactly what they are looking for. > *Schema:* > Initial idea is to expose data via {{((keyspace, table), metric)}} with a > column for each metric value. Could also use a Map or UDT instead of the > column based that can be a bit more specific to each metric type. To that end > there can be a {{metric_type}} column and then a UDT for each metric type > filled in, or a single value with more of a Map style. I am > purposing the column type though as with {{ALLOW FILTERING}} it does allow > more extensive query capabilities. > *Implementations:* > * Use reflection to grab all the metrics from TableMetrics (see: > CASSANDRA-7622 impl). This is easiest and least abrasive towards new metric > implementors... but its reflection and a kinda a bad idea. > * Add a hook in TableMetrics to register with this virtual table when > registering > * Pull from the CassandraMetrics registery (either reporter or iterate > through metrics query on read of virtual table) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18753) Add an optimized default configuration to tests and make it available for new users
[ https://issues.apache.org/jira/browse/CASSANDRA-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823998#comment-17823998 ] Branimir Lambov commented on CASSANDRA-18753: - That test is apparently already fixed. [Latest run|https://app.circleci.com/pipelines/github/blambov/cassandra/606/workflows/628459f1-f3fe-449c-a047-a784cc9711f5/jobs/24959/tests] had only a timeout of {{ActiveCompactionsTest}} -- reduced the number of iterations in the test to fix this. Uploaded final version; I'm ready to commit it but I'd like one last review of the wording in {{NEWS.txt}} and {{cassandra(-latest).yaml}}. > Add an optimized default configuration to tests and make it available for new > users > --- > > Key: CASSANDRA-18753 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18753 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Urgent > Fix For: 5.0-rc, 5.x > > Attachments: > CCM_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch, > > DTEST_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch > > Time Spent: 11h > Remaining Estimate: 0h > > We currently offer only one sample configuration file with Cassandra, and > that file is deliberately configured to disable all new functionality and > incompatible improvements. This works well for legacy users that want to have > a painless upgrade, but is a very bad choice for new users, or anyone wanting > to make comparisons between Cassandra versions or between Cassandra and other > databases. > We offer very little indication, in the database packaging itself, that there > are well-tested configuration choices that can solve known problems and > dramatically improve performance. This is guaranteed to paint the database in > a worse light than it deserves, and will very likely hurt adoption. > We should find a way to offer a very easy way of choosing between "optimized" > and "compatible" defaults. At minimal, we could provide alternate yaml files. > Alternatively, we could build on the {{storage_compatibility_mode}} concept > to grow it into a setting that not only enables/disables certain settings, > but also changes their default values. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19459) test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions fails with SAI
[ https://issues.apache.org/jira/browse/CASSANDRA-19459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Branimir Lambov updated CASSANDRA-19459: Resolution: Fixed Status: Resolved (was: Triage Needed) Fixed by CASSANDRA-19018. > test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions > fails with SAI > --- > > Key: CASSANDRA-19459 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19459 > Project: Cassandra > Issue Type: Bug > Components: Feature/SAI >Reporter: Branimir Lambov >Priority: Normal > > The dtest > {{replica_side_filtering_test::TestSecondaryIndexes::test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions}} > fails when the default secondary index is switched to SAI with > {code} > test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions > failed; it passed 0 out of the required 1 times. > > Subprocess ['nodetool', '-h', 'localhost', '-p', '7200', 'flush'] > exited with non-zero status; exit status: 2; > stderr: error: null > -- StackTrace -- > java.lang.NullPointerException > at java.base/java.util.Objects.requireNonNull(Objects.java:209) > at > org.apache.cassandra.index.sai.disk.v1.segment.SegmentMetadata.(SegmentMetadata.java:102) > at > org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.flush(MemtableIndexWriter.java:166) > at > org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.complete(MemtableIndexWriter.java:125) > at > org.apache.cassandra.index.sai.disk.StorageAttachedIndexWriter.complete(StorageAttachedIndexWriter.java:185) > at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) > at > java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092) > at > org.apache.cassandra.io.sstable.format.SSTableWriter.commit(SSTableWriter.java:289) > at > org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.commit(SimpleSSTableMultiWriter.java:90) > at > org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1354) > at > org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1253) > at > org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:840) > {code} > Discovered while testing CASSANDRA-18753. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18879) Modernize CQLSH datetime conversions
[ https://issues.apache.org/jira/browse/CASSANDRA-18879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18879: - Status: Review In Progress (was: Patch Available) Looks good to me, let's check CI. ||Branch||CI|| |[trunk|https://github.com/driftx/cassandra/tree/CASSANDRA-18879-trunk]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1501/workflows/76d305a4-1a26-443f-bfc5-46b7c668247e], [j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1501/workflows/f0a96daf-b9ef-4da4-aa03-eed27314d697]| > Modernize CQLSH datetime conversions > > > Key: CASSANDRA-18879 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18879 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Brad Schoening >Assignee: Arun Ganesh >Priority: Low > Attachments: cassandra-cqlsh-stdout > > Time Spent: 2h > Remaining Estimate: 0h > > Python 3.x introduced many updates to datetime conversion which allows > simplified conversions. > 1. For example, tracing.py defines a function datetime_from_utc_to_local() > but datetime now has a native function astimezone() which will convert UTC to > local time. > Review the following users of datetime which apply conversions: > * cqlshmain.py > * formatting.py > * tracing.py > Example: > {code:java} > >>> from dateutil import tz > >>> import datetime > >>> a = datetime.datetime.now().astimezone(tz.tzutc()) > >>> a > datetime.datetime(2023, 9, 25, 11, 22, 36, 251705, tzinfo=tzutc()) > >>> b = a.astimezone() > >>> b > datetime.datetime(2023, 9, 25, 14, 22, 36, 251705, > tzinfo=datetime.timezone(datetime.timedelta(seconds=10800), 'EST')) {code} > See [[PEP 495|http://example.com]]] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (5dca216ee -> a333335a9)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 5dca216ee generate docs for fd550e9c new a35a9 generate docs for fd550e9c This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (5dca216ee) \ N -- N -- N refs/heads/asf-staging (a35a9) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4883646 -> 4883646 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18635) Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
[ https://issues.apache.org/jira/browse/CASSANDRA-18635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823956#comment-17823956 ] Stefan Miklosovic commented on CASSANDRA-18635: --- +1, I added 2 comments in the PR. > Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest > --- > > Key: CASSANDRA-18635 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18635 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Brandon Williams >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 5.0-rc, 5.x > > > Seen here: > https://app.circleci.com/pipelines/github/driftx/cassandra/1095/workflows/6114e2e3-8dcc-4bb0-b664-ae7d82c3349f/jobs/33405/tests > {noformat} > junit.framework.AssertionFailedError: expected:<0> but was:<2> > at > org.apache.cassandra.distributed.test.UpgradeSSTablesTest.upgradeSSTablesInterruptsOngoingCompaction(UpgradeSSTablesTest.java:86) > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18753) Add an optimized default configuration to tests and make it available for new users
[ https://issues.apache.org/jira/browse/CASSANDRA-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823946#comment-17823946 ] Branimir Lambov edited comment on CASSANDRA-18753 at 3/6/24 10:07 AM: -- Well, tests [look much better now|https://app.circleci.com/pipelines/github/blambov/cassandra/605/workflows/f567db7c-2231-4c22-8a60-7e43887880d7]. We have only one failure, {{replica_side_filtering_test.TestSecondaryIndexes:test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions}} with SAI. Opened CASSANDRA-19459 for this, and proceeding to merge this ticket. was (Author: blambov): Well, tests [look much better now|https://app.circleci.com/pipelines/github/blambov/cassandra/605/workflows/f567db7c-2231-4c22-8a60-7e43887880d7]. We have only one failure, {{replica_side_filtering_test.TestSecondaryIndexes:test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions}} with SAI. Opened CASSANDRA- 19459 for this, and proceeding to merge this ticket. > Add an optimized default configuration to tests and make it available for new > users > --- > > Key: CASSANDRA-18753 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18753 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Branimir Lambov >Assignee: Branimir Lambov >Priority: Urgent > Fix For: 5.0-rc, 5.x > > Attachments: > CCM_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch, > > DTEST_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch > > Time Spent: 11h > Remaining Estimate: 0h > > We currently offer only one sample configuration file with Cassandra, and > that file is deliberately configured to disable all new functionality and > incompatible improvements. This works well for legacy users that want to have > a painless upgrade, but is a very bad choice for new users, or anyone wanting > to make comparisons between Cassandra versions or between Cassandra and other > databases. > We offer very little indication, in the database packaging itself, that there > are well-tested configuration choices that can solve known problems and > dramatically improve performance. This is guaranteed to paint the database in > a worse light than it deserves, and will very likely hurt adoption. > We should find a way to offer a very easy way of choosing between "optimized" > and "compatible" defaults. At minimal, we could provide alternate yaml files. > Alternatively, we could build on the {{storage_compatibility_mode}} concept > to grow it into a setting that not only enables/disables certain settings, > but also changes their default values. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19459) test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions fails with SAI
Branimir Lambov created CASSANDRA-19459: --- Summary: test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions fails with SAI Key: CASSANDRA-19459 URL: https://issues.apache.org/jira/browse/CASSANDRA-19459 Project: Cassandra Issue Type: Bug Components: Feature/SAI Reporter: Branimir Lambov The dtest {{replica_side_filtering_test::TestSecondaryIndexes::test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions}} fails when the default secondary index is switched to SAI with {code} test_complementary_deletion_with_limit_on_partition_key_column_with_empty_partitions failed; it passed 0 out of the required 1 times. Subprocess ['nodetool', '-h', 'localhost', '-p', '7200', 'flush'] exited with non-zero status; exit status: 2; stderr: error: null -- StackTrace -- java.lang.NullPointerException at java.base/java.util.Objects.requireNonNull(Objects.java:209) at org.apache.cassandra.index.sai.disk.v1.segment.SegmentMetadata.(SegmentMetadata.java:102) at org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.flush(MemtableIndexWriter.java:166) at org.apache.cassandra.index.sai.disk.v1.MemtableIndexWriter.complete(MemtableIndexWriter.java:125) at org.apache.cassandra.index.sai.disk.StorageAttachedIndexWriter.complete(StorageAttachedIndexWriter.java:185) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.Collections$UnmodifiableCollection.forEach(Collections.java:1092) at org.apache.cassandra.io.sstable.format.SSTableWriter.commit(SSTableWriter.java:289) at org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.commit(SimpleSSTableMultiWriter.java:90) at org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1354) at org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1253) at org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:133) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.base/java.lang.Thread.run(Thread.java:840) {code} Discovered while testing CASSANDRA-18753. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18635) Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
[ https://issues.apache.org/jira/browse/CASSANDRA-18635?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823937#comment-17823937 ] Berenguer Blasi commented on CASSANDRA-18635: - Ping. Reviewer needed. > Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest > --- > > Key: CASSANDRA-18635 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18635 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Brandon Williams >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 5.0-rc, 5.x > > > Seen here: > https://app.circleci.com/pipelines/github/driftx/cassandra/1095/workflows/6114e2e3-8dcc-4bb0-b664-ae7d82c3349f/jobs/33405/tests > {noformat} > junit.framework.AssertionFailedError: expected:<0> but was:<2> > at > org.apache.cassandra.distributed.test.UpgradeSSTablesTest.upgradeSSTablesInterruptsOngoingCompaction(UpgradeSSTablesTest.java:86) > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18635) Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest
[ https://issues.apache.org/jira/browse/CASSANDRA-18635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-18635: Status: Needs Committer (was: Patch Available) > Test failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest > --- > > Key: CASSANDRA-18635 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18635 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/java >Reporter: Brandon Williams >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 5.0-rc, 5.x > > > Seen here: > https://app.circleci.com/pipelines/github/driftx/cassandra/1095/workflows/6114e2e3-8dcc-4bb0-b664-ae7d82c3349f/jobs/33405/tests > {noformat} > junit.framework.AssertionFailedError: expected:<0> but was:<2> > at > org.apache.cassandra.distributed.test.UpgradeSSTablesTest.upgradeSSTablesInterruptsOngoingCompaction(UpgradeSSTablesTest.java:86) > {noformat} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19458) Minor bugs in generate.sh -d
[ https://issues.apache.org/jira/browse/CASSANDRA-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-19458: Bug Category: Parent values: Correctness(12982) Complexity: Normal Discovered By: User Report Severity: Normal Status: Open (was: Triage Needed) > Minor bugs in generate.sh -d > > > Key: CASSANDRA-19458 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19458 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > > Option -d does not work if you don't skip repetition of tests. It should > error out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19458) Minor bugs in generate.sh -d
Berenguer Blasi created CASSANDRA-19458: --- Summary: Minor bugs in generate.sh -d Key: CASSANDRA-19458 URL: https://issues.apache.org/jira/browse/CASSANDRA-19458 Project: Cassandra Issue Type: Bug Components: CI Reporter: Berenguer Blasi Option -d does not work if you don't skip repetition of tests. It should error out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19458) Minor bugs in generate.sh -d
[ https://issues.apache.org/jira/browse/CASSANDRA-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi reassigned CASSANDRA-19458: --- Assignee: Berenguer Blasi > Minor bugs in generate.sh -d > > > Key: CASSANDRA-19458 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19458 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > > Option -d does not work if you don't skip repetition of tests. It should > error out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19398) Test Failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading
[ https://issues.apache.org/jira/browse/CASSANDRA-19398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823910#comment-17823910 ] Berenguer Blasi commented on CASSANDRA-19398: - Thanks for the review > Test Failure: > org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading > -- > > Key: CASSANDRA-19398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19398 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 5.0-rc, 5.1 > > > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2646/workflows/bc2bba74-9e56-4bea-8de7-4ff840c4f450/jobs/56028/tests#failed-test-0] > {code:java} > junit.framework.AssertionFailedError at > org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading(UpgradeSSTablesTest.java:220) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19398) Test Failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading
[ https://issues.apache.org/jira/browse/CASSANDRA-19398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-19398: Fix Version/s: 5.1 (was: 5.x) Since Version: 5.0-rc Source Control Link: https://github.com/apache/cassandra/commit/ebe07db6023493b4f0a1968af439fee1e664dbaf Resolution: Fixed Status: Resolved (was: Ready to Commit) > Test Failure: > org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading > -- > > Key: CASSANDRA-19398 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19398 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 5.0-rc, 5.1 > > > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/2646/workflows/bc2bba74-9e56-4bea-8de7-4ff840c4f450/jobs/56028/tests#failed-test-0] > {code:java} > junit.framework.AssertionFailedError at > org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading(UpgradeSSTablesTest.java:220) > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) at > java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) > at > java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43){code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (8b429c8ef9 -> 6e72cf0ee5)
This is an automated email from the ASF dual-hosted git repository. bereng pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 8b429c8ef9 Merge branch 'cassandra-5.0' into trunk new ebe07db602 Test Failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading new 6e72cf0ee5 Merge branch 'cassandra-5.0' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../distributed/test/UpgradeSSTablesTest.java | 58 +++--- 1 file changed, 50 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated: Test Failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading
This is an automated email from the ASF dual-hosted git repository. bereng pushed a commit to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-5.0 by this push: new ebe07db602 Test Failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading ebe07db602 is described below commit ebe07db6023493b4f0a1968af439fee1e664dbaf Author: Bereng AuthorDate: Tue Mar 5 11:29:56 2024 +0100 Test Failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading patch by Berenguer Blasi; reviewed by Brandon Williams for CASSANDRA-19398 --- .../distributed/test/UpgradeSSTablesTest.java | 58 +++--- 1 file changed, 50 insertions(+), 8 deletions(-) diff --git a/test/distributed/org/apache/cassandra/distributed/test/UpgradeSSTablesTest.java b/test/distributed/org/apache/cassandra/distributed/test/UpgradeSSTablesTest.java index 2bdcaf16c6..76c6c189c4 100644 --- a/test/distributed/org/apache/cassandra/distributed/test/UpgradeSSTablesTest.java +++ b/test/distributed/org/apache/cassandra/distributed/test/UpgradeSSTablesTest.java @@ -50,10 +50,11 @@ import static org.apache.cassandra.utils.concurrent.CountDownLatch.newCountDownL public class UpgradeSSTablesTest extends TestBaseImpl { + @Test public void upgradeSSTablesInterruptsOngoingCompaction() throws Throwable { -try (ICluster cluster = init(builder().withNodes(1).start())) +try (ICluster cluster = init(builder().withNodes(1).withInstanceInitializer(CompactionLatchByteman::install).start())) { cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".tbl (pk int, ck int, v text, PRIMARY KEY (pk, ck));"); cluster.get(1).acceptsOnInstance((String ks) -> { @@ -79,10 +80,15 @@ public class UpgradeSSTablesTest extends TestBaseImpl LogAction logAction = cluster.get(1).logs(); logAction.mark(); + Future future = cluster.get(1).asyncAcceptsOnInstance((String ks) -> { ColumnFamilyStore cfs = Keyspace.open(ks).getColumnFamilyStore("tbl"); CompactionManager.instance.submitMaximal(cfs, FBUtilities.nowInSeconds(), false, OperationType.COMPACTION); }).apply(KEYSPACE); + +Assert.assertTrue(cluster.get(1).callOnInstance(() -> CompactionLatchByteman.starting.awaitUninterruptibly(1, TimeUnit.MINUTES))); +cluster.get(1).runOnInstance(() -> { +CompactionLatchByteman.start.decrement();}); Assert.assertEquals(0, cluster.get(1).nodetool("upgradesstables", "-a", KEYSPACE, "tbl")); future.get(); Assert.assertFalse(logAction.grep("Compaction interrupted").getResult().isEmpty()); @@ -136,7 +142,7 @@ public class UpgradeSSTablesTest extends TestBaseImpl @Test public void cleanupDoesNotInterruptUpgradeSSTables() throws Throwable { -try (ICluster cluster = init(builder().withNodes(1).withInstanceInitializer(BB::install).start())) +try (ICluster cluster = init(builder().withNodes(1).withInstanceInitializer(UpgradeSStablesLatchByteman::install).start())) { cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".tbl (pk int, ck int, v text, PRIMARY KEY (pk, ck));"); @@ -160,12 +166,12 @@ public class UpgradeSSTablesTest extends TestBaseImpl LogAction logAction = cluster.get(1).logs(); logAction.mark(); -// Start upgradingsstables - use BB to pause once inside ActiveCompactions.beginCompaction +// Start upgradingsstables - use UpgradeSStablesLatchByteman to pause once inside ActiveCompactions.beginCompaction Thread upgradeThread = new Thread(() -> { cluster.get(1).nodetool("upgradesstables", "-a", KEYSPACE, "tbl"); }); upgradeThread.start(); -Assert.assertTrue(cluster.get(1).callOnInstance(() -> BB.starting.awaitUninterruptibly(1, TimeUnit.MINUTES))); +Assert.assertTrue(cluster.get(1).callOnInstance(() -> UpgradeSStablesLatchByteman.starting.awaitUninterruptibly(1, TimeUnit.MINUTES))); // Start a scrub and make sure that it fails, log check later to make sure it was // because it cannot cancel the active upgrade sstables @@ -173,7 +179,7 @@ public class UpgradeSSTablesTest extends TestBaseImpl // Now resume the upgrade sstables so test can shut down cluster.get(1).runOnInstance(() -> { -BB.start.decrement(); +UpgradeSStablesLatchByteman.start.decrement(); }); upgradeThread.join(); @@ -186,7 +192,7 @@ public class UpgradeSSTablesTest extends TestBaseImpl @Test public void truncateWhileUpgrading() throws Throwable { -try (ICluster cluster = init(builder(
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. bereng pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 6e72cf0ee51a090ece17ff2981e4d101fd9f63f6 Merge: 8b429c8ef9 ebe07db602 Author: Bereng AuthorDate: Wed Mar 6 09:33:14 2024 +0100 Merge branch 'cassandra-5.0' into trunk * cassandra-5.0: Test Failure: org.apache.cassandra.distributed.test.UpgradeSSTablesTest.truncateWhileUpgrading .../distributed/test/UpgradeSSTablesTest.java | 58 +++--- 1 file changed, 50 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19391) Flush metadata snapshot table on every write
[ https://issues.apache.org/jira/browse/CASSANDRA-19391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823906#comment-17823906 ] Sam Tunnicliffe commented on CASSANDRA-19391: - +1 new CI looks good > Flush metadata snapshot table on every write > > > Key: CASSANDRA-19391 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19391 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Low > Fix For: 5.x > > Attachments: ci_summary.html, result_details.tar.gz > > > We depend on the latest snapshot when starting up, flushing avoids gaps > between latest snapshot and the most recent local log entry -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19390) Transformation.Kind should contain an explicit integer id
[ https://issues.apache.org/jira/browse/CASSANDRA-19390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823905#comment-17823905 ] Sam Tunnicliffe commented on CASSANDRA-19390: - +1 new CI looks good > Transformation.Kind should contain an explicit integer id > - > > Key: CASSANDRA-19390 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19390 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Low > Fix For: 5.x > > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org