[jira] [Commented] (CASSANDRA-17009) Sstableverify unit test operate on SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-17009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423135#comment-17423135
]
Berenguer Blasi commented on CASSANDRA-17009:
-
> I accidentally turned off the spell checker in the ide as it turns out...
> blerg I'll fix them, sorry about that.
No need to apologize! I do these sort of things all the time unfortunately lol!
> The fix for this is small, but I was trying to keep the change small,
> incremental and focused on the unit test. No sweat though, I can fix it
> (basically just force an early exit).
And you're absolutely right. But from experience I can tell you each ticket is
sandwiched between CI and contributor review cycle time. That tends to be not
negligible so if it's a few loc of a related fix that'd be my suggestion to
keep things moving. But feel free to do whatever you think is best ofc!
> The issue with the git ignore of test/data sounds weird to me it didn't came
> up sooner. I have to give it another thought.
Ok I'll buy that.
> On the {{Verifier}} and coverage discussion
Icwym and I think you are right. Still I would suggest:
- Adding to all verify test classes a javadoc detailing all the files involved:
VerifyTest, StandaloneVerifierOnSSTablesTest, etc This way a reader will learn
immediately about coverage and what classes are involved instead of wondering
around and finding it out himself. Wdyt?
- Yes replicating those tests would be redundant and we already have existing
tests on the cmd line options making sure the tool doesn't explode but
- The -c test we need to add to {{StandaloneVerifierTest}} bc we have no test
proving this option doesn't explode the tool or that the tool is ignoring it
i.e. Same as with the token option which in this case is another ticket. But
-c, being minimal effort, I'd suggest doing in this ticket.
Wdyt makes sense? I'll move the state of the ticket to help me track things if
you don't mind.
> Sstableverify unit test operate on SSTables
> ---
>
> Key: CASSANDRA-17009
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17009
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/sstable
>Reporter: Brian Houser
>Assignee: Brian Houser
>Priority: Normal
> Time Spent: 2h
> Remaining Estimate: 0h
>
> as part of https://issues.apache.org/jira/browse/CASSANDRA-16009, unit
> coverage is a bit lax and doesn't run through the verifier (based on my
> coverage results).
> There should be a unit test that exercises the internal verifier both for a
> corrupt example and a working example.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17009) Sstableverify unit test operate on SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-17009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-17009: Status: Changes Suggested (was: Review In Progress) > Sstableverify unit test operate on SSTables > --- > > Key: CASSANDRA-17009 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17009 > Project: Cassandra > Issue Type: Sub-task > Components: Tool/sstable >Reporter: Brian Houser >Assignee: Brian Houser >Priority: Normal > Time Spent: 2h > Remaining Estimate: 0h > > as part of https://issues.apache.org/jira/browse/CASSANDRA-16009, unit > coverage is a bit lax and doesn't run through the verifier (based on my > coverage results). > There should be a unit test that exercises the internal verifier both for a > corrupt example and a working example. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423126#comment-17423126 ] Stefan Miklosovic commented on CASSANDRA-14612: --- Successful OWASP check against trunk is visible here - timestamp aroud 08:39:59 - https://ci-cassandra.apache.org/job/Cassandra-devbranch-artifacts/1085/jdk=jdk_1.8_latest,label=cassandra/console I consider this ticket to be fully resolved. > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Assignee: Stefan Miklosovic >Priority: Normal > Labels: build, security > Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1 > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16981) Respect com.sun.management.jxmremote.host
[ https://issues.apache.org/jira/browse/CASSANDRA-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423057#comment-17423057 ] Ekaterina Dimitrova edited comment on CASSANDRA-16981 at 10/1/21, 12:09 AM: It seems the builds and the new 3.11 tests were fine. For the record, the ongoing Jenkins issue started before I pushed the image and it is not related. ||Patch||CI|| |[3.11|https://github.com/apache/cassandra/commit/2bbeecabb787a1aed1593399c29f3b9bcdc1e923]|[Circle|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1131/workflows/52ed7138-c1ab-4604-bfc9-d68c8ba1292c], [Jenkins|https://jenkins-cm4.apache.org/job/Cassandra-devbranch/1152/]| |[4.0|https://github.com/ekaterinadimitrova2/cassandra/pull/182/commits/1e2e1871dddc7a21b20dcb81e4b9ec6f594fafac]|[j8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1134/workflows/82fc0892-6978-41e3-b234-840b24d13fb0], [j11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=16981-4.0]| |[trunk|https://github.com/apache/cassandra/commit/3bdb364edc97839f094fa4a95e445099c3c9ec2a]|[j8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1135/workflows/3e76993a-8ff0-442b-ae19-e3c6abf09ee6], [j11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1135/workflows/a36af32c-ce61-47cf-b0bd-f8343025ca05]| I propagated the patch to all branches and pushed build and unit tests. Should be fine. I can submit full Jenkins build when we solved the issues there was (Author: e.dimitrova): It seems the builds and the new 3.11 tests were fine. For the record, the ongoing Jenkins issue started before I pushed the image and it is not related. ||Patch||CI|| |[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/57983f673d9657d88aecdb4942519134a6c2fa89]|[Circle|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1131/workflows/52ed7138-c1ab-4604-bfc9-d68c8ba1292c], [Jenkins|https://jenkins-cm4.apache.org/job/Cassandra-devbranch/1152/]| |[4.0|https://github.com/ekaterinadimitrova2/cassandra/pull/182/commits/1e2e1871dddc7a21b20dcb81e4b9ec6f594fafac]|[j8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1134/workflows/82fc0892-6978-41e3-b234-840b24d13fb0], [j11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=16981-4.0]| |[trunk|https://github.com/apache/cassandra/commit/3bdb364edc97839f094fa4a95e445099c3c9ec2a]|[j8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1135/workflows/3e76993a-8ff0-442b-ae19-e3c6abf09ee6], [j11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1135/workflows/a36af32c-ce61-47cf-b0bd-f8343025ca05]| I propagated the patch to all branches and pushed build and unit tests. Should be fine. I can submit full Jenkins build when we solved the issues there > Respect com.sun.management.jxmremote.host > -- > > Key: CASSANDRA-16981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16981 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0.x, 4.x > > Attachments: Screenshot 2021-09-27 at 22.27.57.png > > > It seems we don't respect com.sun.management.jxmremote.host in > [createJMXServer()|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/utils/JMXServerUtils.java#L64] > The flow should be - when_ local _is true, we only bind localhost, otherwise > we bind everything, unless com.sun.management.jmxremote.host is set, then we > use that. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16981) Respect com.sun.management.jxmremote.host
[ https://issues.apache.org/jira/browse/CASSANDRA-16981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423057#comment-17423057 ] Ekaterina Dimitrova commented on CASSANDRA-16981: - It seems the builds and the new 3.11 tests were fine. For the record, the ongoing Jenkins issue started before I pushed the image and it is not related. ||Patch||CI|| |[3.11|https://github.com/ekaterinadimitrova2/cassandra/commit/57983f673d9657d88aecdb4942519134a6c2fa89]|[Circle|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1131/workflows/52ed7138-c1ab-4604-bfc9-d68c8ba1292c], [Jenkins|https://jenkins-cm4.apache.org/job/Cassandra-devbranch/1152/]| |[4.0|https://github.com/ekaterinadimitrova2/cassandra/pull/182/commits/1e2e1871dddc7a21b20dcb81e4b9ec6f594fafac]|[j8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1134/workflows/82fc0892-6978-41e3-b234-840b24d13fb0], [j11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=16981-4.0]| |[trunk|https://github.com/apache/cassandra/commit/3bdb364edc97839f094fa4a95e445099c3c9ec2a]|[j8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1135/workflows/3e76993a-8ff0-442b-ae19-e3c6abf09ee6], [j11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1135/workflows/a36af32c-ce61-47cf-b0bd-f8343025ca05]| I propagated the patch to all branches and pushed build and unit tests. Should be fine. I can submit full Jenkins build when we solved the issues there > Respect com.sun.management.jxmremote.host > -- > > Key: CASSANDRA-16981 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16981 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Core >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 3.11.x, 4.0.x, 4.x > > Attachments: Screenshot 2021-09-27 at 22.27.57.png > > > It seems we don't respect com.sun.management.jxmremote.host in > [createJMXServer()|https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/utils/JMXServerUtils.java#L64] > The flow should be - when_ local _is true, we only bind localhost, otherwise > we bind everything, unless com.sun.management.jmxremote.host is set, then we > use that. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Revert "Ninja lock setuptools version for thrift"
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 97108ef Revert "Ninja lock setuptools version for thrift" 97108ef is described below commit 97108ef0a70c432ae241286e4b91ed3658700481 Author: Brandon Williams AuthorDate: Thu Sep 30 18:37:54 2021 -0500 Revert "Ninja lock setuptools version for thrift" This reverts commit 1907bccc2b9abf3845626d0a157a3f42d9ced0a1. --- pylib/requirements.txt | 1 - 1 file changed, 1 deletion(-) diff --git a/pylib/requirements.txt b/pylib/requirements.txt index 2d5e574..17d4ab0 100644 --- a/pylib/requirements.txt +++ b/pylib/requirements.txt @@ -19,4 +19,3 @@ parse pycodestyle psutil thrift==0.9.3 -setuptools==57.4.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16009) sstableverify unit test hardening and docs improvement
[
https://issues.apache.org/jira/browse/CASSANDRA-16009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423050#comment-17423050
]
Brian Houser commented on CASSANDRA-16009:
--
Thanks! I generally wanted to break things up as the documentation is pretty
separate and I've already seen has a different time table. The token range
functionality also seemed distinct and requiring me to ask a few questions
regarding its requirements.
> sstableverify unit test hardening and docs improvement
> --
>
> Key: CASSANDRA-16009
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16009
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Brian Houser
>Priority: Normal
> Labels: low-hanging-fruit
> Fix For: 4.0.x
>
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}. Also docs need to be updated with
> any missing options and the {{token_range}} option needs to be fixed as it
> appears broken.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17018) WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10
[ https://issues.apache.org/jira/browse/CASSANDRA-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper reassigned CASSANDRA-17018: Assignee: Diogenese Topper > WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10 > --- > > Key: CASSANDRA-17018 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17018 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Assignee: Diogenese Topper >Priority: Normal > Labels: pull-request-available > Fix For: NA > > > This ticket is to capture the work associated with publishing the October > 2021 blog post for the Apache Cassandra Changelog #10. > We can close this once the blog post has gone live on the website. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17018) WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10
[ https://issues.apache.org/jira/browse/CASSANDRA-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423042#comment-17423042 ] Diogenese Topper commented on CASSANDRA-17018: -- Pull request: https://github.com/apache/cassandra-website/pull/77 > WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10 > --- > > Key: CASSANDRA-17018 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17018 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > Labels: pull-request-available > Fix For: NA > > > This ticket is to capture the work associated with publishing the October > 2021 blog post for the Apache Cassandra Changelog #10. > We can close this once the blog post has gone live on the website. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16666) Make SSLContext creation pluggable/extensible
[ https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423041#comment-17423041 ] Maulin Vasavada commented on CASSANDRA-1: - For the comment around location of the Kubernetes example's test create file in the test/conf folder, I can follow-up when we make more documentation changes for the new documentation format. > Make SSLContext creation pluggable/extensible > - > > Key: CASSANDRA-1 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Maulin Vasavada >Assignee: Maulin Vasavada >Priority: Normal > Fix For: 4.x > > Attachments: Screenshot from 2021-09-28 10-56-24.png > > > Currently Cassandra creates the SSLContext via SSLFactory.java. SSLFactory is > a final class with static methods and not overridable. The SSLFactory loads > the keys and certs from the file based artifacts for the same. While this > works for many, in the industry where security is stricter and contextual, > this approach falls short. Many big organizations need flexibility to load > the SSL artifacts from a custom resource (like custom Key Management > Solution, HashiCorp Vault, Amazon KMS etc). While JSSE SecurityProvider > architecture allows us flexibility to build our custom mechanisms to validate > and process security artifacts, many times all we need is to build upon > Java's existing extensibility that Trust/Key Manager interfaces provide to > load keystores from various resources in the absence of any customized > requirements on the Keys/Certificate formats. > My proposal here is to make the SSLContext creation pluggable/extensible and > have the current SSLFactory.java implement an extensible interface. > I contributed a similar change that is live now in Apache Kafka (2.6.0) - > https://issues.apache.org/jira/browse/KAFKA-8890 > I can spare some time writing the pluggable interface and run by the required > reviewers. > > Created [CEP-9: Make SSLContext creation > pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable] > > > cc: [~dcapwell] [~djoshi] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17018) WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10
[ https://issues.apache.org/jira/browse/CASSANDRA-17018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-17018: - Authors: Diogenese Topper Change Category: Operability Complexity: Low Hanging Fruit Fix Version/s: NA Impacts: Docs (was: None) Mentor: Anthony Grasso Reviewers: Anthony Grasso, Erick Ramirez, Michael Semb Wever Test and Documentation Plan: * Blog post is titled "Apache Cassandra Changelog #10" * Modified blog index page Labels: pull-request-available (was: ) > WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10 > --- > > Key: CASSANDRA-17018 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17018 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > Labels: pull-request-available > Fix For: NA > > > This ticket is to capture the work associated with publishing the October > 2021 blog post for the Apache Cassandra Changelog #10. > We can close this once the blog post has gone live on the website. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17018) WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10
Diogenese Topper created CASSANDRA-17018: Summary: WEBSITE - October 2021 blog post for Apache Cassandra Changelog #10 Key: CASSANDRA-17018 URL: https://issues.apache.org/jira/browse/CASSANDRA-17018 Project: Cassandra Issue Type: Task Components: Documentation/Blog Reporter: Diogenese Topper This ticket is to capture the work associated with publishing the October 2021 blog post for the Apache Cassandra Changelog #10. We can close this once the blog post has gone live on the website. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17009) Sstableverify unit test operate on SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-17009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423035#comment-17423035
]
Brian Houser commented on CASSANDRA-17009:
--
> I noted a few minor typos in the PR
I accidentally turned off the spell checker in the ide as it turns out... blerg
I'll fix them, sorry about that.
> Sthg weird happened with the file {{cassandra.in.sh}}. Make sure you're
> branching off the right place etc
not sure how that ended up in the mix and it has no changes, I'll remove it. I
was branching off cassandra-4.0 at the time of submitting
> I would use this opportunity to add {{assertOnCleanExit()}} on the already
> existing test when possible
Np, I'll go ahead and do that.
>That bug on the return code I would fix in this ticket. Bear in mind CI is
>heavy and we'll need to ping more people to review this ticket. The less we go
>back and forth the better. So if we can pack it here, within reason, I'd do
>that. Seems to me It'd be a few loc only.
The fix for this is small, but I was trying to keep the change small,
incremental and focused on the unit test. No sweat though, I can fix it
(basically just force an early exit).
> The issue with the git ignore of {{test/data}} sounds weird to me it didn't
> came up sooner. I have to give it another thought.
Yeah it was a weird thing to me when I found it, pretty anti-intuitive and I
had to look up the git docs for fear I was being nutty. I mentioned it on the
slack channel before doing this. We have are ignoring "data/" which ignores
any folder containing data. It hasn't come up because any time that I guess we
added to it we just used the force option (also older files will avoid the
gitignore by virtue of being grandfathered in). It seemed like a good idea to
file an exception, and chatting with folk they said I should just jump on it.
From [https://git-scm.com/docs/gitignore] * If there is a separator at the end
of the pattern then the pattern will only match directories, otherwise the
pattern can match both files and directories.
* For example, a pattern {{doc/frotz/}} matches {{doc/frotz}} directory, but
not {{a/doc/frotz}} directory; however {{frotz/}} matches {{frotz}} and
{{a/frotz}} that is a directory (all paths are relative from the {{.gitignore}}
file).
I think this is the {{frotz/}} case here.Just to make sure I wasn’t crazy, I
created a file inside of test/data to see if was being tracked, my git ignored
it.
I called this “helloworld”. added some text and then ran…
$ git check-ignore -v helloworld
.gitignore:8:data/ helloworld
>The already existing tests only 'scratch the surface'. They pass an argument
>and check the tool doesn't explode. But they don't check the sstables
>themselves to make sure the argument was effective. I.e does '-r' really have
>an effect?
For the most part, this is a verifier, so its job is to tell you if something
is messed up, not necessarily do much about it(or at least that was my
thought). The tests verify you get a verify result. Though I could see about
adding a `-r` as that has the potential to mutate.
There is already a verifyer test (which is mainly all this class exercises)
called VerifyTest, this test already covers said cases. I thought it would be
somewhat redundant to replicate most of the same tests in a unit test. I
thought better to make sure that the happy and unhappy verifyer results are
caught and verify the execution paths.
> The new tests you added are really nice! but for completion purposes I would
> combine them with -q, -e, etc and any other params to make sure they all work
> with corrupted data i.e.
> We seem to be missing a -c test.
I can add these, though as I said, at that point I am mostly doing similar
tests as in VerifyTest (they really unit test that component through the
verifier). It won't add to the coverage of the suite.
> Sstableverify unit test operate on SSTables
> ---
>
> Key: CASSANDRA-17009
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17009
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/sstable
>Reporter: Brian Houser
>Assignee: Brian Houser
>Priority: Normal
> Time Spent: 2h
> Remaining Estimate: 0h
>
> as part of https://issues.apache.org/jira/browse/CASSANDRA-16009, unit
> coverage is a bit lax and doesn't run through the verifier (based on my
> coverage results).
> There should be a unit test that exercises the internal verifier both for a
> corrupt example and a working example.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mai
[jira] [Commented] (CASSANDRA-17017) Add required -f / --force option to nodetool verify
[ https://issues.apache.org/jira/browse/CASSANDRA-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17423017#comment-17423017 ] Caleb Rackliffe commented on CASSANDRA-17017: - +1 > Add required -f / --force option to nodetool verify > --- > > Key: CASSANDRA-17017 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17017 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > > nodetool verify has some pretty significant problems with it (see > CASSANDRA-9947). > Until such time as we do the heavy(er) lift to fix the command, we should > make it harder for people to shoot themselves in the foot with it. Adding a > required "-f" flag to it with a requisite "Do you really know what you're > doing? Check out this JIRA first" seems like it'd be the right thing to do in > the interim. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17017) Add required -f / --force option to nodetool verify
[ https://issues.apache.org/jira/browse/CASSANDRA-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-17017: Reviewers: Caleb Rackliffe Status: Review In Progress (was: Patch Available) > Add required -f / --force option to nodetool verify > --- > > Key: CASSANDRA-17017 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17017 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > > nodetool verify has some pretty significant problems with it (see > CASSANDRA-9947). > Until such time as we do the heavy(er) lift to fix the command, we should > make it harder for people to shoot themselves in the foot with it. Adding a > required "-f" flag to it with a requisite "Do you really know what you're > doing? Check out this JIRA first" seems like it'd be the right thing to do in > the interim. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Ninja lock setuptools version for thrift
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 1907bcc Ninja lock setuptools version for thrift 1907bcc is described below commit 1907bccc2b9abf3845626d0a157a3f42d9ced0a1 Author: Brandon Williams AuthorDate: Thu Sep 30 15:35:51 2021 -0500 Ninja lock setuptools version for thrift --- pylib/requirements.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/pylib/requirements.txt b/pylib/requirements.txt index 17d4ab0..2d5e574 100644 --- a/pylib/requirements.txt +++ b/pylib/requirements.txt @@ -19,3 +19,4 @@ parse pycodestyle psutil thrift==0.9.3 +setuptools==57.4.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-14612: -- Fix Version/s: (was: 4.0.x) (was: 3.11.x) (was: 4.x) (was: 3.0.x) 4.1 4.0.2 3.11.12 3.0.26 > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Assignee: Stefan Miklosovic >Priority: Normal > Labels: build, security > Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1 > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-14612: -- Source Control Link: https://github.com/apache/cassandra/commit/225a4c8faf7a2a67a1a8a360bc4efb70b36f6ae7 Resolution: Fixed Status: Resolved (was: Ready to Commit) Cassandra branches committed, cassandra-builds committed here: https://github.com/apache/cassandra-builds/commit/6d5db5f73702dbbf12b5c720477a24f8def2a504 I am running the build in Jenkins with cassandra-build patch already in, if it fails, I ll open another ticket where I fix that if something is wrong or I will just ninja it. https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1154/ > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Assignee: Stefan Miklosovic >Priority: Normal > Labels: build, security > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-14612: -- Status: Ready to Commit (was: Review In Progress) > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Assignee: Stefan Miklosovic >Priority: Normal > Labels: build, security > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.x > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated (d1a3a0c -> 6d5db5f)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git. from d1a3a0c re-enable legacy TLSv1 and TLSv1.1 in both openjdk-8 and openjdk-11 add 6d5db5f add dependency check into cassandra-artifacts.sh No new revisions were added by this update. Summary of changes: build-scripts/cassandra-artifacts.sh | 43 +++- 1 file changed, 38 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (3e6faca -> 225a4c8)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 3e6faca Do not release new SSTables in offline transactions add 225a4c8 add OWASP dependency check to Ant build No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 86 ++ .build/dependency-check-suppressions.xml | 91 build.xml| 1 + 3 files changed, 178 insertions(+) create mode 100644 .build/build-owasp.xml create mode 100644 .build/dependency-check-suppressions.xml - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 3cc59017a4aa09ea572f7f2bc7c2c9f9bd9bd09e Merge: b6e400d 59f5e57 Author: Stefan Miklosovic AuthorDate: Thu Sep 30 21:36:06 2021 +0200 Merge branch 'cassandra-4.0' into trunk .build/build-owasp.xml | 86 .build/dependency-check-suppressions.xml | 47 + build.xml| 1 + 3 files changed, 134 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (b6e400d -> 3cc5901)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b6e400d Merge branch 'cassandra-4.0' into trunk add 225a4c8 add OWASP dependency check to Ant build add 9bf14a6 Merge branch 'cassandra-3.0' into cassandra-3.11 add 59f5e57 Merge branch 'cassandra-3.11' into cassandra-4.0 new 3cc5901 Merge branch 'cassandra-4.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/build-owasp.xml | 86 .build/dependency-check-suppressions.xml | 47 + build.xml| 1 + 3 files changed, 134 insertions(+) create mode 100644 .build/build-owasp.xml create mode 100644 .build/dependency-check-suppressions.xml - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (c0916d6 -> 59f5e57)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from c0916d6 Merge branch 'cassandra-3.11' into cassandra-4.0 add 225a4c8 add OWASP dependency check to Ant build add 9bf14a6 Merge branch 'cassandra-3.0' into cassandra-3.11 add 59f5e57 Merge branch 'cassandra-3.11' into cassandra-4.0 No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 86 .build/dependency-check-suppressions.xml | 47 + build.xml| 1 + 3 files changed, 134 insertions(+) create mode 100644 .build/build-owasp.xml create mode 100644 .build/dependency-check-suppressions.xml - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (dcc9549 -> 9bf14a6)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from dcc9549 Merge branch 'cassandra-3.0' into cassandra-3.11 add 225a4c8 add OWASP dependency check to Ant build add 9bf14a6 Merge branch 'cassandra-3.0' into cassandra-3.11 No new revisions were added by this update. Summary of changes: .build/build-owasp.xml | 86 .build/dependency-check-suppressions.xml | 73 +++ build.xml| 1 + 3 files changed, 160 insertions(+) create mode 100644 .build/build-owasp.xml create mode 100644 .build/dependency-check-suppressions.xml - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17014) Make -Dtest.methods optional in all Ant test targets
[
https://issues.apache.org/jira/browse/CASSANDRA-17014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422948#comment-17422948
]
Andres de la Peña commented on CASSANDRA-17014:
---
I have tested locally the cases mentioned in the examples, which for trunk are:
{code:java}
ant testsome -Dtest.name=org.apache.cassandra.service.StorageServiceServerTest
ant testsome -Dtest.name=org.apache.cassandra.service.StorageServiceServerTest
-Dtest.methods=testRegularMode,testGetAllRangesEmpty
ant long-testsome -Dtest.name=org.apache.cassandra.cql3.ViewLongTest
ant long-testsome -Dtest.name=org.apache.cassandra.cql3.ViewLongTest
-Dtest.methods=testConflictResolution
ant burn-testsome
-Dtest.name=org.apache.cassandra.utils.memory.LongBufferPoolTest
ant burn-testsome
-Dtest.name=org.apache.cassandra.utils.memory.LongBufferPoolTest
-Dtest.methods=testPoolAllocateWithRecyclePartially
ant cql-test-some -Dtest.name=ListsTest
ant cql-test-some -Dtest.name=ListsTest
-Dtest.methods=testPrecisionTime_getNext_simple
ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.ResourceLeakTest
ant test-jvm-dtest-some
-Dtest.name=org.apache.cassandra.distributed.test.ResourceLeakTest
-Dtest.methods=looperTest
ant stress-test-some
-Dtest.name=org.apache.cassandra.stress.generate.DistributionGaussianTest
ant stress-test-some
-Dtest.name=org.apache.cassandra.stress.generate.DistributionGaussianTest
-Dtest.methods=simpleGaussian
{code}
I have also tried the other branches, where {{burn-testsome}} is only available
since 3.11 and {{stress-test-some}} is available since 4.0.
From these tests, the one with {{testConflictResolution}} doesn't work for 4.0
and trunk because of the parameterization of the \{{ViewLongTest}}, but fixing
that is beyond the scope of this ticket.
> Make -Dtest.methods optional in all Ant test targets
>
>
> Key: CASSANDRA-17014
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17014
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Low
>
> The Ant build file contains several targets to run specific tests:
> * testsome
> * long-testsome
> * burn-testsome
> * cql-test-some
> * stress-test-some
> * test-jvm-dtest-some
> These targets take two arguments, {{test.name}} and {{test.methods}}. The
> first argument, {{test.name}}, is always mandatory. However, {{test.methods}}
> can be either mandatory or optional depending on the target and the project
> branch. For example, in {{testsome}} the argument is mandatory in 3.0 and
> 3.11 and optional in 4.0 and trunk. Also, in trunk {{test.methods}} is
> optional for all the targets except {{cql-test-some}}.
> The purpose of this ticket is making {{test.methods}} consistently optional
> for all the {{*some}} targets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15005) Configurable whilelist for UDFs
[
https://issues.apache.org/jira/browse/CASSANDRA-15005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422947#comment-17422947
]
Josh McKenzie commented on CASSANDRA-15005:
---
ping [~ajs6f] - did you end up using this in production in the interim? And
would you like to pick this back up for a possible 4.1 release? I'm happy to
take on review of this for you; just let me know.
> Configurable whilelist for UDFs
> ---
>
> Key: CASSANDRA-15005
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15005
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Adam Soroka
>Priority: Low
>
> I would like to use the UDF system to distribute some simple calculations on
> values. For some use cases, this would require access only to some Java API
> classes that aren't on the (hardcoded) whitelist (e.g.
> {{java.security.MessageDigest}}). In other cases, it would require access to
> a little non-C* library code, pre-distributed to nodes by out-of-band means.
> As I understand the situation now, the whitelist for types UDFs can use is
> hardcoded in java in
> [UDFunction|[https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/cql3/functions/UDFunction.java#L99].]
> This ticket, then, is a request for a facility that would allow that list to
> be extended via some kind of deployment-time configuration. I realize that
> serious security concerns immediately arise for this kind of functionality,
> but I hope that by restricting it (only used during startup, no exposing the
> whitelist for introspection, etc.) it could be quite practical.
> I'd like very much to assist with this ticket if it is accepted. (I believe I
> have sufficient Java skill to do that, but no real familiarity with C*'s
> codebase, yet. :) )
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-16956:
--
Reviewers: Josh McKenzie
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16666) Make SSLContext creation pluggable/extensible
[ https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422919#comment-17422919 ] Stefan Miklosovic commented on CASSANDRA-1: --- No worries [~maulin.vasavada], I hope that our approach will attract and retain more developers like you. > Make SSLContext creation pluggable/extensible > - > > Key: CASSANDRA-1 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Maulin Vasavada >Assignee: Maulin Vasavada >Priority: Normal > Fix For: 4.x > > Attachments: Screenshot from 2021-09-28 10-56-24.png > > > Currently Cassandra creates the SSLContext via SSLFactory.java. SSLFactory is > a final class with static methods and not overridable. The SSLFactory loads > the keys and certs from the file based artifacts for the same. While this > works for many, in the industry where security is stricter and contextual, > this approach falls short. Many big organizations need flexibility to load > the SSL artifacts from a custom resource (like custom Key Management > Solution, HashiCorp Vault, Amazon KMS etc). While JSSE SecurityProvider > architecture allows us flexibility to build our custom mechanisms to validate > and process security artifacts, many times all we need is to build upon > Java's existing extensibility that Trust/Key Manager interfaces provide to > load keystores from various resources in the absence of any customized > requirements on the Keys/Certificate formats. > My proposal here is to make the SSLContext creation pluggable/extensible and > have the current SSLFactory.java implement an extensible interface. > I contributed a similar change that is live now in Apache Kafka (2.6.0) - > https://issues.apache.org/jira/browse/KAFKA-8890 > I can spare some time writing the pluggable interface and run by the required > reviewers. > > Created [CEP-9: Make SSLContext creation > pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable] > > > cc: [~dcapwell] [~djoshi] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic reassigned CASSANDRA-16956:
-
Assignee: Stefan Miklosovic (was: Josh McKenzie)
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch ssh-publisher-verbose-output created (now 42e66ca)
This is an automated email from the ASF dual-hosted git repository. azotcsit pushed a change to branch ssh-publisher-verbose-output in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git. at 42e66ca Update cassandra_pipeline.groovy No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16666) Make SSLContext creation pluggable/extensible
[ https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422890#comment-17422890 ] Maulin Vasavada commented on CASSANDRA-1: - Thanks [~stefan.miklosovic]. Looking forward to the Friday morning! Can we put a link to the new PR#1243 in my original PR and close it (without merge) with an appropriate comment? Btw, I am very impressed by the Apache Cassandra committer community's seriousness/dedication/attention-to-details toward the contributions (Jon/Stefan and other folks taking pain to cleanup PR branches, running/monitoring CIs for PRs etc on top of the essential thorough-reviews/suggestions on the PRs). Just mind blowing! > Make SSLContext creation pluggable/extensible > - > > Key: CASSANDRA-1 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Maulin Vasavada >Assignee: Maulin Vasavada >Priority: Normal > Fix For: 4.x > > Attachments: Screenshot from 2021-09-28 10-56-24.png > > > Currently Cassandra creates the SSLContext via SSLFactory.java. SSLFactory is > a final class with static methods and not overridable. The SSLFactory loads > the keys and certs from the file based artifacts for the same. While this > works for many, in the industry where security is stricter and contextual, > this approach falls short. Many big organizations need flexibility to load > the SSL artifacts from a custom resource (like custom Key Management > Solution, HashiCorp Vault, Amazon KMS etc). While JSSE SecurityProvider > architecture allows us flexibility to build our custom mechanisms to validate > and process security artifacts, many times all we need is to build upon > Java's existing extensibility that Trust/Key Manager interfaces provide to > load keystores from various resources in the absence of any customized > requirements on the Keys/Certificate formats. > My proposal here is to make the SSLContext creation pluggable/extensible and > have the current SSLFactory.java implement an extensible interface. > I contributed a similar change that is live now in Apache Kafka (2.6.0) - > https://issues.apache.org/jira/browse/KAFKA-8890 > I can spare some time writing the pluggable interface and run by the required > reviewers. > > Created [CEP-9: Make SSLContext creation > pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable] > > > cc: [~dcapwell] [~djoshi] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422889#comment-17422889 ] Josh McKenzie commented on CASSANDRA-12106: --- [~sumanth.pasupuleti] + [~azotcsit] - ready for another round of review. Will plan on doing doc changes once we're all happy with where we landed. > Add ability to blocklist / denylist a CQL partition so all requests are > ignored > --- > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Josh McKenzie >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blocklist / denylist > such partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17017) Add required -f / --force option to nodetool verify
[ https://issues.apache.org/jira/browse/CASSANDRA-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17017: -- Test and Documentation Plan: Trivial revision of existing test; update of option output on the tool pointing to JIRA. Status: Patch Available (was: Open) [PR|https://github.com/apache/cassandra/pull/1246] Kicked off a slow burn of dtests on CI just in case any of them relied on verify somehow and I'm not seeing it by grepping. [JDK8|https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/96/workflows/7686b1fb-267a-464c-8f49-6de00dc8baea] [JDK11|https://app.circleci.com/pipelines/github/josh-mckenzie/cassandra/96/workflows/8dbd2169-1147-4215-889e-3ef4071148d6] > Add required -f / --force option to nodetool verify > --- > > Key: CASSANDRA-17017 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17017 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > > nodetool verify has some pretty significant problems with it (see > CASSANDRA-9947). > Until such time as we do the heavy(er) lift to fix the command, we should > make it harder for people to shoot themselves in the foot with it. Adding a > required "-f" flag to it with a requisite "Do you really know what you're > doing? Check out this JIRA first" seems like it'd be the right thing to do in > the interim. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17017) Add required -f / --force option to nodetool verify
[ https://issues.apache.org/jira/browse/CASSANDRA-17017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17017: -- Change Category: Operability Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Add required -f / --force option to nodetool verify > --- > > Key: CASSANDRA-17017 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17017 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > > nodetool verify has some pretty significant problems with it (see > CASSANDRA-9947). > Until such time as we do the heavy(er) lift to fix the command, we should > make it harder for people to shoot themselves in the foot with it. Adding a > required "-f" flag to it with a requisite "Do you really know what you're > doing? Check out this JIRA first" seems like it'd be the right thing to do in > the interim. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17017) Add required -f / --force option to nodetool verify
Josh McKenzie created CASSANDRA-17017: - Summary: Add required -f / --force option to nodetool verify Key: CASSANDRA-17017 URL: https://issues.apache.org/jira/browse/CASSANDRA-17017 Project: Cassandra Issue Type: Improvement Components: Tool/nodetool Reporter: Josh McKenzie Assignee: Josh McKenzie nodetool verify has some pretty significant problems with it (see CASSANDRA-9947). Until such time as we do the heavy(er) lift to fix the command, we should make it harder for people to shoot themselves in the foot with it. Adding a required "-f" flag to it with a requisite "Do you really know what you're doing? Check out this JIRA first" seems like it'd be the right thing to do in the interim. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17016) Allow reverse iteration of resources during permissions checking
[
https://issues.apache.org/jira/browse/CASSANDRA-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-17016:
--
Test and Documentation Plan: New unit test to confirm the traversal route
works when configured. Documentation in .yaml file.
Status: Patch Available (was: Open)
[PR|https://github.com/apache/cassandra/pull/1245]
Will kick off tests on circle post review.
> Allow reverse iteration of resources during permissions checking
>
>
> Key: CASSANDRA-17016
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17016
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
>
> When we perform authz checks for AuthenticatedUser on a given resource, we
> traverse the resource hierarchy until we either find the required permission
> or we exhaust the traversal. This goes from most specific resource (i.e.
> iResource we're interested in) to the broadest (the root for that iResource
> type).
> Since permissions are a whitelist it isn't possible for a permission granted
> on a more specific resource to be negated or overridden by a grant on a
> resource lower in the hierarchy towards the root. For example, for
> DataResource, the hierarchy goes:
> {code:java}
> root -> keyspace -> table{code}
> So for instance if we:
>
> {code:java}
> GRANT ALL ON KEYSPACE ks TO admin_user;
> {code}
> It is impossible to grant admin_user any set of permissions more restrictive
> than ALL on ks.
> When a request comes in for a user with permissions on a ks for example, you
> can have a path of:
> # Validate SELECT on t1
> # Then check for SELECT on ks
> # Then check for permissions on 'root'
> These unnecessary lookups can negate some of the benefits of caching (and
> pre-warming, which another ticket is in flight for), and lead to issues on
> bounces with timeouts.
> Additionally, the permissions cache ends up storing far more entries than
> necessary, as a subsequent request for ks.t2 by user_x will go through the
> same process and create a second cache entry etc.
> So all this said - this is something we should allow operators to opt-in to;
> impact and performance is going to be highly dependent on the pattern of
> authentication granting and usage on a cluster.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17016) Allow reverse iteration of resources during permissions checking
[
https://issues.apache.org/jira/browse/CASSANDRA-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-17016:
--
Change Category: Operability
Complexity: Normal
Status: Open (was: Triage Needed)
> Allow reverse iteration of resources during permissions checking
>
>
> Key: CASSANDRA-17016
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17016
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
>
> When we perform authz checks for AuthenticatedUser on a given resource, we
> traverse the resource hierarchy until we either find the required permission
> or we exhaust the traversal. This goes from most specific resource (i.e.
> iResource we're interested in) to the broadest (the root for that iResource
> type).
> Since permissions are a whitelist it isn't possible for a permission granted
> on a more specific resource to be negated or overridden by a grant on a
> resource lower in the hierarchy towards the root. For example, for
> DataResource, the hierarchy goes:
> {code:java}
> root -> keyspace -> table{code}
> So for instance if we:
>
> {code:java}
> GRANT ALL ON KEYSPACE ks TO admin_user;
> {code}
> It is impossible to grant admin_user any set of permissions more restrictive
> than ALL on ks.
> When a request comes in for a user with permissions on a ks for example, you
> can have a path of:
> # Validate SELECT on t1
> # Then check for SELECT on ks
> # Then check for permissions on 'root'
> These unnecessary lookups can negate some of the benefits of caching (and
> pre-warming, which another ticket is in flight for), and lead to issues on
> bounces with timeouts.
> Additionally, the permissions cache ends up storing far more entries than
> necessary, as a subsequent request for ks.t2 by user_x will go through the
> same process and create a second cache entry etc.
> So all this said - this is something we should allow operators to opt-in to;
> impact and performance is going to be highly dependent on the pattern of
> authentication granting and usage on a cluster.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17016) Allow reverse iteration of resources during permissions checking
Josh McKenzie created CASSANDRA-17016:
-
Summary: Allow reverse iteration of resources during permissions
checking
Key: CASSANDRA-17016
URL: https://issues.apache.org/jira/browse/CASSANDRA-17016
Project: Cassandra
Issue Type: Improvement
Components: Feature/Authorization
Reporter: Josh McKenzie
Assignee: Josh McKenzie
When we perform authz checks for AuthenticatedUser on a given resource, we
traverse the resource hierarchy until we either find the required permission or
we exhaust the traversal. This goes from most specific resource (i.e. iResource
we're interested in) to the broadest (the root for that iResource type).
Since permissions are a whitelist it isn't possible for a permission granted on
a more specific resource to be negated or overridden by a grant on a resource
lower in the hierarchy towards the root. For example, for DataResource, the
hierarchy goes:
{code:java}
root -> keyspace -> table{code}
So for instance if we:
{code:java}
GRANT ALL ON KEYSPACE ks TO admin_user;
{code}
It is impossible to grant admin_user any set of permissions more restrictive
than ALL on ks.
When a request comes in for a user with permissions on a ks for example, you
can have a path of:
# Validate SELECT on t1
# Then check for SELECT on ks
# Then check for permissions on 'root'
These unnecessary lookups can negate some of the benefits of caching (and
pre-warming, which another ticket is in flight for), and lead to issues on
bounces with timeouts.
Additionally, the permissions cache ends up storing far more entries than
necessary, as a subsequent request for ks.t2 by user_x will go through the same
process and create a second cache entry etc.
So all this said - this is something we should allow operators to opt-in to;
impact and performance is going to be highly dependent on the pattern of
authentication granting and usage on a cluster.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16334) Replica failure causes timeout on multi-DC write
[
https://issues.apache.org/jira/browse/CASSANDRA-16334?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Sorokoumov reassigned CASSANDRA-16334:
Assignee: Aleksandr Sorokoumov
> Replica failure causes timeout on multi-DC write
>
>
> Key: CASSANDRA-16334
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16334
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination, Messaging/Internode
>Reporter: Paulo Motta
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
>
> Inserting a mutation larger than {{max_mutation_size_in_kb}} correctly throws
> a write error on a single DC keyspace with RF=3:
> {noformat}
> cassandra.WriteFailure: Error from server: code=1500 [Replica(s) failed to
> execute write] message="Operation failed - received 0 responses and 3
> failures: UNKNOWN from /127.0.0.3:7000, UNKNOWN from /127.0.0.2:7000, UNKNOWN
> from /127.0.0.1:7000" info={'consistency': 'LOCAL_ONE', 'required_responses':
> 1, 'received_responses': 0, 'failures': 3}
> {noformat}
> The same insert wrongly causes a timeout on a keyspace with 2 dcs (RF=3 each):
> {noformat}
> cassandra.WriteTimeout: Error from server: code=1100 [Coordinator node timed
> out waiting for replica nodes' responses] message="Operation timed out -
> received only 0 responses." info={'consistency': 'LOCAL_ONE',
> 'required_responses': 1, 'received_responses': 0}
> {noformat}
> Reproduction steps:
> {noformat}
> # Setup cluster
> ccm create -n 3:3 test
> for i in {1..6}; do echo 'max_mutation_size_in_kb: 1000' >>
> ~/.ccm/test/node$i/conf/cassandra.yaml; done
> ccm start
> # Create schema
> ccm node1 cqlsh
> CREATE KEYSPACE test WITH replication = {'class': 'NetworkTopologyStrategy',
> 'dc1': 3, 'dc2': 3};
> CREATE TABLE test.test (key int PRIMARY KEY, val blob);
> exit;
> # Insert data
> python
> from cassandra.cluster import Cluster
> session = cluster.connect('test')
> blob = f = open("2mbBlob", "rb").read().hex()
> session.execute("INSERT INTO test (key, val) VALUES (1, textAsBlob('" + blob
> + "'))")
> {noformat}
> Reproduced in 3.11, trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16928) InetAddressAndPort to extend InetAddress
[ https://issues.apache.org/jira/browse/CASSANDRA-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16928: --- Status: Ready to Commit (was: Review In Progress) > InetAddressAndPort to extend InetAddress > > > Key: CASSANDRA-16928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16928 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Logically InetAddressAndPort encapsulates the same information as a simple > InetAddress. Importantly, InetAddressAndPort cannot easily be unpicked from > its dependencies that prohibit it being shared between classloaders. So our > in-jvm dtest API instead uses SocketAddress. To support a clean and efficient > Streaming API we need to be able to treat InetAddressAndPort as such a > cross-classloader type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16926) Mockable FileSystem
[ https://issues.apache.org/jira/browse/CASSANDRA-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16926: --- Status: Ready to Commit (was: Review In Progress) > Mockable FileSystem > --- > > Key: CASSANDRA-16926 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16926 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative file system > implementations so that execution may be consistent between hosts. This > ticket introduces a new File API backed by Path objects, and all File usages > are ported to it. This necessitates the migration away from > RandomAccessReader, but otherwise the change is mostly straight-forward, if > quite expansive. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17013) CEP-10: dtest-api improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17013: --- Status: Review In Progress (was: Patch Available) > CEP-10: dtest-api improvements > -- > > Key: CASSANDRA-17013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17013 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Introduce new dtest-api features to support CEP-10, including byte weaving > support, support for Path objects specifying location, and other minor > ergonomic improvements -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16927) Mockable Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16927: --- Status: Ready to Commit (was: Review In Progress) > Mockable Streaming > -- > > Key: CASSANDRA-16927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16927 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative streaming > implementations, so that execution may be controlled. The ticket encompasses > two sub-tickets: firstly we make InetAddressAndPort extend InetAddress so > that we may 2) introduce a cross-classloader API for establishing a streaming > connection and sending data over it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17013) CEP-10: dtest-api improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17013: --- Status: Ready to Commit (was: Review In Progress) > CEP-10: dtest-api improvements > -- > > Key: CASSANDRA-17013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17013 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Introduce new dtest-api features to support CEP-10, including byte weaving > support, support for Path objects specifying location, and other minor > ergonomic improvements -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16925) Task Execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16925: --- Status: Ready to Commit (was: Review In Progress) > Task Execution > -- > > Key: CASSANDRA-16925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16925 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative executor > implementations that may be externally controlled. This ticket introduces an > ExecutorFactory, and all executor creation is migrated to this facility. The > codebase’s use of executors is inconsistent, and relatedly there are > divergent Future APIs in use, with Netty, Guava and Java APIs in use in > various places. To improve consistency we introduce our own Future API that > implements all of the above, as well as providing other additional ergonomic > improvements. We introduce a new ExecutorPlus API that extends Executor to > supply this extended Future API. All executors and futures are ported to > these new APIs to improve coherency in the codebase, and to support > consistently mocking these implementations. > DebuggableThreadPoolExecutor and its hierarchy is replaced with > ThreadPoolExecutorBase, ThreadPoolExecutorPlus and extending classes with > consistent names, implementations and configurability. > Additionally the ExecutorFactory takes over the allocation of new threads, > and provides a new InfiniteLoopExecutorfacility for common paradigms. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17013) CEP-10: dtest-api improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17013: --- Test and Documentation Plan: none required Status: Patch Available (was: Open) > CEP-10: dtest-api improvements > -- > > Key: CASSANDRA-17013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17013 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Introduce new dtest-api features to support CEP-10, including byte weaving > support, support for Path objects specifying location, and other minor > ergonomic improvements -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16922) CEP-10: Major Prerequisites (Phase 1)
[ https://issues.apache.org/jira/browse/CASSANDRA-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16922: --- Status: Ready to Commit (was: Review In Progress) > CEP-10: Major Prerequisites (Phase 1) > - > > Key: CASSANDRA-16922 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16922 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn >Reporter: Benedict Elliott Smith >Priority: Normal > > This is a tracking Jira for major pre-requisite system refactors for CEP-10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17013) CEP-10: dtest-api improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17013: --- Change Category: Quality Assurance Complexity: Normal Reviewers: Sam Tunnicliffe Assignee: Benedict Elliott Smith Status: Open (was: Triage Needed) > CEP-10: dtest-api improvements > -- > > Key: CASSANDRA-17013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17013 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Introduce new dtest-api features to support CEP-10, including byte weaving > support, support for Path objects specifying location, and other minor > ergonomic improvements -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16922) CEP-10: Major Prerequisites (Phase 1)
[ https://issues.apache.org/jira/browse/CASSANDRA-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16922: --- Reviewers: Aleksey Yeschenko, Caleb Rackliffe, Sam Tunnicliffe Status: Review In Progress (was: Patch Available) > CEP-10: Major Prerequisites (Phase 1) > - > > Key: CASSANDRA-16922 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16922 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn >Reporter: Benedict Elliott Smith >Priority: Normal > > This is a tracking Jira for major pre-requisite system refactors for CEP-10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16922) CEP-10: Major Prerequisites (Phase 1)
[ https://issues.apache.org/jira/browse/CASSANDRA-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16922: --- Test and Documentation Plan: Specified by each sub-task Status: Patch Available (was: In Progress) > CEP-10: Major Prerequisites (Phase 1) > - > > Key: CASSANDRA-16922 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16922 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn >Reporter: Benedict Elliott Smith >Priority: Normal > > This is a tracking Jira for major pre-requisite system refactors for CEP-10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17008) CEP-10: Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17008: --- Status: Review In Progress (was: Patch Available) > CEP-10: Simulator > - > > Key: CASSANDRA-17008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17008 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn, Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > This ticket introduces the Simulator facility proposed by CEP-10, including > mock implementations of: time, blocking concurrency primitives, executors, > messaging, object monitors, identity hashcode, and byte weaving class loaders > to ensure these mock implementations are utilised in place of their originals. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17008) CEP-10: Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17008: --- Test and Documentation Plan: Introduces new test infrastructure, and linearizability verification tests Status: Patch Available (was: In Progress) > CEP-10: Simulator > - > > Key: CASSANDRA-17008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17008 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn, Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > This ticket introduces the Simulator facility proposed by CEP-10, including > mock implementations of: time, blocking concurrency primitives, executors, > messaging, object monitors, identity hashcode, and byte weaving class loaders > to ensure these mock implementations are utilised in place of their originals. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17008) CEP-10: Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422857#comment-17422857 ] Benedict Elliott Smith commented on CASSANDRA-17008: [Patch|https://github.com/belliottsmith/cassandra/tree/17008-trunk] [Pull Request|https://github.com/apache/cassandra/pull/1244] > CEP-10: Simulator > - > > Key: CASSANDRA-17008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17008 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict Elliott Smith >Priority: Normal > > This ticket introduces the Simulator facility proposed by CEP-10, including > mock implementations of: time, blocking concurrency primitives, executors, > messaging, object monitors, identity hashcode, and byte weaving class loaders > to ensure these mock implementations are utilised in place of their originals. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17008) CEP-10: Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17008: --- Change Category: Quality Assurance Complexity: Challenging Component/s: Test/dtest/java Test/burn Fix Version/s: 4.x Reviewers: Sam Tunnicliffe Assignee: Benedict Elliott Smith Status: Open (was: Triage Needed) > CEP-10: Simulator > - > > Key: CASSANDRA-17008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17008 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn, Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > This ticket introduces the Simulator facility proposed by CEP-10, including > mock implementations of: time, blocking concurrency primitives, executors, > messaging, object monitors, identity hashcode, and byte weaving class loaders > to ensure these mock implementations are utilised in place of their originals. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17008) CEP-10: Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17008: --- Summary: CEP-10: Simulator (was: CEP-10: Simulator (Phase 4)) > CEP-10: Simulator > - > > Key: CASSANDRA-17008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17008 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict Elliott Smith >Priority: Normal > > This ticket tracks work related to the final phase of CEP-10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17008) CEP-10: Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-17008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17008: --- Description: This ticket introduces the Simulator facility proposed by CEP-10, including mock implementations of: time, blocking concurrency primitives, executors, messaging, object monitors, identity hashcode, and byte weaving class loaders to ensure these mock implementations are utilised in place of their originals. (was: This ticket tracks work related to the final phase of CEP-10) > CEP-10: Simulator > - > > Key: CASSANDRA-17008 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17008 > Project: Cassandra > Issue Type: Improvement >Reporter: Benedict Elliott Smith >Priority: Normal > > This ticket introduces the Simulator facility proposed by CEP-10, including > mock implementations of: time, blocking concurrency primitives, executors, > messaging, object monitors, identity hashcode, and byte weaving class loaders > to ensure these mock implementations are utilised in place of their originals. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422762#comment-17422762
]
Brandon Williams commented on CASSANDRA-16976:
--
Thanks for digging into this further, this is revealing.
bq. Even though the current logic seems to be reasonable, "unit" name is
confusing.
It is, but I can't think of something better that isn't significantly more
verbose, and though it can be misinterpreted (as I certainly did) "unit" is
correct. I think we'll just need to make sure the documentation calls this out
later.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422752#comment-17422752
]
Aleksei Zotov edited comment on CASSANDRA-16976 at 9/30/21, 12:51 PM:
--
I dug deeper to unit stuff and I feel the current unit logic makes sense.
Possible units are: bytes, ranges, keys. They are applicable to different
compaction types. So "bytes" unit is not a measure of size (like byte, KB, MB,
etc) for progress/total, but rather is a sing that the numbers in
progress/total columns are measured in size (not a number of keys or ranges).
Basically that's why there is this condition that we need to convert
progress/total to human readable size format only if "size measurement" is
applicable:
boolean toFileSize = humanReadable && Unit.isFileSize(unit);
Even though the current logic seems to be reasonable, "unit" name is
confusing. Maybe we can rename "unit" (both in nodetool and the VT)... For
example: "compaction unit" - it does not clarify a lot what it is, however, it
highlights it is not just a size unit.
WDYT guys?
was (Author: azotcsit):
I dug deeper to unit stuff and I feel the current unit logic makes sense.
Possible units are: bytes, ranges, keys. They are applicable to different
compaction types. So "bytes" unit is not a measure of size (like byte, KB, MB,
etc) for progress/total, but rather is a sing that the numbers in
progress/total columns are measured in size (not a number of keys or ranges).
Basically that's why there is this condition that we need to convert
progress/total to human readable size format only if "size measurement" is
applicable:
boolean toFileSize = humanReadable && Unit.isFileSize(unit);
Even though the current logic seems to be reasonable, "unit" name is confusing.
Maybe we can rename "unit"... For example: "compaction unit" - it does not
clarify a lot what it is, however, it highlights it is not just a size unit.
WDYT guys?
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422752#comment-17422752
]
Aleksei Zotov edited comment on CASSANDRA-16976 at 9/30/21, 12:51 PM:
--
I dug deeper to unit stuff and I feel the current unit logic makes sense.
Possible units are: bytes, ranges, keys. They are applicable to different
compaction types. So "bytes" unit is not a measure of size (like byte, KB, MB,
etc) for progress/total, but rather is a sing that the numbers in
progress/total columns are measured in size (not a number of keys or ranges).
Basically that's why there is this condition that we need to convert
progress/total to human readable size format only if "size measurement" is
applicable:
{code:java}
boolean toFileSize = humanReadable && Unit.isFileSize(unit);{code}
Even though the current logic seems to be reasonable, "unit" name is
confusing. Maybe we can rename "unit" (both in nodetool and the VT)... For
example: "compaction unit" - it does not clarify a lot what it is, however, it
highlights it is not just a size unit.
WDYT guys?
was (Author: azotcsit):
I dug deeper to unit stuff and I feel the current unit logic makes sense.
Possible units are: bytes, ranges, keys. They are applicable to different
compaction types. So "bytes" unit is not a measure of size (like byte, KB, MB,
etc) for progress/total, but rather is a sing that the numbers in
progress/total columns are measured in size (not a number of keys or ranges).
Basically that's why there is this condition that we need to convert
progress/total to human readable size format only if "size measurement" is
applicable:
boolean toFileSize = humanReadable && Unit.isFileSize(unit);
Even though the current logic seems to be reasonable, "unit" name is
confusing. Maybe we can rename "unit" (both in nodetool and the VT)... For
example: "compaction unit" - it does not clarify a lot what it is, however, it
highlights it is not just a size unit.
WDYT guys?
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422752#comment-17422752
]
Aleksei Zotov commented on CASSANDRA-16976:
---
I dug deeper to unit stuff and I feel the current unit logic makes sense.
Possible units are: bytes, ranges, keys. They are applicable to different
compaction types. So "bytes" unit is not a measure of size (like byte, KB, MB,
etc) for progress/total, but rather is a sing that the numbers in
progress/total columns are measured in size (not a number of keys or ranges).
Basically that's why there is this condition that we need to convert
progress/total to human readable size format only if "size measurement" is
applicable:
boolean toFileSize = humanReadable && Unit.isFileSize(unit);
Even though the current logic seems to be reasonable, "unit" name is confusing.
Maybe we can rename "unit"... For example: "compaction unit" - it does not
clarify a lot what it is, however, it highlights it is not just a size unit.
WDYT guys?
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422749#comment-17422749
]
Brandon Williams commented on CASSANDRA-16976:
--
bq. set unit to "N/A" in case of "human-readable" output
This is the only way it makes sense to me, to meet the 'human-readable'
contract, units are going to be variable.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422732#comment-17422732
]
Stefan Miklosovic commented on CASSANDRA-16976:
---
Ok, so
1) no order on rows
2) no reshufling of columns - I checked how it is built on tables and they seem
to be sorted by "natural order" which is "aplhabetically" on Strings.
So the last question is about units.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (84ec1dc -> c0916d6)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 84ec1dc Correct the internode message timestamp if sending node has wrapped new 3e6faca Do not release new SSTables in offline transactions new dcc9549 Merge branch 'cassandra-3.0' into cassandra-3.11 new c0916d6 Merge branch 'cassandra-3.11' into cassandra-4.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/db/compaction/CompactionTask.java| 83 ++ .../db/compaction/CompactionTaskTest.java | 34 + 3 files changed, 74 insertions(+), 44 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16975) CompactionTask#runMayThrow should not release new SSTables for offline transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16975:
--
Fix Version/s: (was: 4.0.x)
(was: 3.11.x)
(was: 3.0.x)
4.1
4.0.2
3.11.12
3.0.26
Since Version: NA
Source Control Link:
https://github.com/apache/cassandra/commit/3e6faca572a5ca1de5906b39b8c0a6bf4deb40e9
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> CompactionTask#runMayThrow should not release new SSTables for offline
> transactions
> ---
>
> Key: CASSANDRA-16975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16975
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1
>
>
> Right now, {{CompactionTask#runMayThrow}} releases new SSTables for offline
> transactions
> ([code|https://github.com/apache/cassandra/blob/f7c71f65c000c2c3ef7df1b034b8fdd822a396d8/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L227-L230]).
> This change was added in CASSANDRA-8962, prior to the introduction of
> lifecycle transactions in CASSANDRA-8568. I suspect that this behavior might
> be undesired and could have just fallen through the cracks.
> To my knowledge, this code does not cause any known bugs solely because
> in-tree tools do not access the SSTables they produce before exiting.
> However, if someone is to write, say, offline compaction daemon, it might
> break on subsequent compactions because newly created SSTables will be
> released.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16975) CompactionTask#runMayThrow should not release new SSTables for offline transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422731#comment-17422731
]
Andres de la Peña commented on CASSANDRA-16975:
---
No worries, too many branches :)
Committed to 3.0 as
[3e6faca572a5ca1de5906b39b8c0a6bf4deb40e9|https://github.com/apache/cassandra/commit/3e6faca572a5ca1de5906b39b8c0a6bf4deb40e9]
and merged to
[3.11|https://github.com/apache/cassandra/commit/dcc95492e9b05e1b8ceb3af332d5c0989c7272b0],
[4.0|https://github.com/apache/cassandra/commit/c0916d67be25c700e5de93242afa7244c7fbbb02]
and
[trunk|https://github.com/apache/cassandra/commit/b6e400d0c3c3f74d17261caaa6388e6c70df322f].
> CompactionTask#runMayThrow should not release new SSTables for offline
> transactions
> ---
>
> Key: CASSANDRA-16975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16975
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Right now, {{CompactionTask#runMayThrow}} releases new SSTables for offline
> transactions
> ([code|https://github.com/apache/cassandra/blob/f7c71f65c000c2c3ef7df1b034b8fdd822a396d8/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L227-L230]).
> This change was added in CASSANDRA-8962, prior to the introduction of
> lifecycle transactions in CASSANDRA-8568. I suspect that this behavior might
> be undesired and could have just fallen through the cracks.
> To my knowledge, this code does not cause any known bugs solely because
> in-tree tools do not access the SSTables they produce before exiting.
> However, if someone is to write, say, offline compaction daemon, it might
> break on subsequent compactions because newly created SSTables will be
> released.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422730#comment-17422730
]
Brandon Williams commented on CASSANDRA-16976:
--
bq. I think it should be sorted and it should have same order I mean ... why
not? wdyt Brandon Williams?
As Aleksei pointed out, the VT output is sorted by the PK and clustering
columns..._by default_. Depending on how the user constructs their query, this
could be different, right? I don't think we should get too hung up on making
them match since nodetool has no equivalent flexibility. Nodetool will
eventually give way to VTs as the primary management tool, and VTs should act
like normal CQL tables without any special casing.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (9dd0e55 -> b6e400d)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 9dd0e55 Merge branch 'cassandra-4.0' into trunk new 3e6faca Do not release new SSTables in offline transactions new dcc9549 Merge branch 'cassandra-3.0' into cassandra-3.11 new c0916d6 Merge branch 'cassandra-3.11' into cassandra-4.0 new b6e400d Merge branch 'cassandra-4.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/db/compaction/CompactionTask.java| 83 ++ .../db/compaction/CompactionTaskTest.java | 34 + 3 files changed, 74 insertions(+), 44 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit b6e400d0c3c3f74d17261caaa6388e6c70df322f
Merge: 9dd0e55 c0916d6
Author: Andrés de la Peña
AuthorDate: Thu Sep 30 13:14:45 2021 +0100
Merge branch 'cassandra-4.0' into trunk
# Conflicts:
# src/java/org/apache/cassandra/db/compaction/CompactionTask.java
CHANGES.txt| 1 +
.../cassandra/db/compaction/CompactionTask.java| 83 ++
.../db/compaction/CompactionTaskTest.java | 34 +
3 files changed, 74 insertions(+), 44 deletions(-)
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 9f654e2,74bf246..fe61ab6
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@@ -228,53 -220,48 +228,48 @@@ public class CompactionTask extends Abs
}
if (transaction.isOffline())
+ return;
+
+ // log a bunch of statistics about the result and save to system
table compaction_history
-long durationInNano = System.nanoTime() - start;
++long durationInNano = nanoTime() - start;
+ long dTime = TimeUnit.NANOSECONDS.toMillis(durationInNano);
+ long startsize = inputSizeBytes;
+ long endsize = SSTableReader.getTotalBytes(newSStables);
+ double ratio = (double) endsize / (double) startsize;
+
+ StringBuilder newSSTableNames = new StringBuilder();
+ for (SSTableReader reader : newSStables)
+
newSSTableNames.append(reader.descriptor.baseFilename()).append(",");
+ long totalSourceRows = 0;
+ for (int i = 0; i < mergedRowCounts.length; i++)
+ totalSourceRows += mergedRowCounts[i] * (i + 1);
+
+ String mergeSummary =
updateCompactionHistory(cfs.keyspace.getName(), cfs.getTableName(),
mergedRowCounts, startsize, endsize);
+
+ logger.info(String.format("Compacted (%s) %d sstables to [%s] to
level=%d. %s to %s (~%d%% of original) in %,dms. Read Throughput = %s, Write
Throughput = %s, Row Throughput = ~%,d/s. %,d total partitions merged to %,d.
Partition merge counts were {%s}",
+taskId,
+transaction.originals().size(),
+newSSTableNames.toString(),
+getLevel(),
+
FBUtilities.prettyPrintMemory(startsize),
+FBUtilities.prettyPrintMemory(endsize),
+(int) (ratio * 100),
+dTime,
+
FBUtilities.prettyPrintMemoryPerSecond(startsize, durationInNano),
+
FBUtilities.prettyPrintMemoryPerSecond(endsize, durationInNano),
+(int) totalSourceCQLRows /
(TimeUnit.NANOSECONDS.toSeconds(durationInNano) + 1),
+totalSourceRows,
+totalKeysWritten,
+mergeSummary));
+ if (logger.isTraceEnabled())
{
- Refs.release(Refs.selfRefs(newSStables));
+ logger.trace("CF Total Bytes Compacted: {}",
FBUtilities.prettyPrintMemory(CompactionTask.addToTotalBytesCompacted(endsize)));
+ logger.trace("Actual #keys: {}, Estimated #keys:{}, Err%:
{}", totalKeysWritten, estimatedKeys, ((double)(totalKeysWritten -
estimatedKeys)/totalKeysWritten));
}
- else
- {
- // log a bunch of statistics about the result and save to
system table compaction_history
-
- long durationInNano = nanoTime() - start;
- long dTime = TimeUnit.NANOSECONDS.toMillis(durationInNano);
- long startsize = inputSizeBytes;
- long endsize = SSTableReader.getTotalBytes(newSStables);
- double ratio = (double) endsize / (double) startsize;
-
- StringBuilder newSSTableNames = new StringBuilder();
- for (SSTableReader reader : newSStables)
-
newSSTableNames.append(reader.descriptor.baseFilename()).append(",");
- long totalSourceRows = 0;
- for (int i = 0; i < mergedRowCounts.length; i++)
- totalSourceRows += mergedRowCounts[i] * (i + 1);
-
- String mergeSummary =
updateCompactionHistory(cfs.keyspace.getName(), cfs.getTableName(),
mergedRowCounts, startsize, endsize);
-
- logger.info(String.format
[cassandra] branch cassandra-3.11 updated (7c067b6 -> dcc9549)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7c067b6 Add indication in cassandra.yaml that rpc timeouts going too high will cause memory build up new 3e6faca Do not release new SSTables in offline transactions new dcc9549 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/db/compaction/CompactionTask.java| 81 +-- .../db/compaction/CompactionTaskTest.java | 91 ++ 3 files changed, 130 insertions(+), 43 deletions(-) create mode 100644 test/unit/org/apache/cassandra/db/compaction/CompactionTaskTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Do not release new SSTables in offline transactions
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 3e6faca Do not release new SSTables in offline transactions
3e6faca is described below
commit 3e6faca572a5ca1de5906b39b8c0a6bf4deb40e9
Author: Aleksandr Sorokoumov
AuthorDate: Thu Sep 30 13:01:02 2021 +0100
Do not release new SSTables in offline transactions
patch by Aleksandr Sorokoumov; reviewed by Andrés de la Peña and Branimir
Lambov for CASSANDRA-16975
---
CHANGES.txt| 1 +
.../cassandra/db/compaction/CompactionTask.java| 20 ++---
.../db/compaction/CompactionTaskTest.java | 91 ++
3 files changed, 100 insertions(+), 12 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index cac42fb..6ef52e4 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.26:
+ * Do not release new SSTables in offline transactions (CASSANDRA-16975)
* ArrayIndexOutOfBoundsException in FunctionResource#fromName
(CASSANDRA-16977, CASSANDRA-16995)
* CVE-2015-0886 Security vulnerability in jbcrypt is addressed
(CASSANDRA-9384)
* Avoid useless SSTable reads during single partition queries
(CASSANDRA-16944)
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 3437de7..d29d5e6 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@ -228,18 +228,14 @@ public class CompactionTask extends AbstractCompactionTask
newSSTableNames.append(reader.descriptor.baseFilename()).append(",");
if (offline)
-{
-Refs.release(Refs.selfRefs(newSStables));
-}
-else
-{
-double mbps = dTime > 0 ? (double) endsize / (1024 * 1024) /
((double) dTime / 1000) : 0;
-Summary mergeSummary =
updateCompactionHistory(cfs.keyspace.getName(), cfs.getColumnFamilyName(),
mergedRowCounts, startsize, endsize);
-logger.debug(String.format("Compacted (%s) %d sstables to [%s]
to level=%d. %,d bytes to %,d (~%d%% of original) in %,dms = %fMB/s. %,d
total partitions merged to %,d. Partition merge counts were {%s}",
- taskId,
transaction.originals().size(), newSSTableNames.toString(), getLevel(),
startsize, endsize, (int) (ratio * 100), dTime, mbps,
mergeSummary.totalSourceRows, totalKeysWritten, mergeSummary.partitionMerge));
-logger.trace(String.format("CF Total Bytes Compacted: %,d",
CompactionTask.addToTotalBytesCompacted(endsize)));
-logger.trace("Actual #keys: {}, Estimated #keys:{}, Err%: {}",
totalKeysWritten, estimatedKeys, ((double)(totalKeysWritten -
estimatedKeys)/totalKeysWritten));
-}
+return;
+
+double mbps = dTime > 0 ? (double) endsize / (1024 * 1024) /
((double) dTime / 1000) : 0;
+Summary mergeSummary =
updateCompactionHistory(cfs.keyspace.getName(), cfs.getColumnFamilyName(),
mergedRowCounts, startsize, endsize);
+logger.debug(String.format("Compacted (%s) %d sstables to [%s] to
level=%d. %,d bytes to %,d (~%d%% of original) in %,dms = %fMB/s. %,d total
partitions merged to %,d. Partition merge counts were {%s}",
+ taskId, transaction.originals().size(),
newSSTableNames.toString(), getLevel(), startsize, endsize, (int) (ratio *
100), dTime, mbps, mergeSummary.totalSourceRows, totalKeysWritten,
mergeSummary.partitionMerge));
+logger.trace(String.format("CF Total Bytes Compacted: %,d",
CompactionTask.addToTotalBytesCompacted(endsize)));
+logger.trace("Actual #keys: {}, Estimated #keys:{}, Err%: {}",
totalKeysWritten, estimatedKeys, ((double)(totalKeysWritten -
estimatedKeys)/totalKeysWritten));
}
}
diff --git
a/test/unit/org/apache/cassandra/db/compaction/CompactionTaskTest.java
b/test/unit/org/apache/cassandra/db/compaction/CompactionTaskTest.java
new file mode 100644
index 000..5602a08
--- /dev/null
+++ b/test/unit/org/apache/cassandra/db/compaction/CompactionTaskTest.java
@@ -0,0 +1,91 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit dcc95492e9b05e1b8ceb3af332d5c0989c7272b0
Merge: 7c067b6 3e6faca
Author: Andrés de la Peña
AuthorDate: Thu Sep 30 13:07:26 2021 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
# Conflicts:
# CHANGES.txt
# src/java/org/apache/cassandra/db/compaction/CompactionTask.java
CHANGES.txt| 1 +
.../cassandra/db/compaction/CompactionTask.java| 81 +--
.../db/compaction/CompactionTaskTest.java | 91 ++
3 files changed, 130 insertions(+), 43 deletions(-)
diff --cc CHANGES.txt
index 406efbc,6ef52e4..84c5bcf
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,15 -1,5 +1,16 @@@
-3.0.26:
+3.11.12
+ * Add key validation to ssstablescrub (CASSANDRA-16969)
+ * Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851)
+ * Include SASI components to snapshots (CASSANDRA-15134)
+ * Make assassinate more resilient to missing tokens (CASSANDRA-16847)
+ * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies
(CASSANDRA-16854)
+ * Validate SASI tokenizer options before adding index to schema
(CASSANDRA-15135)
+ * Fixup scrub output when no data post-scrub and clear up old use of row,
which really means partition (CASSANDRA-16835)
+ * Fix ant-junit dependency issue (CASSANDRA-16827)
+ * Reduce thread contention in CommitLogSegment and HintsBuffer
(CASSANDRA-16072)
+ * Avoid sending CDC column if not enabled (CASSANDRA-16770)
+Merged from 3.0:
+ * Do not release new SSTables in offline transactions (CASSANDRA-16975)
* ArrayIndexOutOfBoundsException in FunctionResource#fromName
(CASSANDRA-16977, CASSANDRA-16995)
* CVE-2015-0886 Security vulnerability in jbcrypt is addressed
(CASSANDRA-9384)
* Avoid useless SSTable reads during single partition queries
(CASSANDRA-16944)
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 2efcd11,d29d5e6..b990020
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@@ -233,50 -217,25 +233,45 @@@ public class CompactionTask extends Abs
}
}
+if (transaction.isOffline())
- {
- Refs.release(Refs.selfRefs(newSStables));
- }
- else
- {
- // log a bunch of statistics about the result and save to
system table compaction_history
-
- long durationInNano = System.nanoTime() - start;
- long dTime = TimeUnit.NANOSECONDS.toMillis(durationInNano);
- long startsize = inputSizeBytes;
- long endsize = SSTableReader.getTotalBytes(newSStables);
- double ratio = (double) endsize / (double) startsize;
-
- StringBuilder newSSTableNames = new StringBuilder();
- for (SSTableReader reader : newSStables)
-
newSSTableNames.append(reader.descriptor.baseFilename()).append(",");
- long totalSourceRows = 0;
- for (int i = 0; i < mergedRowCounts.length; i++)
- totalSourceRows += mergedRowCounts[i] * (i + 1);
-
- String mergeSummary =
updateCompactionHistory(cfs.keyspace.getName(), cfs.getTableName(),
mergedRowCounts, startsize, endsize);
- logger.debug(String.format("Compacted (%s) %d sstables to
[%s] to level=%d. %s to %s (~%d%% of original) in %,dms. Read Throughput =
%s, Write Throughput = %s, Row Throughput = ~%,d/s. %,d total partitions
merged to %,d. Partition merge counts were {%s}",
-taskId,
-transaction.originals().size(),
-newSSTableNames.toString(),
-getLevel(),
-
FBUtilities.prettyPrintMemory(startsize),
-
FBUtilities.prettyPrintMemory(endsize),
-(int) (ratio * 100),
-dTime,
-
FBUtilities.prettyPrintMemoryPerSecond(startsize, durationInNano),
-
FBUtilities.prettyPrintMemoryPerSecond(endsize, durationInNano),
-(int) totalSourceCQLRows /
(TimeUnit.NANOSECONDS.toSeconds(durationInNano) + 1),
-totalSourceRows,
-totalKeysWritten,
-mergeSummary));
- logger.trace("CF Total Bytes Compacted: {}",
[cassandra] 01/01: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit c0916d67be25c700e5de93242afa7244c7fbbb02
Merge: 84ec1dc dcc9549
Author: Andrés de la Peña
AuthorDate: Thu Sep 30 13:11:33 2021 +0100
Merge branch 'cassandra-3.11' into cassandra-4.0
# Conflicts:
# src/java/org/apache/cassandra/db/compaction/CompactionTask.java
# test/unit/org/apache/cassandra/db/compaction/CompactionTaskTest.java
CHANGES.txt| 1 +
.../cassandra/db/compaction/CompactionTask.java| 83 ++
.../db/compaction/CompactionTaskTest.java | 34 +
3 files changed, 74 insertions(+), 44 deletions(-)
diff --cc CHANGES.txt
index e613c80,84c5bcf..4c487cc
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,22 -1,16 +1,23 @@@
-3.11.12
+4.0.2
+ * Correct the internode message timestamp if sending node has wrapped
(CASSANDRA-16997)
+ * Avoid race causing us to return null in RangesAtEndpoint (CASSANDRA-16965)
+ * Avoid rewriting all sstables during cleanup when transient replication is
enabled (CASSANDRA-16966)
+ * Prevent CQLSH from failure on Python 3.10 (CASSANDRA-16987)
+ * Avoid trying to acquire 0 permits from the rate limiter when taking
snapshot (CASSANDRA-16872)
+ * Upgrade Caffeine to 2.5.6 (CASSANDRA-15153)
+ * Include SASI components to snapshots (CASSANDRA-15134)
+ * Fix missed wait latencies in the output of `nodetool tpstats -F`
(CASSANDRA-16938)
+ * Remove all the state pollution between tests in SSTableReaderTest
(CASSANDRA-16888)
+ * Delay auth setup until after gossip has settled to avoid unavailables on
startup (CASSANDRA-16783)
+ * Fix clustering order logic in CREATE MATERIALIZED VIEW (CASSANDRA-16898)
+ * org.apache.cassandra.db.rows.ArrayCell#unsharedHeapSizeExcludingData
includes data twice (CASSANDRA-16900)
+ * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies
(CASSANDRA-16854)
+Merged from 3.11:
* Add key validation to ssstablescrub (CASSANDRA-16969)
* Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851)
- * Include SASI components to snapshots (CASSANDRA-15134)
* Make assassinate more resilient to missing tokens (CASSANDRA-16847)
- * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies
(CASSANDRA-16854)
- * Validate SASI tokenizer options before adding index to schema
(CASSANDRA-15135)
- * Fixup scrub output when no data post-scrub and clear up old use of row,
which really means partition (CASSANDRA-16835)
- * Fix ant-junit dependency issue (CASSANDRA-16827)
- * Reduce thread contention in CommitLogSegment and HintsBuffer
(CASSANDRA-16072)
- * Avoid sending CDC column if not enabled (CASSANDRA-16770)
Merged from 3.0:
+ * Do not release new SSTables in offline transactions (CASSANDRA-16975)
* ArrayIndexOutOfBoundsException in FunctionResource#fromName
(CASSANDRA-16977, CASSANDRA-16995)
* CVE-2015-0886 Security vulnerability in jbcrypt is addressed
(CASSANDRA-9384)
* Avoid useless SSTable reads during single partition queries
(CASSANDRA-16944)
diff --cc src/java/org/apache/cassandra/db/compaction/CompactionTask.java
index 13c9725,b990020..74bf246
--- a/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionTask.java
@@@ -220,53 -234,44 +220,48 @@@ public class CompactionTask extends Abs
}
if (transaction.isOffline())
+ return;
+
+ // log a bunch of statistics about the result and save to system
table compaction_history
+ long durationInNano = System.nanoTime() - start;
+ long dTime = TimeUnit.NANOSECONDS.toMillis(durationInNano);
+ long startsize = inputSizeBytes;
+ long endsize = SSTableReader.getTotalBytes(newSStables);
+ double ratio = (double) endsize / (double) startsize;
+
+ StringBuilder newSSTableNames = new StringBuilder();
+ for (SSTableReader reader : newSStables)
+
newSSTableNames.append(reader.descriptor.baseFilename()).append(",");
+ long totalSourceRows = 0;
+ for (int i = 0; i < mergedRowCounts.length; i++)
+ totalSourceRows += mergedRowCounts[i] * (i + 1);
+
+ String mergeSummary =
updateCompactionHistory(cfs.keyspace.getName(), cfs.getTableName(),
mergedRowCounts, startsize, endsize);
-logger.debug(String.format("Compacted (%s) %d sstables to [%s] to
level=%d. %s to %s (~%d%% of original) in %,dms. Read Throughput = %s, Write
Throughput = %s, Row Throughput = ~%,d/s. %,d total partitions merged to %,d.
Partition merge counts were {%s}",
++
++logger.info(String.format("Compacted (%s) %d sstables to [%s] to
level=%d. %s to %s (~%d%% of original) in %,dms. Read
[jira] [Commented] (CASSANDRA-16666) Make SSLContext creation pluggable/extensible
[ https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422722#comment-17422722 ] Stefan Miklosovic commented on CASSANDRA-1: --- I applied few changes. here https://github.com/instaclustr/cassandra/commit/38c2881b063fb58e1622967a94e9e9b8fcc0e60a Kubernetes example did not compile for me and it was also placing conf files produced by test to dir in conf which would polute the repository on the next commit instead of putting them to "build" and interacting with them there. Otherwise it is just more about some formatting and similar stuff. I run a build here: https://app.circleci.com/pipelines/github/instaclustr/cassandra/516/workflows/21747bff-fd40-4016-b900-3f5a8c906653 I created a formal PR here: https://github.com/apache/cassandra/pull/1243 Jenkins build is cooking here: https://ci-cassandra.apache.org/job/Cassandra-devbranch/1153/ Let's just sleep on this, I will probably merge on Friday's morning around 9am UTC+2 if I dont get any sudden shower thoughts :) > Make SSLContext creation pluggable/extensible > - > > Key: CASSANDRA-1 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Maulin Vasavada >Assignee: Maulin Vasavada >Priority: Normal > Fix For: 4.x > > Attachments: Screenshot from 2021-09-28 10-56-24.png > > > Currently Cassandra creates the SSLContext via SSLFactory.java. SSLFactory is > a final class with static methods and not overridable. The SSLFactory loads > the keys and certs from the file based artifacts for the same. While this > works for many, in the industry where security is stricter and contextual, > this approach falls short. Many big organizations need flexibility to load > the SSL artifacts from a custom resource (like custom Key Management > Solution, HashiCorp Vault, Amazon KMS etc). While JSSE SecurityProvider > architecture allows us flexibility to build our custom mechanisms to validate > and process security artifacts, many times all we need is to build upon > Java's existing extensibility that Trust/Key Manager interfaces provide to > load keystores from various resources in the absence of any customized > requirements on the Keys/Certificate formats. > My proposal here is to make the SSLContext creation pluggable/extensible and > have the current SSLFactory.java implement an extensible interface. > I contributed a similar change that is live now in Apache Kafka (2.6.0) - > https://issues.apache.org/jira/browse/KAFKA-8890 > I can spare some time writing the pluggable interface and run by the required > reviewers. > > Created [CEP-9: Make SSLContext creation > pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable] > > > cc: [~dcapwell] [~djoshi] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422713#comment-17422713
]
Stefan Miklosovic edited comment on CASSANDRA-16976 at 9/30/21, 11:56 AM:
--
I think it should be sorted and it should have same order :) I mean ... why
not? wdyt [~brandon.williams]?
So in that case as you put it - if progress happens to be on MiB but total is
on GiB, what I would ideally do is that I would translate MiB's to GiB's and
put GiB as unit.
OR
lets just remove "unit" from nodetool completely if it is inconsistent like
this, I think it does not provide any value here.
Is it possible to put anything else than bytes to vtable?
was (Author: stefan.miklosovic):
I think it should be sorted and it should have same order :) I mean ... why
not? wdyt [~brandon.williams]?
So in that case as you put it - if progress happens to be on MiB but total is
on GiB, what I would ideally do is that I would translate MiB's to GiB's and
put GiB as unit.
OR
lets just remove "unit" from nodetool completely if it is inconsistent like
this, I think it does provides any value here.
Is it possible to put anything else than bytes to vtable?
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422713#comment-17422713
]
Stefan Miklosovic edited comment on CASSANDRA-16976 at 9/30/21, 11:55 AM:
--
I think it should be sorted and it should have same order :) I mean ... why
not? wdyt [~brandon.williams]?
So in that case as you put it - if progress happens to be on MiB but total is
on GiB, what I would ideally do is that I would translate MiB's to GiB's and
put GiB as unit.
OR
lets just remove "unit" from nodetool completely if it is inconsistent like
this, I think it does provides any value here.
Is it possible to put anything else than bytes to vtable?
was (Author: stefan.miklosovic):
I think it should be sorted and it should have same order :) I mean ... why
not? wdyt [~brandon.williams]?
So in that case as you put it - if progress happens to be on MiB but total is
on GiB, what I would ideally do is that I would translate MiB's to GiB's and
put GiB as unit.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422718#comment-17422718
]
Aleksei Zotov commented on CASSANDRA-16976:
---
{quote}I would ideally do is that I would translate MiB's to GiB's and put GiB
as unit.
{quote}
But in such a case the problem for readability is still there. Imagine total is
GiB, progress is KiB. If we convert total to KiB then it will still be a large
unreadable number. WDYT?
PS: looks like we should have started a private chat :D
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422716#comment-17422716
]
Aleksei Zotov commented on CASSANDRA-16976:
---
{quote}There is "sstables" in between of these columns at both places and it
just looks weird.
{quote}
This one bothers me too, but the trick is that the VT columns are sorted (I'm
not sure regarding other tables). I tried to put "sstables" before "progress"
in the table metadata definition but the columns appear in sorted order anyway.
Any suggestions on that? I have not checked it further though.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422713#comment-17422713
]
Stefan Miklosovic commented on CASSANDRA-16976:
---
I think it should be sorted and it should have same order :) I mean ... why
not? wdyt [~brandon.williams]?
So in that case as you put it - if progress happens to be on MiB but total is
on GiB, what I would ideally do is that I would translate MiB's to GiB's and
put GiB as unit.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422708#comment-17422708
]
Aleksei Zotov commented on CASSANDRA-16976:
---
[~stefan.miklosovic]
{quote}are these outputs ordered in same fashion?
{quote}
I have not checked that, but based on my understanding their order might be
different. VTs are sorted based on partition key and clustering columns:
{code:java}
protected final NavigableMap partitions;
{code}
{code:java}
private final NavigableMap, Row> rows;
{code}
whereas nodetool is not additionally sorted. It is not a big deal to sort it,
but I'm wondering if there is any real benefit of doing it. There are no any
order guarantees for nodetool and VTs AFAIK, so I'm not sure having the same
order is smth really necessary. WDYT?
{quote}AbstractNetstatsStreaming
{quote}
Ok, thanks for the reference! I'll take a look.
{quote}Same holds for "progress".
{quote}
That's not smth related to my changes. But in general I agree and feel the
current logic is wrong. In fact we do not recalculate the unit based on
"total"/"progress" actual unit. Moreover, it is theoretically possible that
"total" and "progress" have different units ("total" is GiB, but "progress" is
yet MiB). I feel it makes sense to set unit to "N/A" in case of
"human-readable" output. WDYT?
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422705#comment-17422705
]
Stefan Miklosovic edited comment on CASSANDRA-16976 at 9/30/21, 11:35 AM:
--
Sorry for spamming but as I look at that, we should couple "progress" and
"total" closer together. There is "sstables" in between of these columns at
both places and it just looks weird.
The order of these columns should be: keyspace, table, id, kind, sstables,
total, progress, ratio, unit
was (Author: stefan.miklosovic):
Sorry for spamming but as I look at that, we should couple "progress" and
"totall" closer together. There is "sstables" in between of these columns at
both places and it just looks weird.
The order of these columns should be: keyspace, table, id, kind, sstables,
progress, total, ratio, unit
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422705#comment-17422705
]
Stefan Miklosovic commented on CASSANDRA-16976:
---
Sorry for spamming but as I look at that, we should couple "progress" and
"totall" closer together. There is "sstables" in between of these columns at
both places and it just looks weird.
The order of these columns should be: keyspace, table, id, kind, progress,
total, ratio, unit
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422705#comment-17422705
]
Stefan Miklosovic edited comment on CASSANDRA-16976 at 9/30/21, 11:34 AM:
--
Sorry for spamming but as I look at that, we should couple "progress" and
"totall" closer together. There is "sstables" in between of these columns at
both places and it just looks weird.
The order of these columns should be: keyspace, table, id, kind, sstables,
progress, total, ratio, unit
was (Author: stefan.miklosovic):
Sorry for spamming but as I look at that, we should couple "progress" and
"totall" closer together. There is "sstables" in between of these columns at
both places and it just looks weird.
The order of these columns should be: keyspace, table, id, kind, progress,
total, ratio, unit
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422702#comment-17422702
]
Stefan Miklosovic edited comment on CASSANDRA-16976 at 9/30/21, 11:20 AM:
--
[~azotcsit] By the way, if you look at your human readable output, there is
138.22MiB in "total" but "unit" is still on "bytes", that should be "MiB" and
"total" should be just a number without that unit. Same holds for "progress".
was (Author: stefan.miklosovic):
[~azotcsit] By the way, if you look at your human readable output, there is
138.22MiB in "total" but "unit" is still on "bytes", that should be "MiB" and
"total" should be just a number without that unit.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422702#comment-17422702
]
Stefan Miklosovic commented on CASSANDRA-16976:
---
[~azotcsit] By the way, if you look at your human readable output, there is
138.22MiB in "total" but "unit" is still on "bytes", that should be "MiB" and
"total" should be just a number without that unit.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16914) Implement Virtual Tables for Auth Caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422701#comment-17422701
]
Aleksei Zotov commented on CASSANDRA-16914:
---
[~samt]
Sounds reasonable to me. I'll rework the structure on the weekend and ping you
back for review. Thanks a lot for the detailed reply!
> Implement Virtual Tables for Auth Caches
>
>
> Key: CASSANDRA-16914
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16914
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization, Feature/Virtual Tables
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 4.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{NodeTool}} commands for Auth Caches invalidation were implemented as a part
> of CASSANDRA-16404 ticket. While discussing that ticket it was agreed that
> there is a need to develop the same kind of functionality through Vitrual
> Tables. Unfortunately, VT did not have {{TRUNCATE}} and {{DELETE}} support.
> And CASSANDRA-16806 was created for that reason. Once it is completed,
> further work can be started.
> The goal of this ticket is to create VTs for the following caches:
> * {{CredentialsCache}}
> * {{JmxPermissionsCache}}
> * {{NetworkPermissionsCache}}
> * {{PermissionsCache}}
> * {{RolesCache}}
> The VTs should support reading from and modification of the in the Auth
> Caches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422700#comment-17422700
]
Stefan Miklosovic edited comment on CASSANDRA-16976 at 9/30/21, 11:16 AM:
--
Hi [~azotcsit],
are these outputs ordered in same fashion? If I select everything on vtable and
I execute the nodetool command, is this in the same order? I would like to
preserve this, just to be consistent. Maybe they are as the information is
retrieved from the same "source" and just displayed differently, I just want to
double check. It is not visible from your output as there is just one line in
both.
I am fine with manual testing here, however, a test which is capturing the
output "on the fly" is for example in "AbstractNetstatsStreaming" in dtests if
you feel brave enough (or for other times) where I am continuosly getting the
output from netstats command and I am parsing that there.
was (Author: stefan.miklosovic):
Hi [~azotcsit],
are these outputs ordered in same fashion? If I select everything on vtable and
I execute the nodetool command, is this in the same order? I would like to
preserver this, just to be consistent. Maybe they are as the information is
retrieved from the same "source" and just displayed differently, I just want to
double check. It is not visible from your output as there is just one line in
both.
I am fine with manual testing here, however, a test which is capturing the
output "on the fly" is for example in "AbstractNetstatsStreaming" in dtests if
you feel brave enough (or for other times) where I am continuosly getting the
output from netstats command and I am parsing that there.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422700#comment-17422700
]
Stefan Miklosovic commented on CASSANDRA-16976:
---
Hi [~azotcsit],
are these outputs ordered in same fashion? If I select everything on vtable and
I execute the nodetool command, is this in the same order? I would like to
preserver this, just to be consistent. Maybe they are as the information is
retrieved from the same "source" and just displayed differently, I just want to
double check. It is not visible from your output as there is just one line in
both.
I am fine with manual testing here, however, a test which is capturing the
output "on the fly" is for example in "AbstractNetstatsStreaming" in dtests if
you feel brave enough (or for other times) where I am continuosly getting the
output from netstats command and I am parsing that there.
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422688#comment-17422688
]
Aleksei Zotov edited comment on CASSANDRA-16976 at 9/30/21, 10:57 AM:
--
Thanks Brandon, that's what I wanted to hear!
I made the necessary changes and here are the current outputs from my local:
{code:java}
nodetool compactionstats
pending tasks: 1
- keyspace1.standard1: 1
keyspace table task id completion ratio kind
progress sstables total unit
keyspace1 standard1 14e6c6c0-21d2-11ec-8ef7-bf6daa00e588 99.99%
Compaction 144850779 2144860627 bytes
Active compaction remaining time : 0h00m00s
{code}
{code:java}
nodetool compactionstats --human-readable
pending tasks: 1
- keyspace1.standard1: 1
keyspace table task id completion ratio kind
progress sstables total unit
keyspace1 standard1 1f9ca0d0-21d2-11ec-8ef7-bf6daa00e588 68.04%
Compaction 94.04 MiB 1138.22 MiB bytes
Active compaction remaining time : 0h00m00s
{code}
{code:java}
cqlsh:system_views> SELECT * FROM sstable_tasks;
keyspace_name | table_name | task_id |
completion_ratio | kind | progress | sstables | total| unit
--++--+--++--+--+--+---
keyspace1 | standard1 | 238f6290-1fd4-11ec-8c63-7d65848b040f |
0.212345 | compaction | 16184734 |2 | 76219177 | bytes
{code}
_Unfortunately, I could not get a horizontal scroll bar to the above output
snippets, please, copy and paste them to a text editor to see nice and clear
output._
I have not developed new tests because it seems to be hard to catch compaction
in a certain moment and ensure the expected state. I did not want to fight with
mocking everything either. Taking into account that my changes are pretty small
and straightforward, I feel that's ok. Please, let me know if you think
opposite. However, I'd be glad to get any suggestions on the best way to
introduce tests for this stuff.
[~brandon.williams] [~stefan.miklosovic]
Could you please make a review to this ticket.
was (Author: azotcsit):
Thanks [~brandon.williams]!
I made all necessary changes and here are the current outputs from my local:
{code:java}
nodetool compactionstats
pending tasks: 1
- keyspace1.standard1: 1
keyspace table task id completion ratio kind
progress sstables total unit
keyspace1 standard1 14e6c6c0-21d2-11ec-8ef7-bf6daa00e588 99.99%
Compaction 144850779 2144860627 bytes
Active compaction remaining time : 0h00m00s
{code}
{code:java}
nodetool compactionstats --human-readable
pending tasks: 1
- keyspace1.standard1: 1
keyspace table task id completion ratio kind progress sstables total unit
keyspace1 standard1 1f9ca0d0-21d2-11ec-8ef7-bf6daa00e588 68.04% Compaction
94.04 MiB 1 138.22 MiB bytes Active compaction remaining time : 0h00m00s
{code}
{code:java}
cqlsh:system_views> SELECT * FROM sstable_tasks;
keyspace_name | table_name | task_id |
completion_ratio | kind | progress | sstables | total| unit
--++--+--++--+--+--+---
keyspace1 | standard1 | 238f6290-1fd4-11ec-8c63-7d65848b040f |
0.212345 | compaction | 16184734 |2 | 76219177 | bytes
{code}
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.o
[jira] [Commented] (CASSANDRA-16914) Implement Virtual Tables for Auth Caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422695#comment-17422695
]
Sam Tunnicliffe commented on CASSANDRA-16914:
-
Hey [~azotcsit], sorry it took a while to respond, I agree it would be ideal to
deliver the nodetool & VT stuff in the same release.
Basically, I disagree with the intention of these VTs. One of the motivations
behind VTs (maybe the primary one) was to provide an alternative to JMX for
accessing configuration and metrics data. My perspective is that conceptually
these specific tables should exist in order to provide operators with the
ability to configure and control the auth caches without requiring JMX and, by
extension, nodetool. There is some prior art in the system_views.caches table,
which provides read-only metrics info for the counter, row and key caches. It
would be good to expose similar metrics info for the auth caches, but I see
that as a) not super critical as the datasets are so much smaller for auth
data, the hit/miss/occupancy rates are less significant b) outside the scope of
the original issue, which was to provide an operator-friendly way to invalidate
cached data. Perhaps we can file a follow up ticket to expose this data and
other cache config which can be mutated currently via JMX in a separate set of
VTs.
Putting conceptual concerns aside, I don't see there's any real value in being
able to retrieve the precise values that are currently cached from these VTs.
Firstly, there are existing means to obtain that data if required (i.e. using
the LIST X statements and/or querying the underlying tables directly). Second,
knowing that a particular auth datum it is cached _at this time_ doesn't really
benefit an operator. Consider the flow when an invalidation-triggering event
occurs; for instance, if a user has a privilege revoked, checking the VT to see
if it is cached or not is unnecessary work when an operator can simply issue
the idempotent invalidation query. In CASSANDRA-16404, we didn't implement a
set of query commands to look up precisely which values were cached (why would
we?), only the tools to invalidate them, should that be possible. So just
because it's possible to expose the full set of cached data (or even a subset
of it), it doesn't mean we should.
As well as being a somewhat pointless duplication, exposing the entire contents
of the auth tables in this way broadens the surface area of potential security
issues, as you mentioned. That said, permissions do apply to tables in
system_views, so the exposure is perhaps not as broad as you feared, however I
know there are already concerns about allowing read access to system_auth
tables.
My proposal would be to stick to modelling the invalidation commands as
provided by CASSANDRA-16404. Meaning, I would model the tables such that the
primary keys match the cache keys in the auth caches and not include anything
else:
{code}
cassandra@cqlsh> DESCRIBE TABLE system_views.permissions_cache_entries;
/*
Warning: Table system_views.permissions_cache_entries is a virtual table and
cannot be recreated with CQL.
Structure, for reference:
VIRTUAL TABLE system_views.permissions_cache_entries (
role text,
resource text,
PRIMARY KEY (role, resource)
) WITH comment = 'entries in the permissions cache';
*/
cassandra@cqlsh> SELECT * FROM system_views.permissions_cache_entries;
role | resource |
-+--+
user_a | data/test_ks/test_table1 |
user_b | data/test_ks/test_table2 |
user_b | data/test_ks/test_table3 |
user_c | data/test_ks/test_table4 |
# invalidate permissions for a single user on a single resource
cassandra@cqlsh> DELETE FROM system_views.permissions_cache_entries WHERE
role='user_a' AND resource='data/test_ks/test_table1';
# invalidate all permissions for a single user
cassandra@cqlsh> DELETE FROM system_views.permissions_cache_entries WHERE
role='user_b' ;
# invalidate the entire cache
cassandra@cqlsh> TRUNCATE system_views.permissions_cache_entries;
{code}
As I said, I think we could follow up this work with additional tables to expose
* metrics (analogous to system_views.caches)
* configuration options currently only exposed via JMX or yaml
(capacity/validity/updateinterval)
> Implement Virtual Tables for Auth Caches
>
>
> Key: CASSANDRA-16914
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16914
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization, Feature/Virtual Tables
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 4.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{NodeTool}} commands for Auth Caches invalida
[jira] [Commented] (CASSANDRA-16976) Make nodetool compactionstats and sstable_tasks consistent
[
https://issues.apache.org/jira/browse/CASSANDRA-16976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422688#comment-17422688
]
Aleksei Zotov commented on CASSANDRA-16976:
---
Thanks [~brandon.williams]!
I made all necessary changes and here are the current outputs from my local:
{code:java}
nodetool compactionstats
pending tasks: 1
- keyspace1.standard1: 1
keyspace table task id completion ratio kind
progress sstables total unit
keyspace1 standard1 14e6c6c0-21d2-11ec-8ef7-bf6daa00e588 99.99%
Compaction 144850779 2144860627 bytes
Active compaction remaining time : 0h00m00s
{code}
{code:java}
nodetool compactionstats --human-readable
pending tasks: 1
- keyspace1.standard1: 1
keyspace table task id completion ratio kind progress sstables total unit
keyspace1 standard1 1f9ca0d0-21d2-11ec-8ef7-bf6daa00e588 68.04% Compaction
94.04 MiB 1 138.22 MiB bytes Active compaction remaining time : 0h00m00s
{code}
{code:java}
cqlsh:system_views> SELECT * FROM sstable_tasks;
keyspace_name | table_name | task_id |
completion_ratio | kind | progress | sstables | total| unit
--++--+--++--+--+--+---
keyspace1 | standard1 | 238f6290-1fd4-11ec-8c63-7d65848b040f |
0.212345 | compaction | 16184734 |2 | 76219177 | bytes
{code}
> Make nodetool compactionstats and sstable_tasks consistent
> --
>
> Key: CASSANDRA-16976
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16976
> Project: Cassandra
> Issue Type: Task
> Components: Feature/Virtual Tables, Tool/nodetool
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
>
> Currently there is a difference in the output if {{nodetool compactionstats}}
> command and {{sstable_tasks}} virtual table (originally introduced by
> CASSANDRA-14457). They need to be aligned. The differences are:
>
> ||nodetool||VT||Comment||
> |percentComplete|completion_ratio|column naming is different only|
> |completed|progress|column naming is different only|
> |compaction type|kind|column naming is different only|
> |sstables|NA| |
> Additionally, please, update necessary documentation.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16975) CompactionTask#runMayThrow should not release new SSTables for offline transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422678#comment-17422678
]
Aleksandr Sorokoumov commented on CASSANDRA-16975:
--
Ah, you are right! I am still not used to the fact that 4.0 is not trunk :)
> CompactionTask#runMayThrow should not release new SSTables for offline
> transactions
> ---
>
> Key: CASSANDRA-16975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16975
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Right now, {{CompactionTask#runMayThrow}} releases new SSTables for offline
> transactions
> ([code|https://github.com/apache/cassandra/blob/f7c71f65c000c2c3ef7df1b034b8fdd822a396d8/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L227-L230]).
> This change was added in CASSANDRA-8962, prior to the introduction of
> lifecycle transactions in CASSANDRA-8568. I suspect that this behavior might
> be undesired and could have just fallen through the cracks.
> To my knowledge, this code does not cause any known bugs solely because
> in-tree tools do not access the SSTables they produce before exiting.
> However, if someone is to write, say, offline compaction daemon, it might
> break on subsequent compactions because newly created SSTables will be
> released.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16975) CompactionTask#runMayThrow should not release new SSTables for offline transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422673#comment-17422673
]
Andres de la Peña commented on CASSANDRA-16975:
---
[~Ge] I understand that the patch should also be applied to trunk, is this
right? I have prepared a patch for trunk
[here|https://github.com/apache/cassandra/compare/trunk...adelapena:16975-trunk-review],
with CI runs for
[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/936/workflows/0d59df0e-ac26-4c88-afc5-404bf767c033]
and
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/936/workflows/2f4fd070-4fd2-4753-a4e6-2137379f8997].
The runs contain 100 rounds of the new test, just in case.
> CompactionTask#runMayThrow should not release new SSTables for offline
> transactions
> ---
>
> Key: CASSANDRA-16975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16975
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Right now, {{CompactionTask#runMayThrow}} releases new SSTables for offline
> transactions
> ([code|https://github.com/apache/cassandra/blob/f7c71f65c000c2c3ef7df1b034b8fdd822a396d8/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L227-L230]).
> This change was added in CASSANDRA-8962, prior to the introduction of
> lifecycle transactions in CASSANDRA-8568. I suspect that this behavior might
> be undesired and could have just fallen through the cracks.
> To my knowledge, this code does not cause any known bugs solely because
> in-tree tools do not access the SSTables they produce before exiting.
> However, if someone is to write, say, offline compaction daemon, it might
> break on subsequent compactions because newly created SSTables will be
> released.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16975) CompactionTask#runMayThrow should not release new SSTables for offline transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-16975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422664#comment-17422664
]
Aleksandr Sorokoumov commented on CASSANDRA-16975:
--
Thank you!
> CompactionTask#runMayThrow should not release new SSTables for offline
> transactions
> ---
>
> Key: CASSANDRA-16975
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16975
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Right now, {{CompactionTask#runMayThrow}} releases new SSTables for offline
> transactions
> ([code|https://github.com/apache/cassandra/blob/f7c71f65c000c2c3ef7df1b034b8fdd822a396d8/src/java/org/apache/cassandra/db/compaction/CompactionTask.java#L227-L230]).
> This change was added in CASSANDRA-8962, prior to the introduction of
> lifecycle transactions in CASSANDRA-8568. I suspect that this behavior might
> be undesired and could have just fallen through the cracks.
> To my knowledge, this code does not cause any known bugs solely because
> in-tree tools do not access the SSTables they produce before exiting.
> However, if someone is to write, say, offline compaction daemon, it might
> break on subsequent compactions because newly created SSTables will be
> released.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16966) Avoid rewriting all sstables during nodetool cleanup
[
https://issues.apache.org/jira/browse/CASSANDRA-16966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-16966:
Reviewers: Alex Petrov (was: Alex Petrov, Marcus Eriksson)
> Avoid rewriting all sstables during nodetool cleanup
>
>
> Key: CASSANDRA-16966
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16966
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0.2, 4.1
>
>
> {{nodetool cleanup}} does not skip repaired sstables if transient replication
> is disabled
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16965) Avoid returning null in RangesAtEndpoint
[ https://issues.apache.org/jira/browse/CASSANDRA-16965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16965: Since Version: 4.0-alpha1 Source Control Link: https://github.com/apache/cassandra/commit/aaf72a7decf6964f00adb871333571de66c166a3 Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks! > Avoid returning null in RangesAtEndpoint > > > Key: CASSANDRA-16965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16965 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0.x > > > A race in RangesAtEndpoint can cause us to return null -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16965) Avoid returning null in RangesAtEndpoint
[ https://issues.apache.org/jira/browse/CASSANDRA-16965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16965: Status: Ready to Commit (was: Review In Progress) > Avoid returning null in RangesAtEndpoint > > > Key: CASSANDRA-16965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16965 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0.x > > > A race in RangesAtEndpoint can cause us to return null -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16965) Avoid returning null in RangesAtEndpoint
[ https://issues.apache.org/jira/browse/CASSANDRA-16965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16965: Reviewers: Jon Meredith, Marcus Eriksson (was: Jon Meredith) Status: Review In Progress (was: Patch Available) > Avoid returning null in RangesAtEndpoint > > > Key: CASSANDRA-16965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16965 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0.x > > > A race in RangesAtEndpoint can cause us to return null -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16965) Avoid returning null in RangesAtEndpoint
[ https://issues.apache.org/jira/browse/CASSANDRA-16965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16965: Reviewers: Jon Meredith (was: Jon Meredith, Marcus Eriksson) > Avoid returning null in RangesAtEndpoint > > > Key: CASSANDRA-16965 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16965 > Project: Cassandra > Issue Type: Bug > Components: Local/Other >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0.x > > > A race in RangesAtEndpoint can cause us to return null -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16997) Handle timestamp wraparound for internode messages
[ https://issues.apache.org/jira/browse/CASSANDRA-16997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16997: Fix Version/s: (was: 4.0.x) (was: 4.x) 4.1 4.0.2 Since Version: 4.0-alpha1 Source Control Link: https://github.com/apache/cassandra/commit/84ec1dc97d6358bd569d5467cb150abd0fc8939b Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks! > Handle timestamp wraparound for internode messages > -- > > Key: CASSANDRA-16997 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16997 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0.2, 4.1 > > > In internode messaging we send the lower 4 bytes of the timestamp and then > correct on receiving side if the local time has wrapped (every 50 days), but > we currently don't correct the time if the sending side wrapped before us. > This can cause us to use the wrong timestamp for the message and if the local > System.nanoTime is < 50 days, we get a negative creation time which causes an > AssertionError in MonitorableImpl -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
