[jira] [Comment Edited] (CASSANDRA-17019) JNA 5.6.0 does not support arm64
[ https://issues.apache.org/jira/browse/CASSANDRA-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426007#comment-17426007 ] Stefan Miklosovic edited comment on CASSANDRA-17019 at 10/8/21, 6:49 AM: - Hi [~igmar], we are building ARM here as part of our pipeline (1). Are you sure it does not work on arm64? We are building that, for example, on this node (3) (1) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts/ (2) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts-arm64/ (3) https://ci-cassandra.apache.org/computer/cassandra-arm2/ was (Author: stefan.miklosovic): Hi [~igmar], we are building ARM here as part of our pipeline (1). Are you sure it does not work on arm64? (1) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts/ (2) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts-arm64/ > JNA 5.6.0 does not support arm64 > > > Key: CASSANDRA-17019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17019 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Igamr Palsenberg >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.12, 4.0.2, 4.1 > > > Cassandra depends on net.java.dev.jna.jna version 5.6.0 to do the native > binding into the C library. > JNA 5.6.0 does not support arm64 architecture (Apple M1 devices), causing > cassandra to fail on bootstrap. > Bumping the dependency to 5.9.0 adds arm64 support. Will a PR to bump the > dependency be acceptable ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17019) JNA 5.6.0 does not support arm64
[ https://issues.apache.org/jira/browse/CASSANDRA-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426007#comment-17426007 ] Stefan Miklosovic edited comment on CASSANDRA-17019 at 10/8/21, 6:46 AM: - Hi [~igmar], we are building ARM here as part of our pipeline (1). Are you sure it does not work on arm64? (1) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts/ (2) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts-arm64/ was (Author: stefan.miklosovic): Hi [~igmar], we are building ARM here as part of our pipeline (1). Are you sure it does not work on arm64? (1) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts/ > JNA 5.6.0 does not support arm64 > > > Key: CASSANDRA-17019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17019 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Igamr Palsenberg >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.12, 4.0.2, 4.1 > > > Cassandra depends on net.java.dev.jna.jna version 5.6.0 to do the native > binding into the C library. > JNA 5.6.0 does not support arm64 architecture (Apple M1 devices), causing > cassandra to fail on bootstrap. > Bumping the dependency to 5.9.0 adds arm64 support. Will a PR to bump the > dependency be acceptable ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17019) JNA 5.6.0 does not support arm64
[ https://issues.apache.org/jira/browse/CASSANDRA-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17426007#comment-17426007 ] Stefan Miklosovic commented on CASSANDRA-17019: --- Hi [~igmar], we are building ARM here as part of our pipeline (1). Are you sure it does not work on arm64? (1) https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-artifacts/ > JNA 5.6.0 does not support arm64 > > > Key: CASSANDRA-17019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17019 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Igamr Palsenberg >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.12, 4.0.2, 4.1 > > > Cassandra depends on net.java.dev.jna.jna version 5.6.0 to do the native > binding into the C library. > JNA 5.6.0 does not support arm64 architecture (Apple M1 devices), causing > cassandra to fail on bootstrap. > Bumping the dependency to 5.9.0 adds arm64 support. Will a PR to bump the > dependency be acceptable ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17019) JNA 5.6.0 does not support arm64
[ https://issues.apache.org/jira/browse/CASSANDRA-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-17019: -- Fix Version/s: 3.11.12 > JNA 5.6.0 does not support arm64 > > > Key: CASSANDRA-17019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17019 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Igamr Palsenberg >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.11.12, 4.0.2, 4.1 > > > Cassandra depends on net.java.dev.jna.jna version 5.6.0 to do the native > binding into the C library. > JNA 5.6.0 does not support arm64 architecture (Apple M1 devices), causing > cassandra to fail on bootstrap. > Bumping the dependency to 5.9.0 adds arm64 support. Will a PR to bump the > dependency be acceptable ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17019) JNA 5.6.0 does not support arm64
[ https://issues.apache.org/jira/browse/CASSANDRA-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-17019: -- Fix Version/s: 4.1 4.0.2 > JNA 5.6.0 does not support arm64 > > > Key: CASSANDRA-17019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17019 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Igamr Palsenberg >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0.2, 4.1 > > > Cassandra depends on net.java.dev.jna.jna version 5.6.0 to do the native > binding into the C library. > JNA 5.6.0 does not support arm64 architecture (Apple M1 devices), causing > cassandra to fail on bootstrap. > Bumping the dependency to 5.9.0 adds arm64 support. Will a PR to bump the > dependency be acceptable ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated: turn off owasp dependency check in Jenkins pipeline because of instability
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git The following commit(s) were added to refs/heads/trunk by this push: new 851b7ca turn off owasp dependency check in Jenkins pipeline because of instability 851b7ca is described below commit 851b7cae9c1e992ab9cc6b2e61c27bc785eee8e7 Author: Stefan Miklosovic AuthorDate: Fri Oct 8 08:30:32 2021 +0200 turn off owasp dependency check in Jenkins pipeline because of instability see https://github.com/jeremylong/DependencyCheck/issues/3710 for more details --- build-scripts/cassandra-artifacts.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/build-scripts/cassandra-artifacts.sh b/build-scripts/cassandra-artifacts.sh index 9423789..a807a41 100755 --- a/build-scripts/cassandra-artifacts.sh +++ b/build-scripts/cassandra-artifacts.sh @@ -50,7 +50,9 @@ set +e # disable immediate exit from this point ARTIFACTS_BUILD_RUN=0 ECLIPSE_WARNINGS_RUN=0 -HAS_DEPENDENCY_CHECK_TARGET=$(ant -p build.xml | grep "dependency-check " | wc -l) +#HAS_DEPENDENCY_CHECK_TARGET=$(ant -p build.xml | grep "dependency-check " | wc -l) +# OWASP dep checs are unstable in Jenkins, we are getting 503 errors every now and then from NIST CVE database +HAS_DEPENDENCY_CHECK_TARGET=0 DEPENDENCY_CHECK_VERSION=6.3.2 for x in $(seq 1 3); do - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17019) JNA 5.6.0 does not support arm64
[ https://issues.apache.org/jira/browse/CASSANDRA-17019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic reassigned CASSANDRA-17019: - Assignee: Stefan Miklosovic > JNA 5.6.0 does not support arm64 > > > Key: CASSANDRA-17019 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17019 > Project: Cassandra > Issue Type: Bug > Components: Dependencies >Reporter: Igamr Palsenberg >Assignee: Stefan Miklosovic >Priority: Normal > > Cassandra depends on net.java.dev.jna.jna version 5.6.0 to do the native > binding into the C library. > JNA 5.6.0 does not support arm64 architecture (Apple M1 devices), causing > cassandra to fail on bootstrap. > Bumping the dependency to 5.9.0 adds arm64 support. Will a PR to bump the > dependency be acceptable ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17026) UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList failing on extra log entries
[
https://issues.apache.org/jira/browse/CASSANDRA-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-17026:
Fix Version/s: 4.1
> UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList
> failing on extra log entries
> -
>
> Key: CASSANDRA-17026
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17026
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging, Test/dtest/java
>Reporter: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.1
>
>
> This test, while parameterized and run more than once, expects a single log
> entry describing the reporting exclusion for the single query made with each
> parameter combination.
> However, we’re seeing this:
> {noformat}
> junit.framework.AssertionFailedError:
> Expected size:<1> but was:<2> in:
> <["DEBUG [nioEventLoopGroup-5-2] {instance:id} 2021-10-07T09:49:42,974
> Excluding client exception for /127.0.0.1:56628; address contained in
> client_error_reporting_exclusions",
> "DEBUG [nioEventLoopGroup-5-3] {instance:id} 2021-10-07T09:49:42,984
> Excluding client exception for /127.0.0.1:56634; address contained in
> client_error_reporting_exclusions"]>
> at
> org.apache.cassandra.distributed.test.UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList(UnableToParseClientMessageFromBlockedSubnetTest.java:114)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
> Looking at the logs more deeply, the V3 and V4 protocol cases are actually
> logging twice, once on initially processing the message, and again on channel
> closure. There should be a way to eliminate the double logging...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17026) UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList failing on extra log entries
[
https://issues.apache.org/jira/browse/CASSANDRA-17026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-17026:
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Normal
Discovered By: DTest
Reviewers: Jon Meredith
Severity: Low
Assignee: Caleb Rackliffe
Status: Open (was: Triage Needed)
> UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList
> failing on extra log entries
> -
>
> Key: CASSANDRA-17026
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17026
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging, Test/dtest/java
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.1
>
>
> This test, while parameterized and run more than once, expects a single log
> entry describing the reporting exclusion for the single query made with each
> parameter combination.
> However, we’re seeing this:
> {noformat}
> junit.framework.AssertionFailedError:
> Expected size:<1> but was:<2> in:
> <["DEBUG [nioEventLoopGroup-5-2] {instance:id} 2021-10-07T09:49:42,974
> Excluding client exception for /127.0.0.1:56628; address contained in
> client_error_reporting_exclusions",
> "DEBUG [nioEventLoopGroup-5-3] {instance:id} 2021-10-07T09:49:42,984
> Excluding client exception for /127.0.0.1:56634; address contained in
> client_error_reporting_exclusions"]>
> at
> org.apache.cassandra.distributed.test.UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList(UnableToParseClientMessageFromBlockedSubnetTest.java:114)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}
> Looking at the logs more deeply, the V3 and V4 protocol cases are actually
> logging twice, once on initially processing the message, and again on channel
> closure. There should be a way to eliminate the double logging...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17026) UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList failing on extra log entries
Caleb Rackliffe created CASSANDRA-17026:
---
Summary:
UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList
failing on extra log entries
Key: CASSANDRA-17026
URL: https://issues.apache.org/jira/browse/CASSANDRA-17026
Project: Cassandra
Issue Type: Bug
Components: Observability/Logging, Test/dtest/java
Reporter: Caleb Rackliffe
This test, while parameterized and run more than once, expects a single log
entry describing the reporting exclusion for the single query made with each
parameter combination.
However, we’re seeing this:
{noformat}
junit.framework.AssertionFailedError:
Expected size:<1> but was:<2> in:
<["DEBUG [nioEventLoopGroup-5-2] {instance:id} 2021-10-07T09:49:42,974
Excluding client exception for /127.0.0.1:56628; address contained in
client_error_reporting_exclusions",
"DEBUG [nioEventLoopGroup-5-3] {instance:id} 2021-10-07T09:49:42,984
Excluding client exception for /127.0.0.1:56634; address contained in
client_error_reporting_exclusions"]>
at
org.apache.cassandra.distributed.test.UnableToParseClientMessageFromBlockedSubnetTest.badMessageCausesProtocolExceptionFromExcludeList(UnableToParseClientMessageFromBlockedSubnetTest.java:114)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{noformat}
Looking at the logs more deeply, the V3 and V4 protocol cases are actually
logging twice, once on initially processing the message, and again on channel
closure. There should be a way to eliminate the double logging...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15248) Upgrade Guava to latest on master branch
[
https://issues.apache.org/jira/browse/CASSANDRA-15248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425792#comment-17425792
]
Tomo Suzuki commented on CASSANDRA-15248:
-
Update
{quote}Cassandra-all:3.11's dependency is blocking Beam's Guava dependency
(BEAM-8911).
{quote}
Somehow Beam team managed to unblock the Guava version problem and Beam
declares a newer version of Guava. I'm not waiting for Cassandra's Guava
dependency now.
> Upgrade Guava to latest on master branch
>
>
> Key: CASSANDRA-15248
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15248
> Project: Cassandra
> Issue Type: Task
> Components: Build, Dependencies, Packaging
>Reporter: Abhijit Sarkar
>Priority: Normal
>
> Upgrade Guava to latest on master branch. See
> https://issues.apache.org/jira/browse/CASSANDRA-15245.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17016) Allow reverse iteration of resources during permissions checking
[
https://issues.apache.org/jira/browse/CASSANDRA-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425771#comment-17425771
]
Aleksei Zotov commented on CASSANDRA-17016:
---
LGTM, +1.
> Allow reverse iteration of resources during permissions checking
>
>
> Key: CASSANDRA-17016
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17016
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
>
> When we perform authz checks for AuthenticatedUser on a given resource, we
> traverse the resource hierarchy until we either find the required permission
> or we exhaust the traversal. This goes from most specific resource (i.e.
> iResource we're interested in) to the broadest (the root for that iResource
> type).
> Since permissions are a whitelist it isn't possible for a permission granted
> on a more specific resource to be negated or overridden by a grant on a
> resource lower in the hierarchy towards the root. For example, for
> DataResource, the hierarchy goes:
> {code:java}
> root -> keyspace -> table{code}
> So for instance if we:
>
> {code:java}
> GRANT ALL ON KEYSPACE ks TO admin_user;
> {code}
> It is impossible to grant admin_user any set of permissions more restrictive
> than ALL on ks.
> When a request comes in for a user with permissions on a ks for example, you
> can have a path of:
> # Validate SELECT on t1
> # Then check for SELECT on ks
> # Then check for permissions on 'root'
> These unnecessary lookups can negate some of the benefits of caching (and
> pre-warming, which another ticket is in flight for), and lead to issues on
> bounces with timeouts.
> Additionally, the permissions cache ends up storing far more entries than
> necessary, as a subsequent request for ks.t2 by user_x will go through the
> same process and create a second cache entry etc.
> So all this said - this is something we should allow operators to opt-in to;
> impact and performance is going to be highly dependent on the pattern of
> authentication granting and usage on a cluster.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425761#comment-17425761 ] Stefan Miklosovic edited comment on CASSANDRA-14612 at 10/7/21, 7:36 PM: - I am afraid I need to turn this off in Jenkins. It is just not stable (1). Every now and then there is 503 returned from cve service / database and it makes it quite uncomfortable to not have builds more stable. I think the best way to use this is to leave it in Ant build and invoke it manually. I asked this: https://github.com/jeremylong/DependencyCheck/issues/3710 Disablement in Jenkins will be done in cassandra-builds. (1) https://ci-cassandra.apache.org/job/Cassandra-devbranch-artifacts/1115/jdk=jdk_1.8_latest,label=cassandra/consoleFull was (Author: stefan.miklosovic): I am afraid I need to turn this off in Jenkins. It is just not stable (1). Every now and then there is 503 returned from cve service / database and it makes it quite uncomfortable to not have builds more stable. I think the best way to use this is to leave it in Ant build and invoke it manually. I asked this: https://github.com/jeremylong/DependencyCheck/issues/3710 Disablement in Jenkins will be done in cassandra-builds. (1) > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Assignee: Stefan Miklosovic >Priority: Normal > Labels: build, security > Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1 > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14612) Please add OWASP Dependency Check to the build (pom.xml)
[ https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425761#comment-17425761 ] Stefan Miklosovic commented on CASSANDRA-14612: --- I am afraid I need to turn this off in Jenkins. It is just not stable (1). Every now and then there is 503 returned from cve service / database and it makes it quite uncomfortable to not have builds more stable. I think the best way to use this is to leave it in Ant build and invoke it manually. I asked this: https://github.com/jeremylong/DependencyCheck/issues/3710 Disablement in Jenkins will be done in cassandra-builds. (1) > Please add OWASP Dependency Check to the build (pom.xml) > > > Key: CASSANDRA-14612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14612 > Project: Cassandra > Issue Type: New Feature > Components: Build > Environment: All development, build, test, environments. >Reporter: Albert Baker >Assignee: Stefan Miklosovic >Priority: Normal > Labels: build, security > Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1 > > Original Estimate: 1h > Remaining Estimate: 1h > > Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an > outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to > perform a lookup for each dependant .jar to list any/all known > vulnerabilities for each jar. This step is needed because a manual MITRE CVE > lookup/check on the main component does not include checking for > vulnerabilities in components or in dependant libraries. > OWASP Dependency check : > https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most > Java build/make types (ant, maven, ivy, gradle). > Also, add the appropriate command to the nightly build to generate a report > of all known vulnerabilities in any/all third party libraries/dependencies > that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false > clean aggregate > Generating this report nightly/weekly will help inform the project's > development team if any dependant libraries have a reported known > vulnerailities. Project teams that keep up with removing vulnerabilities on a > weekly basis will help protect businesses that rely on these open source > componets. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16882) Save CircleCI resources with optional test jobs
[
https://issues.apache.org/jira/browse/CASSANDRA-16882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425744#comment-17425744
]
Andres de la Peña commented on CASSANDRA-16882:
---
Here are the patches for all the branches:
||patch||CI||
|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...adelapena:16882-option-7-3.0-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-3.0-v05]|
|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...adelapena:16882-option-7-3.11-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-3.11-v05]|
|[4.0|https://github.com/apache/cassandra/compare/cassandra-4.0...adelapena:16882-option-7-4.0-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-4.0-v05]|
|[trunk|https://github.com/apache/cassandra/compare/trunk...adelapena:16882-option-7-trunk-v05]|[workflows|https://app.circleci.com/pipelines/github/adelapena/cassandra?branch=16882-option-7-trunk-v05]|
I have modified the {{java8_separate_tests}}/{{java11_separate_tests}}
workflows to have an additional approval step for the build. With this change
commits and pushes don't run anything at all until manually started. It adds an
extra click for those workflows, but I think that this way the graphs keep
better symmetry with the
{{java8_pre-commit_tests}}/{{java11_pre-commit_tests}}, so all the four
workflows have the same organisation and number of columns.
As for 3.0 and 3.11, I have tried to homogenise the way the jobs are organised.
Specifically, the approval steps depended on the build job, so one had to wait
for the completion of the build before starting the optional tests. I have
modified this to adopt the 4.0/trunk organisation, where the approval jobs
don't depend on the build and it's the approved job what depends on both the
build and its approval job.
I have sent [this
message|https://lists.apache.org/thread.html/r02822bd5071029b95037b44a2d31fe5bb1c98d9f2391fd0443ca8684%40%3Cdev.cassandra.apache.org%3E]
to the dev mail list informing about the proposed changes.
Just to summarise what we get with this ticket:
* Intermediate commits/pushes don't spend any resources at all unless their
jobs are manually approved.
* For changes that are ready for a final round of review or that are ready to
commit, the {{javaX_pre-commit_tests}} workflows have a single and easily
visible button that runs the most relevant tests. In the future we can always
change the set of tests that should be run here.
* For intermediate steps or special cases such as fixing flaky tests, the
{{javaX_separate_tests}} workflows allow to run any combination of tests
individually.
> Save CircleCI resources with optional test jobs
> ---
>
> Key: CASSANDRA-16882
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16882
> Project: Cassandra
> Issue Type: Task
> Components: CI
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
>
> This ticket implements the addition of approval steps in the CircleCI
> workflows as it was proposed in [this
> email|https://lists.apache.org/thread.html/r57bab800d037c087af01b3779fd266d83b538cdd29c120f74a5dbe63%40%3Cdev.cassandra.apache.org%3E]
> sent to the dev list:
> The current CircleCI configuration automatically runs the unit tests, JVM
> dtests and cqhshlib tests. This is done by default for every commit or, with
> some configuration, for every push.
> Along the lifecycle of a ticket it is quite frequent to have multiple commits
> and pushes, all running these test jobs. I'd say that frequently it is not
> necessary to run the tests for some of those intermediate commits and pushes.
> For example, one can show proofs of concept, or have multiple rounds of
> review before actually running the tests. Running the tests for every change
> can produce an unnecessary expense of CircleCI resources.
> I think we could make running those tests optional, as well as clearly
> specifying in the documentation what are the tests runs that are mandatory
> before actually committing. We could do this in different ways:
> # Make the entire CircleCI workflow optional, so the build job requires
> manual approval. Once the build is approved the mandatory test jobs would
> be run without any further approval, exactly as it's currently done.
> # Make all the test jobs optional, so every test job requires manual
> approval, and the documentation specifies which tests are mandatory in the
> final steps of a ticket.
> # Make all the mandatory test jobs depend on a single optional job, so we
> have a single button to optionally run all the mandatory tests.
> I think any of these ch
[jira] [Commented] (CASSANDRA-17025) Direct memory OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425730#comment-17425730
]
Paulo Motta commented on CASSANDRA-17025:
-
{quote}Increase -Djdk.nio.maxCachedBufferSize
{quote}
[~zachtoews] what are you setting {{-Djdk.nio.maxCachedBufferSize}} to ? This
should be rather small and not big (ie. -Djdk.nio.maxCachedBufferSize=1048576
or -Djdk.nio.maxCachedBufferSize=262144)
Btw you should probably only set this and not mess with -XX:MaxDirectMemorySize.
See
[https://support.datastax.com/s/article/JVM-OOM-direct-buffer-errors-affected-by-unlimited-javanio-cache.]
> Direct memory OOM
> --
>
> Key: CASSANDRA-17025
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17025
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zach Toews
>Priority: Normal
>
> Version: 3.11.11
> Cluster: 9 nodes (1 dc, 3 racks) on aws r5.4xlarge nodes (16 vCPU, 128GB mem)
> Heap size: 20G
> Direct memory buffer: 24G
>
> We stood up this cluster a few months ago in order to migrate an old 2.0
> cluster. Since then, after about 7-10 days a node will begin to experience
> long old gc cycles before logging many "java.lang.OutOfMemoryError: Direct
> buffer memory" exceptions. When the long gc cycles start, the entire cluster
> becomes unresponsive (our application is unable to make queries to any node).
> Restarting cassandra on the failing node resolves the problem, then we have
> to restart every other node in the cluster to prevent them from getting into
> the same state.
>
> We have attempted to:
> * Increase -XX:MaxDirectMemorySize
> * Increase -Djdk.nio.maxCachedBufferSize
> * Update cassandra from 3.11.10 to 3.11.11
> None of these have resolved the problem.
> Since the last failure, we have increased -XX:MaxDirectMemorySize again, and
> are waiting to see if that has any effect.
>
> Old gc collections from system.log:
> {noformat}
> INFO [Service Thread] 2021-10-04 15:24:04,973 GCInspector.java:285 - G1 Old
> Generation GC in 4447ms. Compressed Class Space: 6683064 -> 6677952; G1 Eden
> Space: 16777216 -> 0; G1 Old Gen: 5431375384 -> 745163912; G1 Survivor Space:
> 419430400 -> 0; Metaspace: 54716768 ->
> ...
> INFO [Service Thread] 2021-10-04 15:24:06,985 GCInspector.java:285 - G1 Old
> Generation GC in 1901ms. G1 Old Gen: 745163912 -> 745168360;
> ...
> INFO [Service Thread] 2021-10-04 15:24:09,306 GCInspector.java:285 - G1 Old
> Generation GC in 2046ms. G1 Eden Space: 528482304 -> 0; G1 Old Gen:
> 759785184 -> 761229864;
> ...
> INFO [Service Thread] 2021-10-04 15:24:14,749 GCInspector.java:285 - G1 Old
> Generation GC in 5403ms. G1 Old Gen: 761229864 -> 762168640;
> ...
> INFO [Service Thread] 2021-10-04 15:24:16,782 GCInspector.java:285 - G1 Old
> Generation GC in 1949ms. G1 Old Gen: 762168640 -> 762167640;
> ...
> INFO [Service Thread] 2021-10-04 15:25:09,406 GCInspector.java:285 - G1 Old
> Generation GC in 52302ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 762167640
> -> 762168160;
> ...
> INFO [Service Thread] 2021-10-04 15:25:15,011 GCInspector.java:285 - G1 Old
> Generation GC in 5522ms. G1 Eden Space: 192937984 -> 0; G1 Old Gen:
> 762168160 -> 770098088;
> ...
> INFO [Service Thread] 2021-10-04 15:25:31,453 GCInspector.java:285 - G1 Old
> Generation GC in 16310ms. G1 Eden Space: 201326592 -> 0; G1 Old Gen:
> 770098088 -> 769228400;
> ...
> INFO [Service Thread] 2021-10-04 15:25:33,597 GCInspector.java:285 - G1 Old
> Generation GC in 1984ms. G1 Eden Space: 352321536 -> 0; G1 Old Gen:
> 750824952 -> 751118968;
> ...
> INFO [Service Thread] 2021-10-04 15:25:50,152 GCInspector.java:285 - G1 Old
> Generation GC in 16411ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 751118968
> -> 751645056;
> {noformat}
> Example of direct memory oom from system.log
> {noformat}
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> READ messages were dropped in last 5000 ms: 2 internal and 7 cross node. Mean
> internal dropped latency: 47238 ms and Mean cross-node dropped latency: 45171
> ms
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> COUNTER_MUTATION messages were dropped in last 5000 ms: 1 internal and 60
> cross node. Mean internal dropped latency: 7289 ms and Mean cross-node
> dropped latency: 9509 ms
> ERROR [CounterMutationStage-464] 2021-10-04 15:25:31,484
> JVMStabilityInspector.java:94 - OutOfMemory error letting the JVM handle the
> error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:695) ~[na:1.8.0_292]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
> ~[na:1.8.0_292]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
> ~[na:1.8.0_292]
> at
> org.apache.cassandra.u
[jira] [Comment Edited] (CASSANDRA-17025) Direct memory OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425694#comment-17425694
]
Brandon Williams edited comment on CASSANDRA-17025 at 10/7/21, 5:30 PM:
A node that is up always reports itself as such. Failure detection is
independent of each node to another, so this is expected behavior in the face
of GC problems.
was (Author: brandon.williams):
A node that is up always reports itself as such. Failure detection is
independent of each node to another, this is perfectly normal behavior.
> Direct memory OOM
> --
>
> Key: CASSANDRA-17025
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17025
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zach Toews
>Priority: Normal
>
> Version: 3.11.11
> Cluster: 9 nodes (1 dc, 3 racks) on aws r5.4xlarge nodes (16 vCPU, 128GB mem)
> Heap size: 20G
> Direct memory buffer: 24G
>
> We stood up this cluster a few months ago in order to migrate an old 2.0
> cluster. Since then, after about 7-10 days a node will begin to experience
> long old gc cycles before logging many "java.lang.OutOfMemoryError: Direct
> buffer memory" exceptions. When the long gc cycles start, the entire cluster
> becomes unresponsive (our application is unable to make queries to any node).
> Restarting cassandra on the failing node resolves the problem, then we have
> to restart every other node in the cluster to prevent them from getting into
> the same state.
>
> We have attempted to:
> * Increase -XX:MaxDirectMemorySize
> * Increase -Djdk.nio.maxCachedBufferSize
> * Update cassandra from 3.11.10 to 3.11.11
> None of these have resolved the problem.
> Since the last failure, we have increased -XX:MaxDirectMemorySize again, and
> are waiting to see if that has any effect.
>
> Old gc collections from system.log:
> {noformat}
> INFO [Service Thread] 2021-10-04 15:24:04,973 GCInspector.java:285 - G1 Old
> Generation GC in 4447ms. Compressed Class Space: 6683064 -> 6677952; G1 Eden
> Space: 16777216 -> 0; G1 Old Gen: 5431375384 -> 745163912; G1 Survivor Space:
> 419430400 -> 0; Metaspace: 54716768 ->
> ...
> INFO [Service Thread] 2021-10-04 15:24:06,985 GCInspector.java:285 - G1 Old
> Generation GC in 1901ms. G1 Old Gen: 745163912 -> 745168360;
> ...
> INFO [Service Thread] 2021-10-04 15:24:09,306 GCInspector.java:285 - G1 Old
> Generation GC in 2046ms. G1 Eden Space: 528482304 -> 0; G1 Old Gen:
> 759785184 -> 761229864;
> ...
> INFO [Service Thread] 2021-10-04 15:24:14,749 GCInspector.java:285 - G1 Old
> Generation GC in 5403ms. G1 Old Gen: 761229864 -> 762168640;
> ...
> INFO [Service Thread] 2021-10-04 15:24:16,782 GCInspector.java:285 - G1 Old
> Generation GC in 1949ms. G1 Old Gen: 762168640 -> 762167640;
> ...
> INFO [Service Thread] 2021-10-04 15:25:09,406 GCInspector.java:285 - G1 Old
> Generation GC in 52302ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 762167640
> -> 762168160;
> ...
> INFO [Service Thread] 2021-10-04 15:25:15,011 GCInspector.java:285 - G1 Old
> Generation GC in 5522ms. G1 Eden Space: 192937984 -> 0; G1 Old Gen:
> 762168160 -> 770098088;
> ...
> INFO [Service Thread] 2021-10-04 15:25:31,453 GCInspector.java:285 - G1 Old
> Generation GC in 16310ms. G1 Eden Space: 201326592 -> 0; G1 Old Gen:
> 770098088 -> 769228400;
> ...
> INFO [Service Thread] 2021-10-04 15:25:33,597 GCInspector.java:285 - G1 Old
> Generation GC in 1984ms. G1 Eden Space: 352321536 -> 0; G1 Old Gen:
> 750824952 -> 751118968;
> ...
> INFO [Service Thread] 2021-10-04 15:25:50,152 GCInspector.java:285 - G1 Old
> Generation GC in 16411ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 751118968
> -> 751645056;
> {noformat}
> Example of direct memory oom from system.log
> {noformat}
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> READ messages were dropped in last 5000 ms: 2 internal and 7 cross node. Mean
> internal dropped latency: 47238 ms and Mean cross-node dropped latency: 45171
> ms
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> COUNTER_MUTATION messages were dropped in last 5000 ms: 1 internal and 60
> cross node. Mean internal dropped latency: 7289 ms and Mean cross-node
> dropped latency: 9509 ms
> ERROR [CounterMutationStage-464] 2021-10-04 15:25:31,484
> JVMStabilityInspector.java:94 - OutOfMemory error letting the JVM handle the
> error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:695) ~[na:1.8.0_292]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
> ~[na:1.8.0_292]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
> ~[na:1.8.0_292]
> at
> org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool
[jira] [Commented] (CASSANDRA-16983) Separating CQLSH credentials from the cqlshrc file
[ https://issues.apache.org/jira/browse/CASSANDRA-16983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425698#comment-17425698 ] Stefan Miklosovic commented on CASSANDRA-16983: --- I agree with 2). I have already started it in 16956 (removal from pylib) so we can get rid of that there completely after this is in. Is there anything else to cover? I can run a build and we can get to merging itself I guess? > Separating CQLSH credentials from the cqlshrc file > -- > > Key: CASSANDRA-16983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16983 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Bowen Song >Assignee: Bowen Song >Priority: Normal > Labels: lhf > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, the CQLSH tool accepts credentials (username & password) from the > following 3 places: > 1. the command line parameter "-p" > 2. the cqlshrc file > 3. prompt the user > This is not ideal. > Credentials in the command line is a security risk, because it could be see > by other users on a shared system. > The cqlshrc file is better, but still not good enough. Because the cqlshrc > file is a config file, it's often acceptable to have it as a world readable > file, and share it with other users. It also prevents user from having > multiple sets of credentials, either for the same Cassandra cluster or > different clusters. > To improve the security of CQLSH and make it secure by design, I purpose the > following changes: > * Warn the user if a password is giving in the command line, and recommend > them to use a credential file instead > * Warn the user if credentials are present in the cqlshrc file and the > cqlshrc file is not secure (e.g.: world readable or owned by a different user) > * Deprecate credentials in the cqlshrc, and recommend the user to move them > to a separate credential file. The aim is to not break anything at the > moment, but eventually stop accepting credentials from the cqlshrc file. > * Reject the credentials file if it's not secure, and tell the user how to > secure it. Optionally, prompt the user for password if it's an interactive > session. (Think how does OpenSSH handle insecure credential files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17025) Direct memory OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425694#comment-17425694
]
Brandon Williams commented on CASSANDRA-17025:
--
A node that is up always reports itself as such. Failure detection is
independent of each node to another, this is perfectly normal behavior.
> Direct memory OOM
> --
>
> Key: CASSANDRA-17025
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17025
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zach Toews
>Priority: Normal
>
> Version: 3.11.11
> Cluster: 9 nodes (1 dc, 3 racks) on aws r5.4xlarge nodes (16 vCPU, 128GB mem)
> Heap size: 20G
> Direct memory buffer: 24G
>
> We stood up this cluster a few months ago in order to migrate an old 2.0
> cluster. Since then, after about 7-10 days a node will begin to experience
> long old gc cycles before logging many "java.lang.OutOfMemoryError: Direct
> buffer memory" exceptions. When the long gc cycles start, the entire cluster
> becomes unresponsive (our application is unable to make queries to any node).
> Restarting cassandra on the failing node resolves the problem, then we have
> to restart every other node in the cluster to prevent them from getting into
> the same state.
>
> We have attempted to:
> * Increase -XX:MaxDirectMemorySize
> * Increase -Djdk.nio.maxCachedBufferSize
> * Update cassandra from 3.11.10 to 3.11.11
> None of these have resolved the problem.
> Since the last failure, we have increased -XX:MaxDirectMemorySize again, and
> are waiting to see if that has any effect.
>
> Old gc collections from system.log:
> {noformat}
> INFO [Service Thread] 2021-10-04 15:24:04,973 GCInspector.java:285 - G1 Old
> Generation GC in 4447ms. Compressed Class Space: 6683064 -> 6677952; G1 Eden
> Space: 16777216 -> 0; G1 Old Gen: 5431375384 -> 745163912; G1 Survivor Space:
> 419430400 -> 0; Metaspace: 54716768 ->
> ...
> INFO [Service Thread] 2021-10-04 15:24:06,985 GCInspector.java:285 - G1 Old
> Generation GC in 1901ms. G1 Old Gen: 745163912 -> 745168360;
> ...
> INFO [Service Thread] 2021-10-04 15:24:09,306 GCInspector.java:285 - G1 Old
> Generation GC in 2046ms. G1 Eden Space: 528482304 -> 0; G1 Old Gen:
> 759785184 -> 761229864;
> ...
> INFO [Service Thread] 2021-10-04 15:24:14,749 GCInspector.java:285 - G1 Old
> Generation GC in 5403ms. G1 Old Gen: 761229864 -> 762168640;
> ...
> INFO [Service Thread] 2021-10-04 15:24:16,782 GCInspector.java:285 - G1 Old
> Generation GC in 1949ms. G1 Old Gen: 762168640 -> 762167640;
> ...
> INFO [Service Thread] 2021-10-04 15:25:09,406 GCInspector.java:285 - G1 Old
> Generation GC in 52302ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 762167640
> -> 762168160;
> ...
> INFO [Service Thread] 2021-10-04 15:25:15,011 GCInspector.java:285 - G1 Old
> Generation GC in 5522ms. G1 Eden Space: 192937984 -> 0; G1 Old Gen:
> 762168160 -> 770098088;
> ...
> INFO [Service Thread] 2021-10-04 15:25:31,453 GCInspector.java:285 - G1 Old
> Generation GC in 16310ms. G1 Eden Space: 201326592 -> 0; G1 Old Gen:
> 770098088 -> 769228400;
> ...
> INFO [Service Thread] 2021-10-04 15:25:33,597 GCInspector.java:285 - G1 Old
> Generation GC in 1984ms. G1 Eden Space: 352321536 -> 0; G1 Old Gen:
> 750824952 -> 751118968;
> ...
> INFO [Service Thread] 2021-10-04 15:25:50,152 GCInspector.java:285 - G1 Old
> Generation GC in 16411ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 751118968
> -> 751645056;
> {noformat}
> Example of direct memory oom from system.log
> {noformat}
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> READ messages were dropped in last 5000 ms: 2 internal and 7 cross node. Mean
> internal dropped latency: 47238 ms and Mean cross-node dropped latency: 45171
> ms
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> COUNTER_MUTATION messages were dropped in last 5000 ms: 1 internal and 60
> cross node. Mean internal dropped latency: 7289 ms and Mean cross-node
> dropped latency: 9509 ms
> ERROR [CounterMutationStage-464] 2021-10-04 15:25:31,484
> JVMStabilityInspector.java:94 - OutOfMemory error letting the JVM handle the
> error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:695) ~[na:1.8.0_292]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
> ~[na:1.8.0_292]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
> ~[na:1.8.0_292]
> at
> org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:114)
> ~[apache-cassandra-3.11.11.jar:3.11.11]
> at
> org.apache.cassandra.utils.memory.BufferPool.access$1000(BufferPool.java:50)
> ~[apache-cassandra-3.11.11.jar:3.11.11]
> at
> org.apache.cassandra.utils.memory.BufferPool$L
[jira] [Commented] (CASSANDRA-17025) Direct memory OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425693#comment-17425693
]
Zach Toews commented on CASSANDRA-17025:
[~brandon.williams] To clarify, what we're seeing is the node with the long gc
pauses is marked down by the other nodes while `nodetool status` on the down
nodes says it is still up. While it is in this state the entire cluster becomes
unresponsive until the down node is restarted.
> Direct memory OOM
> --
>
> Key: CASSANDRA-17025
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17025
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zach Toews
>Priority: Normal
>
> Version: 3.11.11
> Cluster: 9 nodes (1 dc, 3 racks) on aws r5.4xlarge nodes (16 vCPU, 128GB mem)
> Heap size: 20G
> Direct memory buffer: 24G
>
> We stood up this cluster a few months ago in order to migrate an old 2.0
> cluster. Since then, after about 7-10 days a node will begin to experience
> long old gc cycles before logging many "java.lang.OutOfMemoryError: Direct
> buffer memory" exceptions. When the long gc cycles start, the entire cluster
> becomes unresponsive (our application is unable to make queries to any node).
> Restarting cassandra on the failing node resolves the problem, then we have
> to restart every other node in the cluster to prevent them from getting into
> the same state.
>
> We have attempted to:
> * Increase -XX:MaxDirectMemorySize
> * Increase -Djdk.nio.maxCachedBufferSize
> * Update cassandra from 3.11.10 to 3.11.11
> None of these have resolved the problem.
> Since the last failure, we have increased -XX:MaxDirectMemorySize again, and
> are waiting to see if that has any effect.
>
> Old gc collections from system.log:
> {noformat}
> INFO [Service Thread] 2021-10-04 15:24:04,973 GCInspector.java:285 - G1 Old
> Generation GC in 4447ms. Compressed Class Space: 6683064 -> 6677952; G1 Eden
> Space: 16777216 -> 0; G1 Old Gen: 5431375384 -> 745163912; G1 Survivor Space:
> 419430400 -> 0; Metaspace: 54716768 ->
> ...
> INFO [Service Thread] 2021-10-04 15:24:06,985 GCInspector.java:285 - G1 Old
> Generation GC in 1901ms. G1 Old Gen: 745163912 -> 745168360;
> ...
> INFO [Service Thread] 2021-10-04 15:24:09,306 GCInspector.java:285 - G1 Old
> Generation GC in 2046ms. G1 Eden Space: 528482304 -> 0; G1 Old Gen:
> 759785184 -> 761229864;
> ...
> INFO [Service Thread] 2021-10-04 15:24:14,749 GCInspector.java:285 - G1 Old
> Generation GC in 5403ms. G1 Old Gen: 761229864 -> 762168640;
> ...
> INFO [Service Thread] 2021-10-04 15:24:16,782 GCInspector.java:285 - G1 Old
> Generation GC in 1949ms. G1 Old Gen: 762168640 -> 762167640;
> ...
> INFO [Service Thread] 2021-10-04 15:25:09,406 GCInspector.java:285 - G1 Old
> Generation GC in 52302ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 762167640
> -> 762168160;
> ...
> INFO [Service Thread] 2021-10-04 15:25:15,011 GCInspector.java:285 - G1 Old
> Generation GC in 5522ms. G1 Eden Space: 192937984 -> 0; G1 Old Gen:
> 762168160 -> 770098088;
> ...
> INFO [Service Thread] 2021-10-04 15:25:31,453 GCInspector.java:285 - G1 Old
> Generation GC in 16310ms. G1 Eden Space: 201326592 -> 0; G1 Old Gen:
> 770098088 -> 769228400;
> ...
> INFO [Service Thread] 2021-10-04 15:25:33,597 GCInspector.java:285 - G1 Old
> Generation GC in 1984ms. G1 Eden Space: 352321536 -> 0; G1 Old Gen:
> 750824952 -> 751118968;
> ...
> INFO [Service Thread] 2021-10-04 15:25:50,152 GCInspector.java:285 - G1 Old
> Generation GC in 16411ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 751118968
> -> 751645056;
> {noformat}
> Example of direct memory oom from system.log
> {noformat}
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> READ messages were dropped in last 5000 ms: 2 internal and 7 cross node. Mean
> internal dropped latency: 47238 ms and Mean cross-node dropped latency: 45171
> ms
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> COUNTER_MUTATION messages were dropped in last 5000 ms: 1 internal and 60
> cross node. Mean internal dropped latency: 7289 ms and Mean cross-node
> dropped latency: 9509 ms
> ERROR [CounterMutationStage-464] 2021-10-04 15:25:31,484
> JVMStabilityInspector.java:94 - OutOfMemory error letting the JVM handle the
> error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:695) ~[na:1.8.0_292]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
> ~[na:1.8.0_292]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
> ~[na:1.8.0_292]
> at
> org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:114)
> ~[apache-cassandra-3.11.11.jar:3.11.11]
> at
> org.apache.cassandra.utils.memory.BufferPool.access$
[jira] [Commented] (CASSANDRA-16983) Separating CQLSH credentials from the cqlshrc file
[ https://issues.apache.org/jira/browse/CASSANDRA-16983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425686#comment-17425686 ] Brandon Williams commented on CASSANDRA-16983: -- I'm fine with 2. Removing windows support entirely from the codebase is going to be a long road and hoisting the entire responsibility of removing it from cqlsh onto this ticket isn't appropriate, and a couple more lines is no big deal. > Separating CQLSH credentials from the cqlshrc file > -- > > Key: CASSANDRA-16983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16983 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Bowen Song >Assignee: Bowen Song >Priority: Normal > Labels: lhf > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, the CQLSH tool accepts credentials (username & password) from the > following 3 places: > 1. the command line parameter "-p" > 2. the cqlshrc file > 3. prompt the user > This is not ideal. > Credentials in the command line is a security risk, because it could be see > by other users on a shared system. > The cqlshrc file is better, but still not good enough. Because the cqlshrc > file is a config file, it's often acceptable to have it as a world readable > file, and share it with other users. It also prevents user from having > multiple sets of credentials, either for the same Cassandra cluster or > different clusters. > To improve the security of CQLSH and make it secure by design, I purpose the > following changes: > * Warn the user if a password is giving in the command line, and recommend > them to use a credential file instead > * Warn the user if credentials are present in the cqlshrc file and the > cqlshrc file is not secure (e.g.: world readable or owned by a different user) > * Deprecate credentials in the cqlshrc, and recommend the user to move them > to a separate credential file. The aim is to not break anything at the > moment, but eventually stop accepting credentials from the cqlshrc file. > * Reject the credentials file if it's not secure, and tell the user how to > secure it. Optionally, prompt the user for password if it's an interactive > session. (Think how does OpenSSH handle insecure credential files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16983) Separating CQLSH credentials from the cqlshrc file
[ https://issues.apache.org/jira/browse/CASSANDRA-16983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425680#comment-17425680 ] Bowen Song commented on CASSANDRA-16983: I'm okay with that. So here's two ways of dropping Windows support from CQLSH: 1. I remove the two lines referencing to the "is_win" variable, and commit my changes. Someone else later look at cqlsh.py code and remove all Windows related code. 2. My change gets merged as it is. Someone else later look at cqlsh.py and remove those two lines along with all other Windows related code. I personally see the option 2 being more attractive, because it will gets this change out sooner, and it'll only create a negligible amount of extra work for the person who works on removing Windows support from CQLSH. > Separating CQLSH credentials from the cqlshrc file > -- > > Key: CASSANDRA-16983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16983 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Bowen Song >Assignee: Bowen Song >Priority: Normal > Labels: lhf > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, the CQLSH tool accepts credentials (username & password) from the > following 3 places: > 1. the command line parameter "-p" > 2. the cqlshrc file > 3. prompt the user > This is not ideal. > Credentials in the command line is a security risk, because it could be see > by other users on a shared system. > The cqlshrc file is better, but still not good enough. Because the cqlshrc > file is a config file, it's often acceptable to have it as a world readable > file, and share it with other users. It also prevents user from having > multiple sets of credentials, either for the same Cassandra cluster or > different clusters. > To improve the security of CQLSH and make it secure by design, I purpose the > following changes: > * Warn the user if a password is giving in the command line, and recommend > them to use a credential file instead > * Warn the user if credentials are present in the cqlshrc file and the > cqlshrc file is not secure (e.g.: world readable or owned by a different user) > * Deprecate credentials in the cqlshrc, and recommend the user to move them > to a separate credential file. The aim is to not break anything at the > moment, but eventually stop accepting credentials from the cqlshrc file. > * Reject the credentials file if it's not secure, and tell the user how to > secure it. Optionally, prompt the user for password if it's an interactive > session. (Think how does OpenSSH handle insecure credential files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17025) Direct memory OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17025:
-
Resolution: Invalid
Status: Resolved (was: Triage Needed)
This jira is for Apache Cassandra development. For assistance with your issue
you will be better served by contacting the community on the user ML or slack:
https://cassandra.apache.org/_/community.html
> Direct memory OOM
> --
>
> Key: CASSANDRA-17025
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17025
> Project: Cassandra
> Issue Type: Bug
>Reporter: Zach Toews
>Priority: Normal
>
> Version: 3.11.11
> Cluster: 9 nodes (1 dc, 3 racks) on aws r5.4xlarge nodes (16 vCPU, 128GB mem)
> Heap size: 20G
> Direct memory buffer: 24G
>
> We stood up this cluster a few months ago in order to migrate an old 2.1
> cluster. Since then, after about 7-10 days a node will begin to experience
> long old gc cycles before logging many "java.lang.OutOfMemoryError: Direct
> buffer memory" exceptions. When the long gc cycles start, the entire cluster
> becomes unresponsive (our application is unable to make queries to any node).
> Restarting cassandra on the failing node resolves the problem, then we have
> to restart every other node in the cluster to prevent them from getting into
> the same state.
>
> We have attempted to:
> * Increase -XX:MaxDirectMemorySize
> * Increase -Djdk.nio.maxCachedBufferSize
> * Update cassandra from 3.11.10 to 3.11.11
> None of these have resolved the problem.
> Since the last failure, we have increased -XX:MaxDirectMemorySize again, and
> are waiting to see if that has any effect.
>
> Old gc collections from system.log:
> {noformat}
> INFO [Service Thread] 2021-10-04 15:24:04,973 GCInspector.java:285 - G1 Old
> Generation GC in 4447ms. Compressed Class Space: 6683064 -> 6677952; G1 Eden
> Space: 16777216 -> 0; G1 Old Gen: 5431375384 -> 745163912; G1 Survivor Space:
> 419430400 -> 0; Metaspace: 54716768 ->
> ...
> INFO [Service Thread] 2021-10-04 15:24:06,985 GCInspector.java:285 - G1 Old
> Generation GC in 1901ms. G1 Old Gen: 745163912 -> 745168360;
> ...
> INFO [Service Thread] 2021-10-04 15:24:09,306 GCInspector.java:285 - G1 Old
> Generation GC in 2046ms. G1 Eden Space: 528482304 -> 0; G1 Old Gen:
> 759785184 -> 761229864;
> ...
> INFO [Service Thread] 2021-10-04 15:24:14,749 GCInspector.java:285 - G1 Old
> Generation GC in 5403ms. G1 Old Gen: 761229864 -> 762168640;
> ...
> INFO [Service Thread] 2021-10-04 15:24:16,782 GCInspector.java:285 - G1 Old
> Generation GC in 1949ms. G1 Old Gen: 762168640 -> 762167640;
> ...
> INFO [Service Thread] 2021-10-04 15:25:09,406 GCInspector.java:285 - G1 Old
> Generation GC in 52302ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 762167640
> -> 762168160;
> ...
> INFO [Service Thread] 2021-10-04 15:25:15,011 GCInspector.java:285 - G1 Old
> Generation GC in 5522ms. G1 Eden Space: 192937984 -> 0; G1 Old Gen:
> 762168160 -> 770098088;
> ...
> INFO [Service Thread] 2021-10-04 15:25:31,453 GCInspector.java:285 - G1 Old
> Generation GC in 16310ms. G1 Eden Space: 201326592 -> 0; G1 Old Gen:
> 770098088 -> 769228400;
> ...
> INFO [Service Thread] 2021-10-04 15:25:33,597 GCInspector.java:285 - G1 Old
> Generation GC in 1984ms. G1 Eden Space: 352321536 -> 0; G1 Old Gen:
> 750824952 -> 751118968;
> ...
> INFO [Service Thread] 2021-10-04 15:25:50,152 GCInspector.java:285 - G1 Old
> Generation GC in 16411ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 751118968
> -> 751645056;
> {noformat}
> Example of direct memory oom from system.log
> {noformat}
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> READ messages were dropped in last 5000 ms: 2 internal and 7 cross node. Mean
> internal dropped latency: 47238 ms and Mean cross-node dropped latency: 45171
> ms
> INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
> COUNTER_MUTATION messages were dropped in last 5000 ms: 1 internal and 60
> cross node. Mean internal dropped latency: 7289 ms and Mean cross-node
> dropped latency: 9509 ms
> ERROR [CounterMutationStage-464] 2021-10-04 15:25:31,484
> JVMStabilityInspector.java:94 - OutOfMemory error letting the JVM handle the
> error:
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:695) ~[na:1.8.0_292]
> at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
> ~[na:1.8.0_292]
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
> ~[na:1.8.0_292]
> at
> org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:114)
> ~[apache-cassandra-3.11.11.jar:3.11.11]
> at
> org.apache.cassandra.utils.memory.BufferPool.access$1000(BufferPool.java:50)
> ~[apache-cassandra-3.11.11.
[jira] [Updated] (CASSANDRA-17025) Direct memory OOM
[
https://issues.apache.org/jira/browse/CASSANDRA-17025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zach Toews updated CASSANDRA-17025:
---
Description:
Version: 3.11.11
Cluster: 9 nodes (1 dc, 3 racks) on aws r5.4xlarge nodes (16 vCPU, 128GB mem)
Heap size: 20G
Direct memory buffer: 24G
We stood up this cluster a few months ago in order to migrate an old 2.0
cluster. Since then, after about 7-10 days a node will begin to experience long
old gc cycles before logging many "java.lang.OutOfMemoryError: Direct buffer
memory" exceptions. When the long gc cycles start, the entire cluster becomes
unresponsive (our application is unable to make queries to any node).
Restarting cassandra on the failing node resolves the problem, then we have to
restart every other node in the cluster to prevent them from getting into the
same state.
We have attempted to:
* Increase -XX:MaxDirectMemorySize
* Increase -Djdk.nio.maxCachedBufferSize
* Update cassandra from 3.11.10 to 3.11.11
None of these have resolved the problem.
Since the last failure, we have increased -XX:MaxDirectMemorySize again, and
are waiting to see if that has any effect.
Old gc collections from system.log:
{noformat}
INFO [Service Thread] 2021-10-04 15:24:04,973 GCInspector.java:285 - G1 Old
Generation GC in 4447ms. Compressed Class Space: 6683064 -> 6677952; G1 Eden
Space: 16777216 -> 0; G1 Old Gen: 5431375384 -> 745163912; G1 Survivor Space:
419430400 -> 0; Metaspace: 54716768 ->
...
INFO [Service Thread] 2021-10-04 15:24:06,985 GCInspector.java:285 - G1 Old
Generation GC in 1901ms. G1 Old Gen: 745163912 -> 745168360;
...
INFO [Service Thread] 2021-10-04 15:24:09,306 GCInspector.java:285 - G1 Old
Generation GC in 2046ms. G1 Eden Space: 528482304 -> 0; G1 Old Gen: 759785184
-> 761229864;
...
INFO [Service Thread] 2021-10-04 15:24:14,749 GCInspector.java:285 - G1 Old
Generation GC in 5403ms. G1 Old Gen: 761229864 -> 762168640;
...
INFO [Service Thread] 2021-10-04 15:24:16,782 GCInspector.java:285 - G1 Old
Generation GC in 1949ms. G1 Old Gen: 762168640 -> 762167640;
...
INFO [Service Thread] 2021-10-04 15:25:09,406 GCInspector.java:285 - G1 Old
Generation GC in 52302ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 762167640
-> 762168160;
...
INFO [Service Thread] 2021-10-04 15:25:15,011 GCInspector.java:285 - G1 Old
Generation GC in 5522ms. G1 Eden Space: 192937984 -> 0; G1 Old Gen: 762168160
-> 770098088;
...
INFO [Service Thread] 2021-10-04 15:25:31,453 GCInspector.java:285 - G1 Old
Generation GC in 16310ms. G1 Eden Space: 201326592 -> 0; G1 Old Gen: 770098088
-> 769228400;
...
INFO [Service Thread] 2021-10-04 15:25:33,597 GCInspector.java:285 - G1 Old
Generation GC in 1984ms. G1 Eden Space: 352321536 -> 0; G1 Old Gen: 750824952
-> 751118968;
...
INFO [Service Thread] 2021-10-04 15:25:50,152 GCInspector.java:285 - G1 Old
Generation GC in 16411ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 751118968
-> 751645056;
{noformat}
Example of direct memory oom from system.log
{noformat}
INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
READ messages were dropped in last 5000 ms: 2 internal and 7 cross node. Mean
internal dropped latency: 47238 ms and Mean cross-node dropped latency: 45171 ms
INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
COUNTER_MUTATION messages were dropped in last 5000 ms: 1 internal and 60 cross
node. Mean internal dropped latency: 7289 ms and Mean cross-node dropped
latency: 9509 ms
ERROR [CounterMutationStage-464] 2021-10-04 15:25:31,484
JVMStabilityInspector.java:94 - OutOfMemory error letting the JVM handle the
error:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:695) ~[na:1.8.0_292]
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
~[na:1.8.0_292]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
~[na:1.8.0_292]
at
org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:114)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool.access$1000(BufferPool.java:50)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.allocate(BufferPool.java:408)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:335)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool.takeFromPool(BufferPool.java:126)
~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.utils.memory.BufferPool.get(BufferPool.java:98)
~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:156)
~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.cache.ChunkCache.load(ChunkCa
[cassandra-builds] branch trunk updated: attempt to workaround force push
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git The following commit(s) were added to refs/heads/trunk by this push: new 3ecf100 attempt to workaround force push 3ecf100 is described below commit 3ecf100efc3bfba30cd701be1978527d2c7bc968 Author: Brandon Williams AuthorDate: Thu Oct 7 11:08:22 2021 -0500 attempt to workaround force push --- docker/build-debs.sh | 2 +- docker/build-rpms.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/build-debs.sh b/docker/build-debs.sh index 335121b..4484d98 100755 --- a/docker/build-debs.sh +++ b/docker/build-debs.sh @@ -29,7 +29,7 @@ fi cd $CASSANDRA_DIR git fetch -git pull +#git pull # clear and refetch tags to account for re-tagging a new sha git tag -d $(git tag) > /dev/null git fetch --tags > /dev/null 2>&1 diff --git a/docker/build-rpms.sh b/docker/build-rpms.sh index 9a2aa55..4e22b75 100755 --- a/docker/build-rpms.sh +++ b/docker/build-rpms.sh @@ -31,7 +31,7 @@ fi cd $CASSANDRA_DIR git fetch -git pull +#git pull # clear and refetch tags to account for re-tagging a new sha git tag -d $(git tag) > /dev/null git fetch --tags > /dev/null 2>&1 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17025) Direct memory OOM
Zach Toews created CASSANDRA-17025:
--
Summary: Direct memory OOM
Key: CASSANDRA-17025
URL: https://issues.apache.org/jira/browse/CASSANDRA-17025
Project: Cassandra
Issue Type: Bug
Reporter: Zach Toews
Version: 3.11.11
Cluster: 9 nodes (1 dc, 3 racks) on aws r5.4xlarge nodes (16 vCPU, 128GB mem)
Heap size: 20G
Direct memory buffer: 24G
We stood up this cluster a few months ago in order to migrate an old 2.1
cluster. Since then, after about 7-10 days a node will begin to experience long
old gc cycles before logging many "java.lang.OutOfMemoryError: Direct buffer
memory" exceptions. When the long gc cycles start, the entire cluster becomes
unresponsive (our application is unable to make queries to any node).
Restarting cassandra on the failing node resolves the problem, then we have to
restart every other node in the cluster to prevent them from getting into the
same state.
We have attempted to:
* Increase -XX:MaxDirectMemorySize
* Increase -Djdk.nio.maxCachedBufferSize
* Update cassandra from 3.11.10 to 3.11.11
None of these have resolved the problem.
Since the last failure, we have increased -XX:MaxDirectMemorySize again, and
are waiting to see if that has any effect.
Old gc collections from system.log:
{noformat}
INFO [Service Thread] 2021-10-04 15:24:04,973 GCInspector.java:285 - G1 Old
Generation GC in 4447ms. Compressed Class Space: 6683064 -> 6677952; G1 Eden
Space: 16777216 -> 0; G1 Old Gen: 5431375384 -> 745163912; G1 Survivor Space:
419430400 -> 0; Metaspace: 54716768 ->
...
INFO [Service Thread] 2021-10-04 15:24:06,985 GCInspector.java:285 - G1 Old
Generation GC in 1901ms. G1 Old Gen: 745163912 -> 745168360;
...
INFO [Service Thread] 2021-10-04 15:24:09,306 GCInspector.java:285 - G1 Old
Generation GC in 2046ms. G1 Eden Space: 528482304 -> 0; G1 Old Gen: 759785184
-> 761229864;
...
INFO [Service Thread] 2021-10-04 15:24:14,749 GCInspector.java:285 - G1 Old
Generation GC in 5403ms. G1 Old Gen: 761229864 -> 762168640;
...
INFO [Service Thread] 2021-10-04 15:24:16,782 GCInspector.java:285 - G1 Old
Generation GC in 1949ms. G1 Old Gen: 762168640 -> 762167640;
...
INFO [Service Thread] 2021-10-04 15:25:09,406 GCInspector.java:285 - G1 Old
Generation GC in 52302ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 762167640
-> 762168160;
...
INFO [Service Thread] 2021-10-04 15:25:15,011 GCInspector.java:285 - G1 Old
Generation GC in 5522ms. G1 Eden Space: 192937984 -> 0; G1 Old Gen: 762168160
-> 770098088;
...
INFO [Service Thread] 2021-10-04 15:25:31,453 GCInspector.java:285 - G1 Old
Generation GC in 16310ms. G1 Eden Space: 201326592 -> 0; G1 Old Gen: 770098088
-> 769228400;
...
INFO [Service Thread] 2021-10-04 15:25:33,597 GCInspector.java:285 - G1 Old
Generation GC in 1984ms. G1 Eden Space: 352321536 -> 0; G1 Old Gen: 750824952
-> 751118968;
...
INFO [Service Thread] 2021-10-04 15:25:50,152 GCInspector.java:285 - G1 Old
Generation GC in 16411ms. G1 Eden Space: 8388608 -> 0; G1 Old Gen: 751118968
-> 751645056;
{noformat}
Example of direct memory oom from system.log
{noformat}
INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
READ messages were dropped in last 5000 ms: 2 internal and 7 cross node. Mean
internal dropped latency: 47238 ms and Mean cross-node dropped latency: 45171 ms
INFO [ScheduledTasks:1] 2021-10-04 15:25:31,484 MessagingService.java:1246 -
COUNTER_MUTATION messages were dropped in last 5000 ms: 1 internal and 60 cross
node. Mean internal dropped latency: 7289 ms and Mean cross-node dropped
latency: 9509 ms
ERROR [CounterMutationStage-464] 2021-10-04 15:25:31,484
JVMStabilityInspector.java:94 - OutOfMemory error letting the JVM handle the
error:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:695) ~[na:1.8.0_292]
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
~[na:1.8.0_292]
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
~[na:1.8.0_292]
at
org.apache.cassandra.utils.memory.BufferPool.allocate(BufferPool.java:114)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool.access$1000(BufferPool.java:50)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.allocate(BufferPool.java:408)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool$LocalPool.access$000(BufferPool.java:335)
~[apache-cassandra-3.11.11.jar:3.11.11]
at
org.apache.cassandra.utils.memory.BufferPool.takeFromPool(BufferPool.java:126)
~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.utils.memory.BufferPool.get(BufferPool.java:98)
~[apache-cassandra-3.11.11.jar:3.11.11]
at org.apache.cassandra.cache.ChunkCache.load(ChunkCache.java:156)
~
[jira] [Updated] (CASSANDRA-16931) Improvements to @Shared annotation
[ https://issues.apache.org/jira/browse/CASSANDRA-16931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16931: --- Source Control Link: [0c444a75e79da8c157813a72d61f2b2f86e187ba|https://github.com/apache/cassandra/commit/0c444a75e79da8c157813a72d61f2b2f86e187ba] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Improvements to @Shared annotation > -- > > Key: CASSANDRA-16931 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16931 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > To support CEP-10 it is necessary to expand our functionality for selectively > sharing classes between class loaders. This ticket introduces functionality > to specify various recursive scopes of sharing, so that you may specify that > any transitive dependencies of the class or interface may be also shared. > That is, you may specify that those types necessary to implement or invoke > methods on the class or interface must also be shared, that inner classes and > types may also be shared, and that parent classes (and their inner classes > and API dependencies) may also be shared. > Additionally, validation has been added to ensure that the specified sharing > level encompasses the transitive closure of classes that will need to be > loaded to use a class. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16929) CEP-10: General Improvements (Phase 2)
[ https://issues.apache.org/jira/browse/CASSANDRA-16929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16929: --- Resolution: Fixed Status: Resolved (was: Open) > CEP-10: General Improvements (Phase 2) > -- > > Key: CASSANDRA-16929 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16929 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn >Reporter: Benedict Elliott Smith >Priority: Normal > > This is a tracking Jira for work related to CEP-10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16930) Introduce Determinism (and other) Configuration Options
[ https://issues.apache.org/jira/browse/CASSANDRA-16930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16930: --- Source Control Link: [fe9cff663b48fecdb964caaded2004e83a0c89f4|https://github.com/apache/cassandra/commit/fe9cff663b48fecdb964caaded2004e83a0c89f4] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Introduce Determinism (and other) Configuration Options > --- > > Key: CASSANDRA-16930 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16930 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Gossip >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > In order to support CEP-10 we must enable deterministic execution within > Cassandra, support disabling certain optional features that are incompatible > with simulation, and support overriding some existing features. > Some determinism will be enabled by the Simulator itself, but there are > aspects of Cassandra that must be made compliant as well. This includes: > compression system-wide, memtable overhead computation, specifying the number > of processors used for computation/decisions, specifing all cache sizes > explicitly, enabling deterministic table IDs, and supporting deterministic > node IDs for TimeUUID creation. > We also want to be able disable sstable activity tracking, and to be able to > happily ignore problems with native filesystem methods and SIGAR. We also > want to be able to override the migration delay, batch sync interval and > system auth replication factor. > For pluggability we also want to be able to specify a custom FailureDetector. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16932) Gossip Fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-16932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16932: --- Fix Version/s: 4.x Since Version: 4.0 Source Control Link: [ce2a0a28bc9ca21e1fae29f2a38448a877db06c3|https://github.com/apache/cassandra/commit/ce2a0a28bc9ca21e1fae29f2a38448a877db06c3] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Gossip Fixes > > > Key: CASSANDRA-16932 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16932 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > Testing with CEP-10 discovered faults with gossip where status updates may be > processed in an order that invalidates their application. These fixes are > necessary for simulation to run correctly, but also potentially affect gossip > time to settle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/03: [CASSANDRA-16930] CEP-10 Phase 2: Improved Configuration For Controlling Determinism
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit fe9cff663b48fecdb964caaded2004e83a0c89f4
Author: Benedict Elliott Smith
AuthorDate: Wed Jul 28 20:03:09 2021 +0100
[CASSANDRA-16930] CEP-10 Phase 2: Improved Configuration For Controlling
Determinism
Co-authored-by: Benedict Elliott Smith
Co-authored-by: Sam Tunnicliffe
Co-authored-by: Caleb Rackliffe
---
.../org/apache/cassandra/auth/AuthKeyspace.java| 5 +-
.../cassandra/concurrent/NamedThreadFactory.java | 1 +
.../concurrent/SingleThreadExecutorPlus.java | 6 +
.../cassandra/concurrent/SyncFutureTask.java | 17 ++-
.../concurrent/ThreadPoolExecutorBuilder.java | 2 +-
.../config/CassandraRelevantProperties.java| 79 ++-
src/java/org/apache/cassandra/config/Config.java | 8 ++
.../cassandra/config/DatabaseDescriptor.java | 36 +
.../org/apache/cassandra/db/ColumnFamilyStore.java | 6 +
src/java/org/apache/cassandra/db/Memtable.java | 22 ++-
.../db/commitlog/BatchCommitLogService.java| 4 +-
.../apache/cassandra/db/lifecycle/LogReplica.java | 10 +-
.../org/apache/cassandra/gms/FailureDetector.java | 3 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 147 -
.../apache/cassandra/io/sstable/Descriptor.java| 3 +-
.../cassandra/io/sstable/format/SSTableReader.java | 12 +-
.../org/apache/cassandra/io/util/PathUtils.java| 2 +-
.../locator/AbstractReplicaCollection.java | 18 +++
.../org/apache/cassandra/net/RequestCallbacks.java | 8 +-
.../apache/cassandra/schema/CompressionParams.java | 13 +-
.../cassandra/schema/MigrationCoordinator.java | 3 +-
src/java/org/apache/cassandra/schema/TableId.java | 5 +
.../org/apache/cassandra/schema/TableMetadata.java | 5 +-
.../org/apache/cassandra/service/ClientState.java | 7 +-
.../apache/cassandra/service/StorageService.java | 33 -
.../org/apache/cassandra/utils/ByteBufferUtil.java | 2 +
src/java/org/apache/cassandra/utils/Clock.java | 4 +-
.../org/apache/cassandra/utils/FBUtilities.java| 12 +-
src/java/org/apache/cassandra/utils/Hex.java | 17 +++
.../org/apache/cassandra/utils/LazyToString.java | 36 +
.../org/apache/cassandra/utils/MonotonicClock.java | 6 +-
.../org/apache/cassandra/utils/NativeLibrary.java | 43 --
.../org/apache/cassandra/utils/SigarLibrary.java | 4 +-
src/java/org/apache/cassandra/utils/UUIDGen.java | 60 -
.../apache/cassandra/utils/concurrent/Threads.java | 135 +++
.../apache/cassandra/utils/memory/HeapPool.java| 76 ++-
.../cassandra/utils/memory/MemtableAllocator.java | 2 +-
.../cassandra/utils/memory/MemtablePool.java | 3 +-
.../apache/cassandra/utils/memory/NativePool.java | 4 +-
.../apache/cassandra/utils/memory/SlabPool.java| 4 +-
.../config/DatabaseDescriptorRefTest.java | 1 +
.../unit/org/apache/cassandra/db/CellSpecTest.java | 2 +-
.../cassandra/db/ClusteringHeapSizeTest.java | 2 +-
.../org/apache/cassandra/db/NativeCellTest.java| 2 +-
.../cassandra/db/filter/ColumnFilterTest.java | 1 +
.../apache/cassandra/gms/ArrivalWindowTest.java| 8 ++
46 files changed, 780 insertions(+), 99 deletions(-)
diff --git a/src/java/org/apache/cassandra/auth/AuthKeyspace.java
b/src/java/org/apache/cassandra/auth/AuthKeyspace.java
index a57257c..2271eff 100644
--- a/src/java/org/apache/cassandra/auth/AuthKeyspace.java
+++ b/src/java/org/apache/cassandra/auth/AuthKeyspace.java
@@ -19,6 +19,7 @@ package org.apache.cassandra.auth;
import java.util.concurrent.TimeUnit;
+import org.apache.cassandra.config.CassandraRelevantProperties;
import org.apache.cassandra.cql3.statements.schema.CreateTableStatement;
import org.apache.cassandra.schema.TableId;
import org.apache.cassandra.schema.TableMetadata;
@@ -35,6 +36,8 @@ public final class AuthKeyspace
{
}
+private static final int DEFAULT_RF =
CassandraRelevantProperties.SYSTEM_AUTH_DEFAULT_RF.getInt();
+
/**
* Generation is used as a timestamp for automatic table creation on
startup.
* If you make any changes to the tables below, make sure to increment the
@@ -109,7 +112,7 @@ public final class AuthKeyspace
public static KeyspaceMetadata metadata()
{
return KeyspaceMetadata.create(SchemaConstants.AUTH_KEYSPACE_NAME,
- KeyspaceParams.simple(1),
+ KeyspaceParams.simple(DEFAULT_RF),
Tables.of(Roles, RoleMembers,
RolePermissions, ResourceRoleIndex, NetworkPermissions));
}
}
diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.j
[cassandra] branch trunk updated (5b82447 -> 0c444a7)
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 5b82447 [CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes
and version bump
new ce2a0a2 [CASSANDRA-16932] CEP-10 Phase 2: Minor Gossip Fixes
new fe9cff6 [CASSANDRA-16930] CEP-10 Phase 2: Improved Configuration For
Controlling Determinism
new 0c444a7 [CASSANDRA-16931] CEP-10 Phase 2: Improve DTest @Shared
Annotation Functionality
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../org/apache/cassandra/auth/AuthKeyspace.java| 5 +-
.../cassandra/concurrent/ExecutorBuilder.java | 4 +
.../concurrent/ExecutorBuilderFactory.java | 6 +
.../cassandra/concurrent/ExecutorFactory.java | 4 +
.../apache/cassandra/concurrent/ExecutorPlus.java | 5 +
.../apache/cassandra/concurrent/Interruptible.java | 5 +
.../concurrent/LocalAwareExecutorPlus.java | 5 +
.../LocalAwareSequentialExecutorPlus.java | 5 +
.../cassandra/concurrent/NamedThreadFactory.java | 1 +
.../cassandra/concurrent/ResizableThreadPool.java | 5 +
.../concurrent/ScheduledExecutorPlus.java | 5 +
.../concurrent/SequentialExecutorPlus.java | 6 +
.../apache/cassandra/concurrent/Shutdownable.java | 5 +
.../concurrent/SingleThreadExecutorPlus.java | 6 +
.../cassandra/concurrent/SyncFutureTask.java | 17 +-
.../apache/cassandra/concurrent/TaskFactory.java | 4 +
.../concurrent/ThreadPoolExecutorBuilder.java | 2 +-
.../config/CassandraRelevantProperties.java| 79 ++-
src/java/org/apache/cassandra/config/Config.java | 8 +
.../cassandra/config/DatabaseDescriptor.java | 36
.../cassandra/config/ParameterizedClass.java | 5 +
.../org/apache/cassandra/db/ColumnFamilyStore.java | 6 +
src/java/org/apache/cassandra/db/Memtable.java | 22 +-
.../db/commitlog/BatchCommitLogService.java| 4 +-
.../apache/cassandra/db/lifecycle/LogReplica.java | 10 +-
.../cassandra/exceptions/CassandraException.java | 5 +
.../apache/cassandra/exceptions/ExceptionCode.java | 4 +
.../exceptions/RequestExecutionException.java | 5 +
.../cassandra/exceptions/TransportException.java | 5 +
.../org/apache/cassandra/gms/FailureDetector.java | 3 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 170 ---
.../apache/cassandra/io/sstable/Descriptor.java| 3 +-
.../cassandra/io/sstable/format/SSTableReader.java | 12 +-
.../apache/cassandra/io/util/DataInputPlus.java| 4 +
.../apache/cassandra/io/util/DataOutputPlus.java | 6 +-
.../org/apache/cassandra/io/util/FileReader.java | 1 -
src/java/org/apache/cassandra/io/util/Memory.java | 2 +-
.../org/apache/cassandra/io/util/PathUtils.java| 2 +-
...ewindableDataInput.java => ReadableMemory.java} | 13 +-
.../locator/AbstractReplicaCollection.java | 18 ++
.../org/apache/cassandra/net/RequestCallbacks.java | 8 +-
.../apache/cassandra/schema/CompressionParams.java | 13 +-
.../cassandra/schema/MigrationCoordinator.java | 3 +-
src/java/org/apache/cassandra/schema/TableId.java | 5 +
.../org/apache/cassandra/schema/TableMetadata.java | 5 +-
.../org/apache/cassandra/service/ClientState.java | 7 +-
.../apache/cassandra/service/StorageService.java | 37 +++-
.../cassandra/streaming/StreamingChannel.java | 5 +
.../streaming/StreamingDataInputPlus.java | 4 +
.../streaming/StreamingDataOutputPlus.java | 5 +
.../org/apache/cassandra/utils/ByteBufferUtil.java | 2 +
src/java/org/apache/cassandra/utils/Clock.java | 6 +-
src/java/org/apache/cassandra/utils/Closeable.java | 3 +
.../org/apache/cassandra/utils/FBUtilities.java| 12 +-
src/java/org/apache/cassandra/utils/Hex.java | 17 ++
.../java/org/apache/cassandra/utils}/Isolated.java | 4 +-
.../{WrappedBoolean.java => LazyToString.java} | 30 ++-
.../org/apache/cassandra/utils/MonotonicClock.java | 8 +-
.../cassandra/utils/MonotonicClockTranslation.java | 3 +
.../org/apache/cassandra/utils/NativeLibrary.java | 43 ++--
.../java/org/apache/cassandra/utils}/Shared.java | 8 +-
.../org/apache/cassandra/utils/SigarLibrary.java | 4 +-
src/java/org/apache/cassandra/utils/UUIDGen.java | 60 +-
.../org/apache/cassandra/utils/WithResources.java | 3 +
.../cassandra/utils/concurrent/Awaitable.java | 3 +
.../cassandra/utils/concurrent/Condition.java | 4 +
.../cassandra/utils/concurrent/CountDownLatch.java | 4 +
.../apache/cassandra/utils/concurrent/Future.java | 4 +
.../apache/cassandra/utils/concurrent/Promise.java | 5
[cassandra] 01/03: [CASSANDRA-16932] CEP-10 Phase 2: Minor Gossip Fixes
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit ce2a0a28bc9ca21e1fae29f2a38448a877db06c3
Author: Benedict Elliott Smith
AuthorDate: Mon Apr 26 12:09:20 2021 +0100
[CASSANDRA-16932] CEP-10 Phase 2: Minor Gossip Fixes
* Ensure we apply new states in correct order so as not to lose TOKEN
message
* Permit replacement nodes to join the ring with lower heartbeat state than
node being replaced
---
src/java/org/apache/cassandra/gms/Gossiper.java| 23 +-
.../apache/cassandra/service/StorageService.java | 4 +++-
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 3219145..e2ebedc 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -1587,12 +1587,33 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
if (!hasMajorVersion3Nodes())
localState.removeMajorVersion3LegacyApplicationStates();
+// need to run STATUS or STATUS_WITH_PORT first to handle BOOT_REPLACE
correctly (else won't be a member, so TOKENS won't be processed)
for (Entry updatedEntry :
updatedStates)
{
+switch (updatedEntry.getKey())
+{
+default:
+continue;
+case STATUS:
+if
(localState.containsApplicationState(ApplicationState.STATUS_WITH_PORT))
+continue;
+case STATUS_WITH_PORT:
+}
+doOnChangeNotifications(addr, updatedEntry.getKey(),
updatedEntry.getValue());
+}
+
+for (Entry updatedEntry :
updatedStates)
+{
+switch (updatedEntry.getKey())
+{
+// We should have alredy handled these two states above:
+case STATUS_WITH_PORT:
+case STATUS:
+continue;
+}
// filters out legacy change notifications
// only if local state already indicates that the peer has the new
fields
if ((ApplicationState.INTERNAL_IP == updatedEntry.getKey() &&
localState.containsApplicationState(ApplicationState.INTERNAL_ADDRESS_AND_PORT))
-||(ApplicationState.STATUS == updatedEntry.getKey() &&
localState.containsApplicationState(ApplicationState.STATUS_WITH_PORT))
|| (ApplicationState.RPC_ADDRESS == updatedEntry.getKey() &&
localState.containsApplicationState(ApplicationState.NATIVE_ADDRESS_AND_PORT)))
continue;
doOnChangeNotifications(addr, updatedEntry.getKey(),
updatedEntry.getValue());
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 1638505..5c0f4bb 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -2728,7 +2728,9 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
tokensToUpdateInMetadata.add(token);
tokensToUpdateInSystemKeyspace.add(token);
}
-else if (Gossiper.instance.compareEndpointStartup(endpoint,
currentOwner) > 0)
+// Note: in test scenarios, there may not be any delta between the
heartbeat generations of the old
+// and new nodes, so we first check whether the new endpoint is
marked as a replacement for the old.
+else if
(endpoint.equals(tokenMetadata.getReplacementNode(currentOwner).orElse(null))
|| Gossiper.instance.compareEndpointStartup(endpoint, currentOwner) > 0)
{
tokensToUpdateInMetadata.add(token);
tokensToUpdateInSystemKeyspace.add(token);
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 03/03: [CASSANDRA-16931] CEP-10 Phase 2: Improve DTest @Shared Annotation Functionality
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 0c444a75e79da8c157813a72d61f2b2f86e187ba
Author: Benedict Elliott Smith
AuthorDate: Wed Jul 28 20:04:03 2021 +0100
[CASSANDRA-16931] CEP-10 Phase 2: Improve DTest @Shared Annotation
Functionality
---
.../cassandra/concurrent/ExecutorBuilder.java | 4 +
.../concurrent/ExecutorBuilderFactory.java | 6 +
.../cassandra/concurrent/ExecutorFactory.java | 4 +
.../apache/cassandra/concurrent/ExecutorPlus.java | 5 +
.../apache/cassandra/concurrent/Interruptible.java | 5 +
.../concurrent/LocalAwareExecutorPlus.java | 5 +
.../LocalAwareSequentialExecutorPlus.java | 5 +
.../cassandra/concurrent/ResizableThreadPool.java | 5 +
.../concurrent/ScheduledExecutorPlus.java | 5 +
.../concurrent/SequentialExecutorPlus.java | 6 +
.../apache/cassandra/concurrent/Shutdownable.java | 5 +
.../apache/cassandra/concurrent/TaskFactory.java | 4 +
.../cassandra/config/ParameterizedClass.java | 5 +
.../cassandra/exceptions/CassandraException.java | 5 +
.../apache/cassandra/exceptions/ExceptionCode.java | 4 +
.../exceptions/RequestExecutionException.java | 5 +
.../cassandra/exceptions/TransportException.java | 5 +
.../apache/cassandra/io/util/DataInputPlus.java| 4 +
.../apache/cassandra/io/util/DataOutputPlus.java | 6 +-
.../org/apache/cassandra/io/util/FileReader.java | 1 -
src/java/org/apache/cassandra/io/util/Memory.java | 2 +-
.../util/ReadableMemory.java} | 14 +-
.../cassandra/streaming/StreamingChannel.java | 5 +
.../streaming/StreamingDataInputPlus.java | 4 +
.../streaming/StreamingDataOutputPlus.java | 5 +
src/java/org/apache/cassandra/utils/Clock.java | 2 +
src/java/org/apache/cassandra/utils/Closeable.java | 3 +
.../java/org/apache/cassandra/utils}/Isolated.java | 4 +-
.../org/apache/cassandra/utils/MonotonicClock.java | 2 +
.../cassandra/utils/MonotonicClockTranslation.java | 3 +
.../java/org/apache/cassandra/utils}/Shared.java | 8 +-
.../org/apache/cassandra/utils/WithResources.java | 3 +
.../cassandra/utils/concurrent/Awaitable.java | 3 +
.../cassandra/utils/concurrent/Condition.java | 4 +
.../cassandra/utils/concurrent/CountDownLatch.java | 4 +
.../apache/cassandra/utils/concurrent/Future.java | 4 +
.../apache/cassandra/utils/concurrent/Promise.java | 5 +
.../cassandra/utils/concurrent/RunnableFuture.java | 5 +
.../cassandra/utils/concurrent/Semaphore.java | 3 +
.../concurrent/UncheckedInterruptedException.java | 5 +
.../cassandra/utils/concurrent/WaitQueue.java | 5 +
.../org/apache/cassandra/distributed/Cluster.java | 3 +-
.../distributed/impl/AbstractCluster.java | 239 +++--
.../cassandra/distributed/impl/Coordinator.java| 2 +-
.../distributed/impl/INodeProvisionStrategy.java | 6 +-
.../cassandra/distributed/impl/InstanceConfig.java | 2 -
.../apache/cassandra/distributed/impl/Query.java | 5 +-
.../cassandra/distributed/shared/ClusterUtils.java | 1 +
.../test/BootstrapBinaryDisabledTest.java | 2 +-
...AsHibernatingNodeWithoutReplaceAddressTest.java | 8 +-
.../upgrade/MixedModeMessageForwardTest.java | 3 +-
51 files changed, 409 insertions(+), 49 deletions(-)
diff --git a/src/java/org/apache/cassandra/concurrent/ExecutorBuilder.java
b/src/java/org/apache/cassandra/concurrent/ExecutorBuilder.java
index 89ca28a..c1d39d5 100644
--- a/src/java/org/apache/cassandra/concurrent/ExecutorBuilder.java
+++ b/src/java/org/apache/cassandra/concurrent/ExecutorBuilder.java
@@ -26,11 +26,15 @@ import java.util.concurrent.TimeUnit;
import com.google.common.annotations.VisibleForTesting;
import org.apache.cassandra.utils.JVMStabilityInspector;
+import org.apache.cassandra.utils.Shared;
+
+import static org.apache.cassandra.utils.Shared.Scope.SIMULATION;
/**
* Configure an executor before creating it.
* See {@link ThreadPoolExecutorBuilder}
*/
+@Shared(scope = SIMULATION)
public interface ExecutorBuilder
{
/**
diff --git
a/src/java/org/apache/cassandra/concurrent/ExecutorBuilderFactory.java
b/src/java/org/apache/cassandra/concurrent/ExecutorBuilderFactory.java
index f96def8..465226a 100644
--- a/src/java/org/apache/cassandra/concurrent/ExecutorBuilderFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/ExecutorBuilderFactory.java
@@ -18,6 +18,11 @@
package org.apache.cassandra.concurrent;
+import org.apache.cassandra.utils.Shared;
+
+import static org.apache.cassandra.utils.Shared.Recursive.INTERFACES;
+import static org.apache.cassandra.utils.Shared.Scope.SIMULATION;
+
/**
* Entry point for configuring and creating new executors.
*
@@ -29,6 +34,7 @@ package org.apache.cass
[jira] [Comment Edited] (CASSANDRA-16518) Node restart during joining sets protocol version to V3
[
https://issues.apache.org/jira/browse/CASSANDRA-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425598#comment-17425598
]
Brandon Williams edited comment on CASSANDRA-16518 at 10/7/21, 2:34 PM:
Your analysis looks spot on to me. I believe the check from CASSANDRA-15193 is
seeing the unknown version and capping the protocol. One possible way to solve
this may be to get of preferred_ip altogether, an idea I've floated on
CASSANDRA-16718, which is another problem regarding it. [~samt] wdyt?
was (Author: brandon.williams):
Your analysis looks spot on to me. I believe check from CASSANDRA-15193 is
seeing the unknown version and capping the protocol. One possible way to solve
this may be to get of preferred_ip altogether, an idea I've floated on
CASSANDRA-16718, which is another problem regarding it. [~samt] wdyt?
> Node restart during joining sets protocol version to V3
> ---
>
> Key: CASSANDRA-16518
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16518
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Joseph Clay
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.x
>
>
> While joining nodes to a cluster, an old node crashed. The old node was
> recovered however clients (datastax java) refused to connect to it.
> The driver error:
> {noformat}
> Detected added or restarted Cassandra host /: but ignoring it since
> it does not support the version V4 of the native protocol which is currently
> in use.{noformat}
> In the recovered node cassandra logs:
> {noformat}
> INFO o.a.c.transport.ConfiguredLimit Detected peers which do not fully
> support protocol V4. Capping max negotiable version to V3{noformat}
> I confirmed that ALL the nodes in the cluster, joining or otherwise, were
> apache-cassandra-3.11.6 so that error message was rather confusing.
> Eventually after digging through the code we got to the bottom of the issue:
> https://issues.apache.org/jira/browse/CASSANDRA-15193 adds a check for node
> version, which reverts the protocol version to V3 if any peer fails the
> version check. Joining nodes have NULL for their version in the peers table,
> which fails the version check.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16518) Node restart during joining sets protocol version to V3
[
https://issues.apache.org/jira/browse/CASSANDRA-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425598#comment-17425598
]
Brandon Williams commented on CASSANDRA-16518:
--
Your analysis looks spot on to me. I believe check from CASSANDRA-15193 is
seeing the unknown version and capping the protocol. One possible way to solve
this may be to get of preferred_ip altogether, an idea I've floated on
CASSANDRA-16718, which is another problem regarding it.
> Node restart during joining sets protocol version to V3
> ---
>
> Key: CASSANDRA-16518
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16518
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Joseph Clay
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.x
>
>
> While joining nodes to a cluster, an old node crashed. The old node was
> recovered however clients (datastax java) refused to connect to it.
> The driver error:
> {noformat}
> Detected added or restarted Cassandra host /: but ignoring it since
> it does not support the version V4 of the native protocol which is currently
> in use.{noformat}
> In the recovered node cassandra logs:
> {noformat}
> INFO o.a.c.transport.ConfiguredLimit Detected peers which do not fully
> support protocol V4. Capping max negotiable version to V3{noformat}
> I confirmed that ALL the nodes in the cluster, joining or otherwise, were
> apache-cassandra-3.11.6 so that error message was rather confusing.
> Eventually after digging through the code we got to the bottom of the issue:
> https://issues.apache.org/jira/browse/CASSANDRA-15193 adds a check for node
> version, which reverts the protocol version to V3 if any peer fails the
> version check. Joining nodes have NULL for their version in the peers table,
> which fails the version check.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16518) Node restart during joining sets protocol version to V3
[
https://issues.apache.org/jira/browse/CASSANDRA-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425598#comment-17425598
]
Brandon Williams edited comment on CASSANDRA-16518 at 10/7/21, 2:31 PM:
Your analysis looks spot on to me. I believe check from CASSANDRA-15193 is
seeing the unknown version and capping the protocol. One possible way to solve
this may be to get of preferred_ip altogether, an idea I've floated on
CASSANDRA-16718, which is another problem regarding it. [~samt] wdyt?
was (Author: brandon.williams):
Your analysis looks spot on to me. I believe check from CASSANDRA-15193 is
seeing the unknown version and capping the protocol. One possible way to solve
this may be to get of preferred_ip altogether, an idea I've floated on
CASSANDRA-16718, which is another problem regarding it.
> Node restart during joining sets protocol version to V3
> ---
>
> Key: CASSANDRA-16518
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16518
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Joseph Clay
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.x
>
>
> While joining nodes to a cluster, an old node crashed. The old node was
> recovered however clients (datastax java) refused to connect to it.
> The driver error:
> {noformat}
> Detected added or restarted Cassandra host /: but ignoring it since
> it does not support the version V4 of the native protocol which is currently
> in use.{noformat}
> In the recovered node cassandra logs:
> {noformat}
> INFO o.a.c.transport.ConfiguredLimit Detected peers which do not fully
> support protocol V4. Capping max negotiable version to V3{noformat}
> I confirmed that ALL the nodes in the cluster, joining or otherwise, were
> apache-cassandra-3.11.6 so that error message was rather confusing.
> Eventually after digging through the code we got to the bottom of the issue:
> https://issues.apache.org/jira/browse/CASSANDRA-15193 adds a check for node
> version, which reverts the protocol version to V3 if any peer fails the
> version check. Joining nodes have NULL for their version in the peers table,
> which fails the version check.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 06/06: [CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes and version bump
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 5b82447098ad634900f8892297ef82083eadb954
Author: Benedict Elliott Smith
AuthorDate: Thu Jul 29 18:06:32 2021 +0100
[CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes and version bump
---
build.xml | 2 +-
.../cassandra/concurrent/ExecutorFactory.java | 2 +-
.../cql3/functions/JavaBasedUDFunction.java| 1 +
.../cassandra/cql3/functions/UDFunction.java | 2 +-
.../org/apache/cassandra/gms/EndpointState.java| 5 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 10 +
.../org/apache/cassandra/gms/HeartBeatState.java | 5 +-
.../cassandra/io/sstable/format/SSTableReader.java | 2 +
src/java/org/apache/cassandra/io/util/File.java| 8 +-
.../apache/cassandra/triggers/TriggerExecutor.java | 2 +-
.../org/apache/cassandra/distributed/Cluster.java | 1 -
.../distributed/impl/AbstractCluster.java | 103 +-
.../cassandra/distributed/impl/Coordinator.java| 4 +-
.../impl/DelegatingInvokableInstance.java | 62
.../impl/DirectStreamingConnectionFactory.java | 386 +
.../cassandra/distributed/impl/Instance.java | 247 ++---
.../cassandra/distributed/impl/InstanceConfig.java | 22 +-
.../cassandra/distributed/impl/InstanceKiller.java | 1 +
.../distributed/impl/IsolatedExecutor.java | 95 +++--
.../apache/cassandra/distributed/impl/Query.java | 110 ++
.../distributed/impl/UnsafeGossipHelper.java | 265 ++
.../apache/cassandra/distributed/test/CASTest.java | 32 +-
.../distributed/upgrade/UpgradeTestBase.java | 3 +-
.../cassandra/db/RangeTombstoneListTest.java | 3 +-
.../utils/concurrent/ImmediateFutureTest.java | 3 +-
25 files changed, 1141 insertions(+), 235 deletions(-)
diff --git a/build.xml b/build.xml
index f5acaff..b8eabb8 100644
--- a/build.xml
+++ b/build.xml
@@ -532,7 +532,7 @@
-
+
diff --git a/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
b/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
index 9c7a2cf..52ba94a 100644
--- a/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
@@ -146,7 +146,7 @@ public interface ExecutorFactory extends
ExecutorBuilderFactory.Jmxable
{
-private static final FileSystem filesystem = FileSystems.getDefault();
+private static FileSystem filesystem = FileSystems.getDefault();
+
public enum WriteMode { OVERWRITE, APPEND }
public static String pathSeparator()
@@ -604,5 +605,10 @@ public class File implements Comparable
throw new IllegalStateException("Cannot read from an empty path");
return path;
}
+
+public static void unsafeSetFilesystem(FileSystem fs)
+{
+filesystem = fs;
+}
}
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 298ac56..c76c6bd 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -44,7 +44,7 @@ public class TriggerExecutor
public static final TriggerExecutor instance = new TriggerExecutor();
private final Map cachedTriggers =
Maps.newConcurrentMap();
-private final ClassLoader parent =
Thread.currentThread().getContextClassLoader();
+private final ClassLoader parent = TriggerExecutor.class.getClassLoader();
private volatile ClassLoader customClassLoader;
private TriggerExecutor()
diff --git a/test/distributed/org/apache/cassandra/distributed/Cluster.java
b/test/distributed/org/apache/cassandra/distributed/Cluster.java
index a613fc5..05ea799 100644
--- a/test/distributed/org/apache/cassandra/distributed/Cluster.java
+++ b/test/distributed/org/apache/cassandra/distributed/Cluster.java
@@ -34,7 +34,6 @@ import org.apache.cassandra.distributed.shared.Versions;
@Shared
public class Cluster extends AbstractCluster
{
-
private Cluster(Builder builder)
{
super(builder);
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
index eca9088..2f146bf 100644
---
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
@@ -20,6 +20,9 @@ package org.apache.cassandra.distributed.impl;
import java.lang.annotation.Annotation;
import java.net.InetSocketAddress;
+import java.nio.file.FileSystem;
+import java.nio.file.Files;
+import java.nio.fil
[cassandra] 04/06: [CASSANDRA-16928] CEP-10 Phase 1: InetAddressAndPort extends InetSocketAddress
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit aae7e8b07c84476de893b473a13cdc6d9e260573
Author: Benedict Elliott Smith
AuthorDate: Fri Apr 16 12:01:25 2021 +0100
[CASSANDRA-16928] CEP-10 Phase 1: InetAddressAndPort extends
InetSocketAddress
patch by Benedict; reviewed by Sam Tunnicliffe, Caleb Rackliffe and Aleksei
Zotov for CASSANDRA-16928
---
.../org/apache/cassandra/audit/AuditLogEntry.java | 6 +-
.../org/apache/cassandra/db/SystemKeyspace.java| 37 +++
.../db/virtual/InternodeInboundTable.java | 2 +-
.../db/virtual/InternodeOutboundTable.java | 2 +-
.../dht/tokenallocator/OfflineTokenAllocator.java | 2 +-
.../cassandra/locator/DynamicEndpointSnitch.java | 2 +-
.../cassandra/locator/Ec2MultiRegionSnitch.java| 2 +-
.../apache/cassandra/locator/IEndpointSnitch.java | 6 ++
.../cassandra/locator/InetAddressAndPort.java | 113 -
.../cassandra/locator/RackInferringSnitch.java | 4 +-
.../cassandra/net/InboundConnectionInitiator.java | 2 +-
.../cassandra/net/InboundConnectionSettings.java | 8 +-
.../cassandra/net/OutboundConnectionInitiator.java | 4 +-
.../cassandra/net/OutboundConnectionSettings.java | 2 +-
.../org/apache/cassandra/net/SocketFactory.java| 2 +-
.../repair/SystemDistributedKeyspace.java | 2 +-
.../repair/consistent/LocalSessionInfo.java| 2 +-
.../cassandra/repair/consistent/LocalSessions.java | 6 +-
.../cassandra/schema/SchemaAnnouncementEvent.java | 4 +-
.../apache/cassandra/service/StorageService.java | 29 +++---
.../cassandra/service/TruncateResponseHandler.java | 2 +-
.../service/reads/repair/ReadRepairEvent.java | 4 +-
.../management/ProgressInfoCompositeData.java | 4 +-
.../SessionCompleteEventCompositeData.java | 4 +-
.../management/SessionInfoCompositeData.java | 8 +-
.../cassandra/tools/nodetool/HostStatWithPort.java | 4 +-
.../apache/cassandra/tracing/TraceKeyspace.java| 8 +-
src/java/org/apache/cassandra/transport/Event.java | 10 +-
.../org/apache/cassandra/transport/Server.java | 2 +-
.../cassandra/transport/messages/ErrorMessage.java | 4 +-
.../org/apache/cassandra/utils/FBUtilities.java| 2 +-
src/java/org/apache/cassandra/utils/Mx4jTool.java | 2 +-
src/java/org/apache/cassandra/utils/UUIDGen.java | 2 +-
.../distributed/impl/DistributedTestSnitch.java| 2 +-
.../cassandra/distributed/impl/Instance.java | 4 +-
.../cassandra/distributed/test/StreamingTest.java | 8 +-
.../cassandra/audit/AuditLoggerAuthTest.java | 4 +-
.../cassandra/dht/RangeFetchMapCalculatorTest.java | 2 +-
.../org/apache/cassandra/gms/GossiperTest.java | 6 +-
.../locator/InetAddressAndPortSerializerTest.java | 4 +-
.../cassandra/locator/InetAddressAndPortTest.java | 16 +--
.../apache/cassandra/net/ForwardingInfoTest.java | 4 +-
.../apache/cassandra/net/MessagingServiceTest.java | 12 +--
.../org/apache/cassandra/repair/RepairJobTest.java | 8 +-
.../service/WriteResponseHandlerTest.java | 2 +-
.../service/WriteResponseHandlerTransientTest.java | 2 +-
.../cassandra/transport/CQLUserAuditTest.java | 6 +-
47 files changed, 200 insertions(+), 173 deletions(-)
diff --git a/src/java/org/apache/cassandra/audit/AuditLogEntry.java
b/src/java/org/apache/cassandra/audit/AuditLogEntry.java
index 3a015c5..02db076 100644
--- a/src/java/org/apache/cassandra/audit/AuditLogEntry.java
+++ b/src/java/org/apache/cassandra/audit/AuditLogEntry.java
@@ -76,10 +76,10 @@ public class AuditLogEntry
StringBuilder builder = new StringBuilder(100);
builder.append("user:").append(user)
.append("|host:").append(host)
- .append("|source:").append(source.address);
-if (source.port > 0)
+ .append("|source:").append(source.getAddress());
+if (source.getPort() > 0)
{
-builder.append("|port:").append(source.port);
+builder.append("|port:").append(source.getPort());
}
builder.append("|timestamp:").append(timestamp)
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index a7cbe40..7e8c2b5 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.db;
import java.io.IOError;
import java.io.IOException;
import java.net.InetAddress;
+import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.time.Instant;
import java.util.*;
@@ -36,7 +37,7 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.SetMultimap;
import com.google.common.collect.Sets;
import
[cassandra] branch trunk updated (f5fb1b0 -> 5b82447)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. omit f5fb1b0 [CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes and version bump omit 50278db [CASSANDRA-16927] CEP-10 Phase 1: Refactor Streaming omit e2721b0 [CASSANDRA-16928] CEP-10 Phase 1: InetAddressAndPort extends InetSocketAddress omit 767f98e [CASSANDRA-16926] CEP-10 Phase 1: Mockable Filesystem omit e215d2a [CASSANDRA-16925] CEP-10 Phase 1: Mockable Task Execution omit 5c3a05a [CASSANDRA-16924] CEP-10 Phase 1: Mockable Blocking Concurrency Primitives new e5b92e1 [CASSANDRA-16924] CEP-10 Phase 1: Mockable Blocking Concurrency Primitives new be1f050 [CASSANDRA-16925] CEP-10 Phase 1: Mockable Task Execution new 6a1d9de [CASSANDRA-16926] CEP-10 Phase 1: Mockable Filesystem new aae7e8b [CASSANDRA-16928] CEP-10 Phase 1: InetAddressAndPort extends InetSocketAddress new 6812fdd [CASSANDRA-16927] CEP-10 Phase 1: Refactor Streaming new 5b82447 [CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes and version bump This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (f5fb1b0) \ N -- N -- N refs/heads/trunk (5b82447) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16983) Separating CQLSH credentials from the cqlshrc file
[ https://issues.apache.org/jira/browse/CASSANDRA-16983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425531#comment-17425531 ] Brandon Williams commented on CASSANDRA-16983: -- bq. what do you think about the complete removal of Windows from cqlsh? I think Dinesh makes a good point, users interested in Windows support are best served by the project as a whole via WSL, so cqlsh should follow suit. > Separating CQLSH credentials from the cqlshrc file > -- > > Key: CASSANDRA-16983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16983 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Bowen Song >Assignee: Bowen Song >Priority: Normal > Labels: lhf > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, the CQLSH tool accepts credentials (username & password) from the > following 3 places: > 1. the command line parameter "-p" > 2. the cqlshrc file > 3. prompt the user > This is not ideal. > Credentials in the command line is a security risk, because it could be see > by other users on a shared system. > The cqlshrc file is better, but still not good enough. Because the cqlshrc > file is a config file, it's often acceptable to have it as a world readable > file, and share it with other users. It also prevents user from having > multiple sets of credentials, either for the same Cassandra cluster or > different clusters. > To improve the security of CQLSH and make it secure by design, I purpose the > following changes: > * Warn the user if a password is giving in the command line, and recommend > them to use a credential file instead > * Warn the user if credentials are present in the cqlshrc file and the > cqlshrc file is not secure (e.g.: world readable or owned by a different user) > * Deprecate credentials in the cqlshrc, and recommend the user to move them > to a separate credential file. The aim is to not break anything at the > moment, but eventually stop accepting credentials from the cqlshrc file. > * Reject the credentials file if it's not secure, and tell the user how to > secure it. Optionally, prompt the user for password if it's an interactive > session. (Think how does OpenSSH handle insecure credential files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17016) Allow reverse iteration of resources during permissions checking
[
https://issues.apache.org/jira/browse/CASSANDRA-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksei Zotov updated CASSANDRA-17016:
--
Authors: Aleksei Zotov, Josh McKenzie (was: Josh McKenzie)
> Allow reverse iteration of resources during permissions checking
>
>
> Key: CASSANDRA-17016
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17016
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
>
> When we perform authz checks for AuthenticatedUser on a given resource, we
> traverse the resource hierarchy until we either find the required permission
> or we exhaust the traversal. This goes from most specific resource (i.e.
> iResource we're interested in) to the broadest (the root for that iResource
> type).
> Since permissions are a whitelist it isn't possible for a permission granted
> on a more specific resource to be negated or overridden by a grant on a
> resource lower in the hierarchy towards the root. For example, for
> DataResource, the hierarchy goes:
> {code:java}
> root -> keyspace -> table{code}
> So for instance if we:
>
> {code:java}
> GRANT ALL ON KEYSPACE ks TO admin_user;
> {code}
> It is impossible to grant admin_user any set of permissions more restrictive
> than ALL on ks.
> When a request comes in for a user with permissions on a ks for example, you
> can have a path of:
> # Validate SELECT on t1
> # Then check for SELECT on ks
> # Then check for permissions on 'root'
> These unnecessary lookups can negate some of the benefits of caching (and
> pre-warming, which another ticket is in flight for), and lead to issues on
> bounces with timeouts.
> Additionally, the permissions cache ends up storing far more entries than
> necessary, as a subsequent request for ks.t2 by user_x will go through the
> same process and create a second cache entry etc.
> So all this said - this is something we should allow operators to opt-in to;
> impact and performance is going to be highly dependent on the pattern of
> authentication granting and usage on a cluster.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17016) Allow reverse iteration of resources during permissions checking
[
https://issues.apache.org/jira/browse/CASSANDRA-17016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425530#comment-17425530
]
Aleksei Zotov commented on CASSANDRA-17016:
---
I feel some small improvements can be made, so I posted a few comments to the
PR.
> Allow reverse iteration of resources during permissions checking
>
>
> Key: CASSANDRA-17016
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17016
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
>
> When we perform authz checks for AuthenticatedUser on a given resource, we
> traverse the resource hierarchy until we either find the required permission
> or we exhaust the traversal. This goes from most specific resource (i.e.
> iResource we're interested in) to the broadest (the root for that iResource
> type).
> Since permissions are a whitelist it isn't possible for a permission granted
> on a more specific resource to be negated or overridden by a grant on a
> resource lower in the hierarchy towards the root. For example, for
> DataResource, the hierarchy goes:
> {code:java}
> root -> keyspace -> table{code}
> So for instance if we:
>
> {code:java}
> GRANT ALL ON KEYSPACE ks TO admin_user;
> {code}
> It is impossible to grant admin_user any set of permissions more restrictive
> than ALL on ks.
> When a request comes in for a user with permissions on a ks for example, you
> can have a path of:
> # Validate SELECT on t1
> # Then check for SELECT on ks
> # Then check for permissions on 'root'
> These unnecessary lookups can negate some of the benefits of caching (and
> pre-warming, which another ticket is in flight for), and lead to issues on
> bounces with timeouts.
> Additionally, the permissions cache ends up storing far more entries than
> necessary, as a subsequent request for ks.t2 by user_x will go through the
> same process and create a second cache entry etc.
> So all this said - this is something we should allow operators to opt-in to;
> impact and performance is going to be highly dependent on the pattern of
> authentication granting and usage on a cluster.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16983) Separating CQLSH credentials from the cqlshrc file
[ https://issues.apache.org/jira/browse/CASSANDRA-16983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425513#comment-17425513 ] Stefan Miklosovic commented on CASSANDRA-16983: --- That's the vibe I am getting from [~djoshi]. I am on the fence here, I personaly do not mind to drop it completely and a user would have to use WSL if she needs that. We need some arbiter in this matter. [~brandon.williams] what do you think about the complete removal of Windows from cqlsh? > Separating CQLSH credentials from the cqlshrc file > -- > > Key: CASSANDRA-16983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16983 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Bowen Song >Assignee: Bowen Song >Priority: Normal > Labels: lhf > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, the CQLSH tool accepts credentials (username & password) from the > following 3 places: > 1. the command line parameter "-p" > 2. the cqlshrc file > 3. prompt the user > This is not ideal. > Credentials in the command line is a security risk, because it could be see > by other users on a shared system. > The cqlshrc file is better, but still not good enough. Because the cqlshrc > file is a config file, it's often acceptable to have it as a world readable > file, and share it with other users. It also prevents user from having > multiple sets of credentials, either for the same Cassandra cluster or > different clusters. > To improve the security of CQLSH and make it secure by design, I purpose the > following changes: > * Warn the user if a password is giving in the command line, and recommend > them to use a credential file instead > * Warn the user if credentials are present in the cqlshrc file and the > cqlshrc file is not secure (e.g.: world readable or owned by a different user) > * Deprecate credentials in the cqlshrc, and recommend the user to move them > to a separate credential file. The aim is to not break anything at the > moment, but eventually stop accepting credentials from the cqlshrc file. > * Reject the credentials file if it's not secure, and tell the user how to > secure it. Optionally, prompt the user for password if it's an interactive > session. (Think how does OpenSSH handle insecure credential files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16983) Separating CQLSH credentials from the cqlshrc file
[ https://issues.apache.org/jira/browse/CASSANDRA-16983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425508#comment-17425508 ] Bowen Song commented on CASSANDRA-16983: [~stefan.miklosovic] Are you saying that CQLSH should drop the support for Windows? Because if I remove all Windows related stuff from my PR, credentials file will not work on Windows for sure. > Separating CQLSH credentials from the cqlshrc file > -- > > Key: CASSANDRA-16983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16983 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Bowen Song >Assignee: Bowen Song >Priority: Normal > Labels: lhf > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, the CQLSH tool accepts credentials (username & password) from the > following 3 places: > 1. the command line parameter "-p" > 2. the cqlshrc file > 3. prompt the user > This is not ideal. > Credentials in the command line is a security risk, because it could be see > by other users on a shared system. > The cqlshrc file is better, but still not good enough. Because the cqlshrc > file is a config file, it's often acceptable to have it as a world readable > file, and share it with other users. It also prevents user from having > multiple sets of credentials, either for the same Cassandra cluster or > different clusters. > To improve the security of CQLSH and make it secure by design, I purpose the > following changes: > * Warn the user if a password is giving in the command line, and recommend > them to use a credential file instead > * Warn the user if credentials are present in the cqlshrc file and the > cqlshrc file is not secure (e.g.: world readable or owned by a different user) > * Deprecate credentials in the cqlshrc, and recommend the user to move them > to a separate credential file. The aim is to not break anything at the > moment, but eventually stop accepting credentials from the cqlshrc file. > * Reject the credentials file if it's not secure, and tell the user how to > secure it. Optionally, prompt the user for password if it's an interactive > session. (Think how does OpenSSH handle insecure credential files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16932) Gossip Fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-16932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16932: --- Status: Ready to Commit (was: Review In Progress) > Gossip Fixes > > > Key: CASSANDRA-16932 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16932 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Testing with CEP-10 discovered faults with gossip where status updates may be > processed in an order that invalidates their application. These fixes are > necessary for simulation to run correctly, but also potentially affect gossip > time to settle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16932) Gossip Fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-16932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16932: --- Test and Documentation Plan: Simulator discovered this flaw, and is being provided in a follow-up ticket Status: Patch Available (was: Open) > Gossip Fixes > > > Key: CASSANDRA-16932 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16932 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Testing with CEP-10 discovered faults with gossip where status updates may be > processed in an order that invalidates their application. These fixes are > necessary for simulation to run correctly, but also potentially affect gossip > time to settle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16932) Gossip Fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-16932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16932: --- Reviewers: Sam Tunnicliffe, Benedict Elliott Smith (was: Benedict Elliott Smith, Sam Tunnicliffe) Sam Tunnicliffe, Benedict Elliott Smith (was: Sam Tunnicliffe) Status: Review In Progress (was: Patch Available) > Gossip Fixes > > > Key: CASSANDRA-16932 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16932 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Testing with CEP-10 discovered faults with gossip where status updates may be > processed in an order that invalidates their application. These fixes are > necessary for simulation to run correctly, but also potentially affect gossip > time to settle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated (c2256c7 -> 2d0b7d3)
This is an automated email from the ASF dual-hosted git repository. benedict pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git. from c2256c7 Unify sstableverify test across versions and fix, use single node. add 6016da3 CASSANDRA-16928: fix byteman compilation add 2d0b7d3 CASSANDRA 16927: 4.0 uses SO_KEEPALIVE so no need to monitor keep-alive messages No new revisions were added by this update. Summary of changes: bootstrap_test.py | 9 + byteman/request_verb_timing.btm| 4 ++-- upgrade_tests/upgrade_through_versions_test.py | 6 -- 3 files changed, 11 insertions(+), 8 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17013) CEP-10: dtest-api improvements
[ https://issues.apache.org/jira/browse/CASSANDRA-17013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-17013: --- Fix Version/s: 4.x Source Control Link: [f5fb1b0bd32b5dc7da13ec66d43acbdad7fe9dbf|https://github.com/apache/cassandra/commit/f5fb1b0bd32b5dc7da13ec66d43acbdad7fe9dbf] Resolution: Fixed Status: Resolved (was: Ready to Commit) > CEP-10: dtest-api improvements > -- > > Key: CASSANDRA-17013 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17013 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Labels: pull-request-available > Fix For: 4.x > > > Introduce new dtest-api features to support CEP-10, including byte weaving > support, support for Path objects specifying location, and other minor > ergonomic improvements -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16927) Mockable Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16927: --- Source Control Link: [50278db72bda760c6fcfee5007e630aed4870658|https://github.com/apache/cassandra/commit/50278db72bda760c6fcfee5007e630aed4870658] (was: [e2721b05ab44183baa2cd4bde9b20862c0eadc90|https://github.com/apache/cassandra/commit/e2721b05ab44183baa2cd4bde9b20862c0eadc90]) > Mockable Streaming > -- > > Key: CASSANDRA-16927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16927 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > To support CEP-10 it is necessary to support alternative streaming > implementations, so that execution may be controlled. The ticket encompasses > two sub-tickets: firstly we make InetAddressAndPort extend InetAddress so > that we may 2) introduce a cross-classloader API for establishing a streaming > connection and sending data over it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16927) Mockable Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16927: --- Fix Version/s: 4.x Source Control Link: [e2721b05ab44183baa2cd4bde9b20862c0eadc90|https://github.com/apache/cassandra/commit/e2721b05ab44183baa2cd4bde9b20862c0eadc90] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Mockable Streaming > -- > > Key: CASSANDRA-16927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16927 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > To support CEP-10 it is necessary to support alternative streaming > implementations, so that execution may be controlled. The ticket encompasses > two sub-tickets: firstly we make InetAddressAndPort extend InetAddress so > that we may 2) introduce a cross-classloader API for establishing a streaming > connection and sending data over it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16928) InetAddressAndPort to extend InetAddress
[ https://issues.apache.org/jira/browse/CASSANDRA-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16928: --- Fix Version/s: 4.x Source Control Link: [e2721b05ab44183baa2cd4bde9b20862c0eadc90|https://github.com/apache/cassandra/commit/e2721b05ab44183baa2cd4bde9b20862c0eadc90] Resolution: Fixed Status: Resolved (was: Ready to Commit) > InetAddressAndPort to extend InetAddress > > > Key: CASSANDRA-16928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16928 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > Logically InetAddressAndPort encapsulates the same information as a simple > InetAddress. Importantly, InetAddressAndPort cannot easily be unpicked from > its dependencies that prohibit it being shared between classloaders. So our > in-jvm dtest API instead uses SocketAddress. To support a clean and efficient > Streaming API we need to be able to treat InetAddressAndPort as such a > cross-classloader type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16926) Mockable FileSystem
[ https://issues.apache.org/jira/browse/CASSANDRA-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16926: --- Fix Version/s: 4.x Source Control Link: [767f98e1b7e6a6f2662406cf6d472b1c96150121|https://github.com/apache/cassandra/commit/767f98e1b7e6a6f2662406cf6d472b1c96150121] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Mockable FileSystem > --- > > Key: CASSANDRA-16926 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16926 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > To support CEP-10 it is necessary to support alternative file system > implementations so that execution may be consistent between hosts. This > ticket introduces a new File API backed by Path objects, and all File usages > are ported to it. This necessitates the migration away from > RandomAccessReader, but otherwise the change is mostly straight-forward, if > quite expansive. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16925) Task Execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16925: --- Fix Version/s: 4.x Source Control Link: [e215d2a5b25991b47cec66cc1a970d835b89005b|https://github.com/apache/cassandra/commit/e215d2a5b25991b47cec66cc1a970d835b89005b] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Task Execution > -- > > Key: CASSANDRA-16925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16925 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > To support CEP-10 it is necessary to support alternative executor > implementations that may be externally controlled. This ticket introduces an > ExecutorFactory, and all executor creation is migrated to this facility. The > codebase’s use of executors is inconsistent, and relatedly there are > divergent Future APIs in use, with Netty, Guava and Java APIs in use in > various places. To improve consistency we introduce our own Future API that > implements all of the above, as well as providing other additional ergonomic > improvements. We introduce a new ExecutorPlus API that extends Executor to > supply this extended Future API. All executors and futures are ported to > these new APIs to improve coherency in the codebase, and to support > consistently mocking these implementations. > DebuggableThreadPoolExecutor and its hierarchy is replaced with > ThreadPoolExecutorBase, ThreadPoolExecutorPlus and extending classes with > consistent names, implementations and configurability. > Additionally the ExecutorFactory takes over the allocation of new threads, > and provides a new InfiniteLoopExecutorfacility for common paradigms. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16924) Blocking Concurrency Primitives
[ https://issues.apache.org/jira/browse/CASSANDRA-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16924: --- Fix Version/s: 4.x Source Control Link: [5c3a05a5787f7ab0b46e3a6401d64f376f6c9d9f|https://github.com/apache/cassandra/commit/5c3a05a5787f7ab0b46e3a6401d64f376f6c9d9f] Resolution: Fixed Status: Resolved (was: Ready to Commit) > Blocking Concurrency Primitives > --- > > Key: CASSANDRA-16924 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16924 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > To support CEP-10 it is necessary to support alternative implementations of > the blocking concurrency primitives we use on the project. At the same time, > the project is very inconsistent in its usage of these APIs, so this work > includes a number of improvements to the coherency of the codebase. > This ticket introduces new abstractions and standardises old ones, migrating > all blocking concurrency operations besides Futures returned by Executors to > these new APIs. This includes a migration of SimpleCondition to a new > Conditioninterface, new CountDownLatch and Semaphore interfaces, and new > static factory methods for creating these objects (as well as WaitQueue) that > can be intercepted by byte weaving. Additionally the internal Netty Future > implementation is improved to support more general use (though this is only > fully realised in a later ticket), OpOrder is improved to more easily support > mocking WaitQueue. > Finally we standardise the propagation of InterruptedExecption, with the new > UncheckedInterruptedException, so that simulations may be terminated cleanly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16922) CEP-10: Major Prerequisites (Phase 1)
[ https://issues.apache.org/jira/browse/CASSANDRA-16922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benedict Elliott Smith updated CASSANDRA-16922: --- Fix Version/s: 4.x Source Control Link: [f5fb1b0bd32b5dc7da13ec66d43acbdad7fe9dbf|https://github.com/apache/cassandra/commit/f5fb1b0bd32b5dc7da13ec66d43acbdad7fe9dbf] Resolution: Fixed Status: Resolved (was: Ready to Commit) > CEP-10: Major Prerequisites (Phase 1) > - > > Key: CASSANDRA-16922 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16922 > Project: Cassandra > Issue Type: Improvement > Components: Test/burn >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.x > > > This is a tracking Jira for major pre-requisite system refactors for CEP-10 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 06/06: [CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes and version bump
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f5fb1b0bd32b5dc7da13ec66d43acbdad7fe9dbf
Author: Benedict Elliott Smith
AuthorDate: Thu Jul 29 18:06:32 2021 +0100
[CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes and version bump
---
build.xml | 2 +-
.../cassandra/concurrent/ExecutorFactory.java | 2 +-
.../cql3/functions/JavaBasedUDFunction.java| 1 +
.../cassandra/cql3/functions/UDFunction.java | 2 +-
.../org/apache/cassandra/gms/EndpointState.java| 5 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 10 +
.../org/apache/cassandra/gms/HeartBeatState.java | 5 +-
.../cassandra/io/sstable/format/SSTableReader.java | 2 +
src/java/org/apache/cassandra/io/util/File.java| 8 +-
.../apache/cassandra/triggers/TriggerExecutor.java | 2 +-
.../org/apache/cassandra/distributed/Cluster.java | 1 -
.../distributed/impl/AbstractCluster.java | 103 +-
.../cassandra/distributed/impl/Coordinator.java| 4 +-
.../impl/DelegatingInvokableInstance.java | 62
.../impl/DirectStreamingConnectionFactory.java | 386 +
.../cassandra/distributed/impl/Instance.java | 247 ++---
.../cassandra/distributed/impl/InstanceConfig.java | 22 +-
.../cassandra/distributed/impl/InstanceKiller.java | 1 +
.../distributed/impl/IsolatedExecutor.java | 95 +++--
.../apache/cassandra/distributed/impl/Query.java | 110 ++
.../distributed/impl/UnsafeGossipHelper.java | 265 ++
.../apache/cassandra/distributed/test/CASTest.java | 32 +-
.../distributed/upgrade/UpgradeTestBase.java | 3 +-
.../cassandra/db/RangeTombstoneListTest.java | 3 +-
.../utils/concurrent/ImmediateFutureTest.java | 3 +-
25 files changed, 1141 insertions(+), 235 deletions(-)
diff --git a/build.xml b/build.xml
index f5acaff..b8eabb8 100644
--- a/build.xml
+++ b/build.xml
@@ -532,7 +532,7 @@
-
+
diff --git a/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
b/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
index 9c7a2cf..52ba94a 100644
--- a/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/ExecutorFactory.java
@@ -146,7 +146,7 @@ public interface ExecutorFactory extends
ExecutorBuilderFactory.Jmxable
{
-private static final FileSystem filesystem = FileSystems.getDefault();
+private static FileSystem filesystem = FileSystems.getDefault();
+
public enum WriteMode { OVERWRITE, APPEND }
public static String pathSeparator()
@@ -604,5 +605,10 @@ public class File implements Comparable
throw new IllegalStateException("Cannot read from an empty path");
return path;
}
+
+public static void unsafeSetFilesystem(FileSystem fs)
+{
+filesystem = fs;
+}
}
diff --git a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
index 298ac56..c76c6bd 100644
--- a/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
+++ b/src/java/org/apache/cassandra/triggers/TriggerExecutor.java
@@ -44,7 +44,7 @@ public class TriggerExecutor
public static final TriggerExecutor instance = new TriggerExecutor();
private final Map cachedTriggers =
Maps.newConcurrentMap();
-private final ClassLoader parent =
Thread.currentThread().getContextClassLoader();
+private final ClassLoader parent = TriggerExecutor.class.getClassLoader();
private volatile ClassLoader customClassLoader;
private TriggerExecutor()
diff --git a/test/distributed/org/apache/cassandra/distributed/Cluster.java
b/test/distributed/org/apache/cassandra/distributed/Cluster.java
index a613fc5..05ea799 100644
--- a/test/distributed/org/apache/cassandra/distributed/Cluster.java
+++ b/test/distributed/org/apache/cassandra/distributed/Cluster.java
@@ -34,7 +34,6 @@ import org.apache.cassandra.distributed.shared.Versions;
@Shared
public class Cluster extends AbstractCluster
{
-
private Cluster(Builder builder)
{
super(builder);
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
index eca9088..2f146bf 100644
---
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
@@ -20,6 +20,9 @@ package org.apache.cassandra.distributed.impl;
import java.lang.annotation.Annotation;
import java.net.InetSocketAddress;
+import java.nio.file.FileSystem;
+import java.nio.file.Files;
+import java.nio.fil
[cassandra] 04/06: [CASSANDRA-16928] CEP-10 Phase 1: InetAddressAndPort extends InetSocketAddress
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit e2721b05ab44183baa2cd4bde9b20862c0eadc90
Author: Benedict Elliott Smith
AuthorDate: Fri Apr 16 12:01:25 2021 +0100
[CASSANDRA-16928] CEP-10 Phase 1: InetAddressAndPort extends
InetSocketAddress
---
.../org/apache/cassandra/audit/AuditLogEntry.java | 6 +-
.../org/apache/cassandra/db/SystemKeyspace.java| 37 +++
.../db/virtual/InternodeInboundTable.java | 2 +-
.../db/virtual/InternodeOutboundTable.java | 2 +-
.../dht/tokenallocator/OfflineTokenAllocator.java | 2 +-
.../cassandra/locator/DynamicEndpointSnitch.java | 2 +-
.../cassandra/locator/Ec2MultiRegionSnitch.java| 2 +-
.../apache/cassandra/locator/IEndpointSnitch.java | 6 ++
.../cassandra/locator/InetAddressAndPort.java | 113 -
.../cassandra/locator/RackInferringSnitch.java | 4 +-
.../cassandra/net/InboundConnectionInitiator.java | 2 +-
.../cassandra/net/InboundConnectionSettings.java | 8 +-
.../cassandra/net/OutboundConnectionInitiator.java | 4 +-
.../cassandra/net/OutboundConnectionSettings.java | 2 +-
.../org/apache/cassandra/net/SocketFactory.java| 2 +-
.../repair/SystemDistributedKeyspace.java | 2 +-
.../repair/consistent/LocalSessionInfo.java| 2 +-
.../cassandra/repair/consistent/LocalSessions.java | 6 +-
.../cassandra/schema/SchemaAnnouncementEvent.java | 4 +-
.../apache/cassandra/service/StorageService.java | 29 +++---
.../cassandra/service/TruncateResponseHandler.java | 2 +-
.../service/reads/repair/ReadRepairEvent.java | 4 +-
.../management/ProgressInfoCompositeData.java | 4 +-
.../SessionCompleteEventCompositeData.java | 4 +-
.../management/SessionInfoCompositeData.java | 8 +-
.../cassandra/tools/nodetool/HostStatWithPort.java | 4 +-
.../apache/cassandra/tracing/TraceKeyspace.java| 8 +-
src/java/org/apache/cassandra/transport/Event.java | 10 +-
.../org/apache/cassandra/transport/Server.java | 2 +-
.../cassandra/transport/messages/ErrorMessage.java | 4 +-
.../org/apache/cassandra/utils/FBUtilities.java| 2 +-
src/java/org/apache/cassandra/utils/Mx4jTool.java | 2 +-
src/java/org/apache/cassandra/utils/UUIDGen.java | 2 +-
.../distributed/impl/DistributedTestSnitch.java| 2 +-
.../cassandra/distributed/impl/Instance.java | 4 +-
.../cassandra/distributed/test/StreamingTest.java | 8 +-
.../cassandra/audit/AuditLoggerAuthTest.java | 4 +-
.../cassandra/dht/RangeFetchMapCalculatorTest.java | 2 +-
.../org/apache/cassandra/gms/GossiperTest.java | 6 +-
.../locator/InetAddressAndPortSerializerTest.java | 4 +-
.../cassandra/locator/InetAddressAndPortTest.java | 16 +--
.../apache/cassandra/net/ForwardingInfoTest.java | 4 +-
.../apache/cassandra/net/MessagingServiceTest.java | 12 +--
.../org/apache/cassandra/repair/RepairJobTest.java | 8 +-
.../service/WriteResponseHandlerTest.java | 2 +-
.../service/WriteResponseHandlerTransientTest.java | 2 +-
.../cassandra/transport/CQLUserAuditTest.java | 6 +-
47 files changed, 200 insertions(+), 173 deletions(-)
diff --git a/src/java/org/apache/cassandra/audit/AuditLogEntry.java
b/src/java/org/apache/cassandra/audit/AuditLogEntry.java
index 3a015c5..02db076 100644
--- a/src/java/org/apache/cassandra/audit/AuditLogEntry.java
+++ b/src/java/org/apache/cassandra/audit/AuditLogEntry.java
@@ -76,10 +76,10 @@ public class AuditLogEntry
StringBuilder builder = new StringBuilder(100);
builder.append("user:").append(user)
.append("|host:").append(host)
- .append("|source:").append(source.address);
-if (source.port > 0)
+ .append("|source:").append(source.getAddress());
+if (source.getPort() > 0)
{
-builder.append("|port:").append(source.port);
+builder.append("|port:").append(source.getPort());
}
builder.append("|timestamp:").append(timestamp)
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index a7cbe40..7e8c2b5 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.db;
import java.io.IOError;
import java.io.IOException;
import java.net.InetAddress;
+import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.time.Instant;
import java.util.*;
@@ -36,7 +37,7 @@ import com.google.common.collect.ImmutableSet;
import com.google.common.collect.SetMultimap;
import com.google.common.collect.Sets;
import com.google.common.io.ByteStreams;
-import com.google.common.util.concurrent.ListenableFuture;
+
import org.apac
[cassandra] branch trunk updated (2e2db4d -> f5fb1b0)
This is an automated email from the ASF dual-hosted git repository.
benedict pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 2e2db4d Verify correct ownership of attached locations on disk at
startup
new 5c3a05a [CASSANDRA-16924] CEP-10 Phase 1: Mockable Blocking
Concurrency Primitives
new e215d2a [CASSANDRA-16925] CEP-10 Phase 1: Mockable Task Execution
new 767f98e [CASSANDRA-16926] CEP-10 Phase 1: Mockable Filesystem
new e2721b0 [CASSANDRA-16928] CEP-10 Phase 1: InetAddressAndPort extends
InetSocketAddress
new 50278db [CASSANDRA-16927] CEP-10 Phase 1: Refactor Streaming
new f5fb1b0 [CASSANDRA-17013] CEP-10 Phase 1: in-jvm-dtest-api changes
and version bump
The 6 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
build.xml | 23 +-
checkstyle.xml | 42 +-
checkstyle_suppressions.xml| 34 +-
.../org/apache/cassandra/audit/AuditLogEntry.java | 6 +-
.../cassandra/audit/AuditLogEntryCategory.java | 2 +-
src/java/org/apache/cassandra/auth/AuthCache.java | 47 +-
.../apache/cassandra/auth/AuthenticatedUser.java | 2 -
.../apache/cassandra/batchlog/BatchlogManager.java | 13 +-
.../apache/cassandra/cache/AutoSavingCache.java| 82 +--
.../AbstractLocalAwareExecutorService.java | 228 ---
.../DebuggableScheduledThreadPoolExecutor.java | 127
.../concurrent/DebuggableThreadPoolExecutor.java | 426
.../cassandra/concurrent/ExecutionFailure.java | 179 +
.../cassandra/concurrent/ExecutorBuilder.java | 92 +++
.../concurrent/ExecutorBuilderFactory.java | 81 +++
.../cassandra/concurrent/ExecutorFactory.java | 266
.../apache/cassandra/concurrent/ExecutorLocal.java | 43 --
.../cassandra/concurrent/ExecutorLocals.java | 85 ++-
.../apache/cassandra/concurrent/ExecutorPlus.java | 183 ++
.../apache/cassandra/concurrent/FutureTask.java| 149 +
.../concurrent/FutureTaskWithResources.java| 57 ++
.../cassandra/concurrent/ImmediateExecutor.java| 113 +++-
.../cassandra/concurrent/InfiniteLoopExecutor.java | 95 ++-
.../FSError.java => concurrent/Interruptible.java} | 39 +-
.../concurrent/JMXEnabledSingleThreadExecutor.java | 81 ---
.../concurrent/JMXEnabledThreadPoolExecutor.java | 191 --
.../LocalAwareExecutorPlus.java} | 9 +-
.../concurrent/LocalAwareExecutorService.java | 77 ---
.../LocalAwareSequentialExecutorPlus.java} | 9 +-
.../LocalAwareSingleThreadExecutorPlus.java| 14 +-
.../LocalAwareThreadPoolExecutorPlus.java | 14 +-
.../cassandra/concurrent/NamedThreadFactory.java | 112 +++-
.../cassandra/concurrent/ResizableThreadPool.java | 39 +-
...orMBean.java => ResizableThreadPoolMXBean.java} | 2 +-
.../apache/cassandra/concurrent/SEPExecutor.java | 94 ++-
.../org/apache/cassandra/concurrent/SEPWorker.java | 10 +-
.../ScheduledExecutorPlus.java}| 12 +-
.../cassandra/concurrent/ScheduledExecutors.java | 14 +-
.../ScheduledThreadPoolExecutorPlus.java | 240 +++
.../concurrent/SequentialExecutorPlus.java | 53 ++
.../cassandra/concurrent/SharedExecutorPool.java | 19 +-
...{ResizableThreadPool.java => Shutdownable.java} | 21 +-
.../concurrent/SingleThreadExecutorPlus.java | 100 +++
.../org/apache/cassandra/concurrent/Stage.java | 120 ++--
.../cassandra/concurrent/SyncFutureTask.java | 70 ++
.../apache/cassandra/concurrent/TaskFactory.java | 178 +
.../concurrent/ThreadPoolExecutorBase.java | 186 ++
.../concurrent/ThreadPoolExecutorBuilder.java | 204 ++
.../concurrent/ThreadPoolExecutorJMXAdapter.java | 246 +++
.../concurrent/ThreadPoolExecutorPlus.java | 125
.../cassandra/concurrent/WrappedExecutorPlus.java | 178 +
.../config/CassandraRelevantProperties.java| 2 +
.../cassandra/config/DatabaseDescriptor.java | 101 +--
.../apache/cassandra/config/EncryptionOptions.java | 1 +
.../cassandra/config/YamlConfigurationLoader.java | 8 +-
src/java/org/apache/cassandra/cql3/CQL3Type.java | 1 -
.../cassandra/cql3/conditions/ColumnCondition.java | 2 +-
.../cassandra/cql3/functions/AbstractFunction.java | 1 -
.../cql3/functions/JavaBasedUDFunction.java| 3 +-
.../cql3/functions/ScriptBasedUDFunction.java | 2 +-
.../cql3/functions/UDFExecutorService.java | 48 +-
.../cassandra/cql3/functions/UDFunction.java | 11 +-
.../cassandra/cql3/functions/types/TypeCodec.java | 2 -
.../cassandra/cql3/selection/ResultSetBuilder.java | 2 +-
[jira] [Commented] (CASSANDRA-16983) Separating CQLSH credentials from the cqlshrc file
[ https://issues.apache.org/jira/browse/CASSANDRA-16983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425492#comment-17425492 ] Stefan Miklosovic commented on CASSANDRA-16983: --- hi [~Bowen Song], could you please go one more time over your PR and remove all Windows specific stuff? This work is related to CASSANDRA-16956 where I am touching pylib/cqlsh. Would you be so nice to go over the PR there and verify my changes related to Windows make sense? Thanks! > Separating CQLSH credentials from the cqlshrc file > -- > > Key: CASSANDRA-16983 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16983 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Bowen Song >Assignee: Bowen Song >Priority: Normal > Labels: lhf > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently, the CQLSH tool accepts credentials (username & password) from the > following 3 places: > 1. the command line parameter "-p" > 2. the cqlshrc file > 3. prompt the user > This is not ideal. > Credentials in the command line is a security risk, because it could be see > by other users on a shared system. > The cqlshrc file is better, but still not good enough. Because the cqlshrc > file is a config file, it's often acceptable to have it as a world readable > file, and share it with other users. It also prevents user from having > multiple sets of credentials, either for the same Cassandra cluster or > different clusters. > To improve the security of CQLSH and make it secure by design, I purpose the > following changes: > * Warn the user if a password is giving in the command line, and recommend > them to use a credential file instead > * Warn the user if credentials are present in the cqlshrc file and the > cqlshrc file is not secure (e.g.: world readable or owned by a different user) > * Deprecate credentials in the cqlshrc, and recommend the user to move them > to a separate credential file. The aim is to not break anything at the > moment, but eventually stop accepting credentials from the cqlshrc file. > * Reject the credentials file if it's not secure, and tell the user how to > secure it. Optionally, prompt the user for password if it's an interactive > session. (Think how does OpenSSH handle insecure credential files) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425480#comment-17425480
]
Stefan Miklosovic commented on CASSANDRA-16956:
---
Wow this is super tedious to go through. I can get rid of the specific Windowsy
code but I do not know what to do with all the comments which are also
mentioning the fact that "this has to be done like that because on Windows
...". I am not going to rewrite it (even if I knew how) just because we are not
supporting it anymore. Let's not touch what is not broken. We do not need to
super super dilligent here.
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16916) Add support for IF EXISTS and IF NOT EXISTS in ALTER statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425475#comment-17425475
]
Benjamin Lerer commented on CASSANDRA-16916:
[~subkanthi] Let me look into it. I am sure that I can find you something :-)
> Add support for IF EXISTS and IF NOT EXISTS in ALTER statements
> ---
>
> Key: CASSANDRA-16916
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16916
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Syntax
>Reporter: Benjamin Lerer
>Assignee: Jogesh Anand
>Priority: Normal
>
> It would make sense to add support for {{IF EXISTS}} and {{IF NOT EXISTS}} in
> the different {{ALTER}} statements.
> For example:
> * {{ALTER TABLE IF EXISTS myTable ...}}
> * {{ALTER TABLE myTable ADD IF NOT EXISTS ...}}
> * {{ALTER TABLE myTable DROP IF EXISTS ...}}
> * {{ALTER TYPE IF EXISTS myType ...}}
> * {{ALTER TYPE myType ADD IF NOT EXISTS ...}}
> +Additional info for newcomers:+
> In order to implement this change you will need to change the {{Parser.g}}
> ANTLR file located in the src/antlr directory and the java classes
> corresponding to the different alter statements located in the
> {{org.apache.cassandra.cql3.statements.schema}} package. You can look at the
> CreateTableStatement class to see how it was done there.
> The unit test for the CQL logic are located under
> {{org.apache.cassandra.cql3.validation}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16839) Truncation snapshots unnecessarily created on node startup
[
https://issues.apache.org/jira/browse/CASSANDRA-16839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-16839:
--
Reviewers: Andres de la Peña
> Truncation snapshots unnecessarily created on node startup
> --
>
> Key: CASSANDRA-16839
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16839
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Paulo Motta
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> When testing cassandra 4.0 on ccm I noticed that everytime I restart a node,
> truncation snapshots are created for the tables {{system.table_estimates}}
> and {{system.size_estimates}}:
> {noformat}
> $ ccm create -n 1 test -s
> $ ccm node1 stop
> $ ccm node1 start
> $ ccm node1 stop
> $ ccm node1 start
> $ ccm node1 nodetool listsnapshots
> Snapshot Details:
> Snapshot name Keyspace name Column family name True
> size Size on disk
> truncated-1628599001857-table_estimates systemtable_estimates0
> bytes 13 bytes
> truncated-1628599099560-table_estimates systemtable_estimates0
> bytes 13 bytes
> truncated-1628599001736-size_estimates systemsize_estimates 0
> bytes 13 bytes
> truncated-1628599057438-table_estimates systemtable_estimates6.16
> KiB 6.19 KiB
> truncated-1628599099458-size_estimates systemsize_estimates 0
> bytes 13 bytes
> truncated-1628599057340-size_estimates systemsize_estimates 5.73
> KiB 5.76 KiB
> Total TrueDiskSpaceUsed: 0 bytes
> {noformat}
> Not sure if this is expected behavior, but feels like a bug to me.
> Reproduced on 4.0, not sure if it reproduces on lower versions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16562) Fix flaky testSkipScrubCorruptedCounterRowWithTool
[
https://issues.apache.org/jira/browse/CASSANDRA-16562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425437#comment-17425437
]
Andres de la Peña edited comment on CASSANDRA-16562 at 10/7/21, 10:20 AM:
--
It seems that the failures in {{ScrubTest}} are still there and the patches
still apply after an easy rebase. The test failures only appear when
compression is used. The fix used in CASSANDRA-16532 is able to fix the
occasional failures in 3.11, as it's shown why these multiplexer runs:
||PR||CI||
|[16562-3.0|https://github.com/apache/cassandra/pull/1253]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/966/workflows/ecefeef6-16b3-436c-b62f-8b76649dd864]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/975/workflows/38f961fc-d98d-428e-93f5-2cdc7870b4f0]|
|[16562-3.11|https://github.com/apache/cassandra/pull/1254]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/965/workflows/eed40245-98e2-4afc-b7a0-8bc93f819006]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/974/workflows/cec11724-773a-453d-a3d2-acb28de7fdce]|
|[cassandra-3.0|https://github.com/apache/cassandra/tree/cassandra-3.0]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/966/workflows/ecefeef6-16b3-436c-b62f-8b76649dd864]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/975/workflows/38f961fc-d98d-428e-93f5-2cdc7870b4f0]|
|[cassandra-3.11|https://github.com/apache/cassandra/tree/cassandra-3.11]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/965/workflows/eed40245-98e2-4afc-b7a0-8bc93f819006]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/971/workflows/73804303-470a-4b15-8a0a-a5bde2794471]|
However, the situation is different for 3.0. In that case,
{{testScrubCorruptedCounterRow}} has been [consistently
failing|https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0-test-compression/lastBuild/jdk=jdk_1.8_latest,label=cassandra,split=2/testReport/org.apache.cassandra.db/ScrubTest/testScrubCorruptedCounterRow/]
for a while, and the fix from #16532 doesn't change that. It seems that the
test somehow gets stuck in a deadlock and it ends up failing with a timeout.
Bisect shows that this test has been failing since CASSANDRA-14284, and indeed
the test
[passes|https://app.circleci.com/pipelines/github/adelapena/cassandra/976/workflows/a1e4ad08-c517-4a6b-a130-14f210af1545]
if we temporarily revert the changes in {{CompressedRandomAccessReader}} done
by that ticket. I'm still trying to figure out how verifying the checksum
before uncompressing is causing the deadlock.
CC [~blerer]
was (Author: adelapena):
It seems that the failures in {{ScrubTest}} are still there and the patches
still apply after an easy rebase. The test failures only appear when
compression is used. The fix used in CASSANDRA-16532 is able to fix the
occasional failures in 3.11, as it's shown why these multiplexer runs:
||PR||CI||
|[16562-3.0|https://github.com/apache/cassandra/pull/1253]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/966/workflows/ecefeef6-16b3-436c-b62f-8b76649dd864]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/975/workflows/38f961fc-d98d-428e-93f5-2cdc7870b4f0]|
|[16562-3.11|https://github.com/apache/cassandra/pull/1254]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/965/workflows/eed40245-98e2-4afc-b7a0-8bc93f819006]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/974/workflows/cec11724-773a-453d-a3d2-acb28de7fdce]|
|[cassandra-3.0|https://github.com/apache/cassandra/tree/cassandra-3.0]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/966/workflows/ecefeef6-16b3-436c-b62f-8b76649dd864]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/975/workflows/38f961fc-d98d-428e-93f5-2cdc7870b4f0]|
|[cassandra-3.11|https://github.com/apache/cassandra/tree/cassandra-3.11]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/965/workflows/eed40245-98e2-4afc-b7a0-8bc93f819006]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/971/workflows/73804303-470a-4b15-8a0a-a5bde2794471]|
However, the situation is different for 3.0. In that case,
{{testScrubCorruptedCounterRow}} has been [consistently
failing|https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0-test-compression/lastBuild/jdk=jdk_1.8_latest,label=cassandra,split=2/testReport/org.apache.cassandra.db/ScrubTest/testScrubCorruptedCounterRow/]
for a while, and the fix from #16532 doesn't change that. It seems that the
test somehow gets stuck in a deadlock and it ends up failing with a timeout.
Bisect sho
[jira] [Commented] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425456#comment-17425456
]
Berenguer Blasi commented on CASSANDRA-16956:
-
On second thoughts, maybe you want to do dtests in another ticket to make this
more digestible and less big-bang? just a suggestion.
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16562) Fix flaky testSkipScrubCorruptedCounterRowWithTool
[
https://issues.apache.org/jira/browse/CASSANDRA-16562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16562:
Description: See CASSANDRA-16532 where extra flakiness was detected on 3.0
and 3.11 branches for {{testSkipScrubCorruptedCounterRowWithTool}} (was: See
CASSANDRA-16532 where extra flakiness was detected on 3.0 and 3.11 branches
fortestSkipScrubCorruptedCounterRowWithTool)
> Fix flaky testSkipScrubCorruptedCounterRowWithTool
> --
>
> Key: CASSANDRA-16562
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16562
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> See CASSANDRA-16532 where extra flakiness was detected on 3.0 and 3.11
> branches for {{testSkipScrubCorruptedCounterRowWithTool}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16562) Fix flaky testSkipScrubCorruptedCounterRowWithTool
[ https://issues.apache.org/jira/browse/CASSANDRA-16562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16562: Reviewers: Berenguer Blasi > Fix flaky testSkipScrubCorruptedCounterRowWithTool > -- > > Key: CASSANDRA-16562 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16562 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Berenguer Blasi >Assignee: Andres de la Peña >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > See CASSANDRA-16532 where extra flakiness was detected on 3.0 and 3.11 > branches fortestSkipScrubCorruptedCounterRowWithTool -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425449#comment-17425449
]
Stefan Miklosovic commented on CASSANDRA-16956:
---
1) yes
2) yes
3) I ll look, thanks, this is somehow tangential to CASSANDRA-19683 when it
comes to Windows support. If we go with the latest comment of Dinesh, I think
we need to get rid of all the stuff in there.
4) ok
5) I have not covered dtests yet, will do.
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16562) Fix flaky testSkipScrubCorruptedCounterRowWithTool
[
https://issues.apache.org/jira/browse/CASSANDRA-16562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425437#comment-17425437
]
Andres de la Peña commented on CASSANDRA-16562:
---
It seems that the failures in {{ScrubTest}} are still there and the patches
still apply after an easy rebase. The test failures only appear when
compression is used. The fix used in CASSANDRA-16532 is able to fix the
occasional failures in 3.11, as it's shown why these multiplexer runs:
||PR||CI||
|[16562-3.0|https://github.com/apache/cassandra/pull/1253]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/966/workflows/ecefeef6-16b3-436c-b62f-8b76649dd864]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/975/workflows/38f961fc-d98d-428e-93f5-2cdc7870b4f0]|
|[16562-3.11|https://github.com/apache/cassandra/pull/1254]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/965/workflows/eed40245-98e2-4afc-b7a0-8bc93f819006]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/974/workflows/cec11724-773a-453d-a3d2-acb28de7fdce]|
|[cassandra-3.0|https://github.com/apache/cassandra/tree/cassandra-3.0]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/966/workflows/ecefeef6-16b3-436c-b62f-8b76649dd864]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/975/workflows/38f961fc-d98d-428e-93f5-2cdc7870b4f0]|
|[cassandra-3.11|https://github.com/apache/cassandra/tree/cassandra-3.11]|[testsome|https://app.circleci.com/pipelines/github/adelapena/cassandra/965/workflows/eed40245-98e2-4afc-b7a0-8bc93f819006]
[test-compression|https://app.circleci.com/pipelines/github/adelapena/cassandra/971/workflows/73804303-470a-4b15-8a0a-a5bde2794471]|
However, the situation is different for 3.0. In that case,
{{testScrubCorruptedCounterRow}} has been [consistently
failing|https://ci-cassandra.apache.org/view/Cassandra%203.0/job/Cassandra-3.0-test-compression/lastBuild/jdk=jdk_1.8_latest,label=cassandra,split=2/testReport/org.apache.cassandra.db/ScrubTest/testScrubCorruptedCounterRow/]
for a while, and the fix from #16532 doesn't change that. It seems that the
test somehow gets stuck in a deadlock and it ends up failing with a timeout.
Bisect shows that this test has been failing since CASSANDRA-14284, and indeed
the test passes if we revert the changes in {{CompressedRandomAccessReader}}
done by that ticket. I'm still trying to figure out how verifying the checksum
before uncompressing is causing the deadlock.
CC [~blerer]
> Fix flaky testSkipScrubCorruptedCounterRowWithTool
> --
>
> Key: CASSANDRA-16562
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16562
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> See CASSANDRA-16532 where extra flakiness was detected on 3.0 and 3.11
> branches fortestSkipScrubCorruptedCounterRowWithTool
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425432#comment-17425432
]
Berenguer Blasi edited comment on CASSANDRA-16956 at 10/7/21, 9:27 AM:
---
On a first cursory pass I have these q I hope make sense:
- Do we want to remove {{windows_timer_interval}} from config?
- There are some text comments with Windows info in them. Should we remove that
as well?-
- pylib/cqlshlib folder has some windows references that might deserve a look.
- {{grep -ri windows --exclude \*.java}} lists a few other occurrences I would
look into.
- dtests have lots of windows stuff I would look into as well.
was (Author: bereng):
On a first cursory pass I have these q I hope make sense:
- Do we want to remove {{windows_timer_interval}} from config?
- There are some text comments with Windows info in them. Should we remove that
as well?-
pylib/cqlshlib folder has some windows references that might deserve a look.
- {{grep -ri windows --exclude \*.java}} lists a few other occurrences I would
look into.
- dtests have lots of windows stuff I would look into as well.
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16956:
Reviewers: Berenguer Blasi, Josh McKenzie (was: Josh McKenzie)
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425432#comment-17425432
]
Berenguer Blasi commented on CASSANDRA-16956:
-
On a first cursory pass I have these q I hope make sense:
- Do we want to remove {{windows_timer_interval}} from config?
- There are some text comments with Windows info in them. Should we remove that
as well?-
pylib/cqlshlib folder has some windows references that might deserve a look.
- {{grep -ri windows --exclude \*.java}} lists a few other occurrences I would
look into.
- dtests have lots of windows stuff I would look into as well.
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch trunk updated (bf8b17a -> e283e1a)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git. from bf8b17a Merge pull request #28 from belliottsmith/17013-trunk add 1b20b22 [maven-release-plugin] prepare release 0.0.10 add 2139b4c [maven-release-plugin] prepare for next development iteration add e283e1a Update CHANGES.txt No new revisions were added by this update. Summary of changes: CHANGES.txt | 13 + pom.xml | 2 +- 2 files changed, 14 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14795) Expose information about stored hints via JMX
[
https://issues.apache.org/jira/browse/CASSANDRA-14795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425425#comment-17425425
]
Aleksandr Sorokoumov commented on CASSANDRA-14795:
--
I have a suspicion that the test times out because the entire suite takes too
long to run for the given timeout. We observe it as
{{HintsServiceTest.testListPendingHints}} failure because it is the last test
case in the suite.
For test report and logs see e.g.
https://nightlies.apache.org/cassandra/devbranch/Cassandra-devbranch/1187/
{{test.timeout}} is set to 240 seconds. On successful runs the suite takes a
bit longer than 200 seconds to finish, each case taking between 30 and 60
seconds. As a speculation, a small hiccup or a slight deviation in test time
might lead to a timeout. I'd like to verify this idea by increasing
{{test.timeout}} to 360 seconds and re-running the CI. If this theory is
correct, {{HintsServiceTest}} should succeed and overall test duration might
exceed 240 seconds on 1 or more attempts.
[~azotcsit], [~e.dimitrova] Can I kindly ask you to re-run
https://ci-cassandra.apache.org/job/Cassandra-devbranch/1187/?
> Expose information about stored hints via JMX
> -
>
> Key: CASSANDRA-14795
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14795
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Observability
>Reporter: Aleksandr Sorokoumov
>Assignee: Aleksandr Sorokoumov
>Priority: Low
> Fix For: 4.x
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Currently there is no way to determine what kind of hints a node has, apart
> from looking at the filenames (thus host-ids) on disk. Having a way to access
> this information would help with debugging hint creation/replay scenarios.
> In addition to the JMX method, there is a new nodetool command:
> {noformat}$ bin/nodetool -h 127.0.0.1 -p 7100 listendpointspendinghints
> Host ID Address Rack DC Status Total files Newest Oldest
> 5762b140-3fdf-4057-9ca7-05c070ccc9c3 127.0.0.2 rack1 datacenter1 DOWN 2
> 2018-09-18 14:05:18,835 2018-09-18 14:05:08,811
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16518) Node restart during joining sets protocol version to V3
[
https://issues.apache.org/jira/browse/CASSANDRA-16518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17425372#comment-17425372
]
Joseph Clay commented on CASSANDRA-16518:
-
I was finally able to reproduce. So the issue only reproduces if you configure
cassandra so that it uses preferred_ip. You need to set cassandra so that it
has two ip address per node and broadcast address a different ip to listen
address.
Here is how i reproduced it in CCM:
{noformat}
# Set ccm directory here
CCM_DIR=~/.ccm
# create cluster but don't start it
ccm create test -v 3.11.11 -n 3
# Configure cluster to use GossipingPropertyFileSnitch with prefer_local and
different ips for broadcast and listen addresses
rm $CCM_DIR/test/node*/conf/cassandra-topology.properties
sed -i 's/# prefer_local=true/prefer_local=true/'
$CCM_DIR/test/node*/conf/cassandra-rackdc.properties
sed -i 's/endpoint_snitch.*/endpoint_snitch: GossipingPropertyFileSnitch/'
$CCM_DIR/test/node*/conf/cassandra.yaml
sed -i 's/ - seeds:.*/ - seeds: 127.0.1.1/'
$CCM_DIR/test/node*/conf/cassandra.yaml
echo 'broadcast_address: 127.0.1.1' >> $CCM_DIR/test/node1/conf/cassandra.yaml
echo 'broadcast_address: 127.0.1.2' >> $CCM_DIR/test/node2/conf/cassandra.yaml
echo 'broadcast_address: 127.0.1.3' >> $CCM_DIR/test/node3/conf/cassandra.yaml
echo 'listen_on_broadcast_address: true' >>
$CCM_DIR/test/node1/conf/cassandra.yaml
echo 'listen_on_broadcast_address: true' >>
$CCM_DIR/test/node2/conf/cassandra.yaml
echo 'listen_on_broadcast_address: true' >>
$CCM_DIR/test/node3/conf/cassandra.yaml
# Start nodes1 and 2 need to skip wait as the above config breaks it
ccm node1 start --skip-wait-other-notice
ccm node2 start --skip-wait-other-notice
# Wait for both nodes to be UN then put roughly a GB of data into the cluster
ccm stress write n=500
# Slow down streaming so joining takes long enough to see issue then start 3rd
node
ccm node1 nodetool setstreamthroughput 1
ccm node2 nodetool setstreamthroughput 1
sed -i 's/auto_bootstrap.*/auto_bootstrap: true/'
$CCM_DIR/test/node3/conf/cassandra.yaml
ccm node3 start --skip-wait-other-notice
# Wait for streaming to start then check which node that node3 is streaming off
ccm node3 nodetool netstats
# On my cluster node1 was streaming so i stop & started node 2
ccm node2 stop
ccm node2 start --skip-wait-other-notice{noformat}
After that the restarted node logged this:
INFO [main] 2021-10-07 17:54:40,637 ConfiguredLimit.java:108 - Detected peers
which do not fully support protocol V4. Capping max negotiable version to V3
Also if you tried to cqlsh to the restarted node:
Connection error: ('Unable to connect to any servers', \{'127.0.0.2:9042':
DriverException('ProtocolError returned from server while using explicitly set
client protocol_version 4')})
Errors in the node logs corresponding to attempted connection:
WARN [epollEventLoopGroup-2-6] 2021-10-07 17:56:04,822 NoSpamLogger.java:94 -
Protocol exception with client networking:
org.apache.cassandra.transport.ProtocolException: Invalid or unsupported
protocol version (4); supported versions are (3/v3, 4/v4, 5/v5-beta)
You can cqlsh if you force the protocol version: cqlsh 127.0.0.2
--protocol-version=3
Peers table:
{noformat}
peer | data_center | host_id | preferred_ip
| rack | release_version | rpc_address | schema_version
| tokens
---+-+--+--+---+-+-+--+--
127.0.1.1 | dc1 | d4d86d82-d12b-4bae-a1a4-7f0ded9d9b79 |127.0.0.1
| rack1 | 3.11.11 | 127.0.0.1 | d25c2da9-b17e-3aba-8682-6cf063aaca51
| {'-9223372036854775808'}
127.0.1.3 |null | null |127.0.0.3
| null |null |null | null
| null{noformat}
As you can see node3 the joining node has all nulls except for peer and
preferred_ip columns. I believe the null in the release_version column causes
the version check to fail.
> Node restart during joining sets protocol version to V3
> ---
>
> Key: CASSANDRA-16518
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16518
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Joseph Clay
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.x
>
>
> While joining nodes to a cluster, an old node crashed. The old node was
> recovered however clients (datastax java) refused to connect to it.
> The driver error:
> {noformat}
> Detected added or restarted Cassandra host /: but ignoring it since
> it does not support th
