[jira] [Updated] (CASSANDRA-16079) Improve dtest runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16079: --- Status: In Progress (was: Patch Available) > Improve dtest runtime > - > > Key: CASSANDRA-16079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16079 > Project: Cassandra > Issue Type: Improvement > Components: CI >Reporter: Adam Holmberg >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-09-19 at 12.32.21.png > > > A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a > [30% increase in run > time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. > While that change was accepted, we wanted to spin out a ticket to optimize > dtests in an attempt to gain back some of that runtime. > At this time we don't have concrete improvements in mind, so the first order > of this ticket will be to analyze the state of things currently, and try to > ascertain some valuable optimizations. Once the problems are understood, we > will break down subtasks to divide the work. > Some areas to consider: > * cluster reuse > * C* startup optimizations > * Tests that should be ported to in-JVM dtest or even unit tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16079) Improve dtest runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-16079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233453#comment-17233453 ] Michael Semb Wever commented on CASSANDRA-16079: Down to 4 failures: https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/208/#showFailuresLink I still need to test the novnode dtests, and also to identify which dtests need normal bootstrap… > Improve dtest runtime > - > > Key: CASSANDRA-16079 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16079 > Project: Cassandra > Issue Type: Improvement > Components: CI >Reporter: Adam Holmberg >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta > > Attachments: Screenshot 2020-09-19 at 12.32.21.png > > > A recent ticket, CASSANDRA-13701, changed the way dtests run, resulting in a > [30% increase in run > time|https://www.mail-archive.com/dev@cassandra.apache.org/msg15606.html]. > While that change was accepted, we wanted to spin out a ticket to optimize > dtests in an attempt to gain back some of that runtime. > At this time we don't have concrete improvements in mind, so the first order > of this ticket will be to analyze the state of things currently, and try to > ascertain some valuable optimizations. Once the problems are understood, we > will break down subtasks to divide the work. > Some areas to consider: > * cluster reuse > * C* startup optimizations > * Tests that should be ported to in-JVM dtest or even unit tests -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233507#comment-17233507
]
Sam Tunnicliffe commented on CASSANDRA-16275:
-
bq. It came from this change, which includes ServerError in retry logic.
Cool, that was my understanding also.
Whichever order we do things in, we're going to have some breakage, to try and
minimize that I propose the following:
1. Commit this fix. {{auth_test.py::TestAuth::test_handle_corrupt_role}} will
start to fail on all CI jobs.
2. Update the {{cassandra-test}} branch to at least
{{e1fc528a0deec67f725f066204e8c1ea8b26bbde}}. I expect all dtest CI jobs to
start failing here due to the incompatibilities in the base docker image.
3. Publish new docker images which install the updated driver in the base
image, so no conflicting updates are pulled.
4. Update cassandra's .circle/config.yml to use the new docker images. Build
should start to work again, but with a handful of new failures for V5 tests.
{{auth_test.py}} should start passing again.
5. Commit CASSANDRA-15299 to trunk. This will fix the failing V5 tests.
Most of the disruption will occur between steps 2 and 5, so I suggest we wait
until INFRA-21103 is resolved before starting.
[~aholmber] [~mck] does that sound like a plan to you?
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Test and Documentation Plan: Dtest only change
Status: Patch Available (was: Open)
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16275) Update python driver used by cassandra-dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-16275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16275:
Reviewers: Adam Holmberg
> Update python driver used by cassandra-dtest
>
>
> Key: CASSANDRA-16275
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16275
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/python
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
>
> In order to commit CASSANDRA-15299, the python driver used by the dtests
> needs to include PYTHON-1258, support for V5 framing.
> Updating the python driver's cassandra-test branch to latest trunk causes 1
> additional dtest failure in
> {{auth_test.py::TestAuth::test_handle_corrupt_role_data}} because the
> {{ServerError}} response is now subject to the configured {{retry_policy}}.
> This means the error ultimately returned from the driver is
> {{NoHostAvailable}}, rather than {{ServerError}}.
> I'll open a dtest pr to change the expectation in the test and we can commit
> that when the cassandra-test branch is updated.
> cc [~aholmber] [~aboudreault]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13325) Bring back the accepted encryption protocols list as configurable option
[
https://issues.apache.org/jira/browse/CASSANDRA-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233511#comment-17233511
]
Berenguer Blasi commented on CASSANDRA-13325:
-
Went for a review and only found minor nits I left as comments.
> Bring back the accepted encryption protocols list as configurable option
>
>
> Key: CASSANDRA-13325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13325
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Nachiket Patil
>Assignee: Jon Meredith
>Priority: Low
> Fix For: 4.0-beta
>
> Attachments: trunk.diff
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> With CASSANDRA-10508, the hard coded list of accepted encryption protocols
> was eliminated. For some use cases, it is necessary to restrict the
> encryption protocols used for communication between client and server.
> Default JVM way of negotiations allows the best encryption protocol that
> client can use.
> e.g. I have set Cassandra to use encryption. Ideally client and server
> negotiate to use best protocol (TLSv1.2). But a malicious client might force
> TLSv1.0 which is susceptible to POODLE attacks.
> At the moment only way to restrict the encryption protocol is using the
> {{jdk.tls.client.protocols}} systems property. If I dont have enough access
> to modify this property, I dont have any way of restricting the encryption
> protocols.
> I am proposing bring back the accepted_protocols property but make it
> configurable. If not specified, let the JVM take care of the TLS negotiations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-13325) Bring back the accepted encryption protocols list as configurable option
[
https://issues.apache.org/jira/browse/CASSANDRA-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-13325:
Reviewers: Berenguer Blasi
> Bring back the accepted encryption protocols list as configurable option
>
>
> Key: CASSANDRA-13325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13325
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Nachiket Patil
>Assignee: Jon Meredith
>Priority: Low
> Fix For: 4.0-beta
>
> Attachments: trunk.diff
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> With CASSANDRA-10508, the hard coded list of accepted encryption protocols
> was eliminated. For some use cases, it is necessary to restrict the
> encryption protocols used for communication between client and server.
> Default JVM way of negotiations allows the best encryption protocol that
> client can use.
> e.g. I have set Cassandra to use encryption. Ideally client and server
> negotiate to use best protocol (TLSv1.2). But a malicious client might force
> TLSv1.0 which is susceptible to POODLE attacks.
> At the moment only way to restrict the encryption protocol is using the
> {{jdk.tls.client.protocols}} systems property. If I dont have enough access
> to modify this property, I dont have any way of restricting the encryption
> protocols.
> I am proposing bring back the accepted_protocols property but make it
> configurable. If not specified, let the JVM take care of the TLS negotiations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16173) Update "Getting Started" document for Windows users
[ https://issues.apache.org/jira/browse/CASSANDRA-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233548#comment-17233548 ] João Reis commented on CASSANDRA-16173: --- Just a heads up, we can probably close either this one or [CASSANDRA-15887|https://issues.apache.org/jira/browse/CASSANDRA-15887] as duplicate. > Update "Getting Started" document for Windows users > --- > > Key: CASSANDRA-16173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16173 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Yuki Morishita >Priority: Normal > Fix For: 4.0 > > > This is a documentation follow up to CASSANDRA-16171. > Since we are removing support for Windows, we should update ["Getting > Started" > guide|https://cassandra.apache.org/doc/latest/getting_started/index.html] to > include how-to's for Windows users for setting up Cassandra for dev > evnironment. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9608) Support Java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233556#comment-17233556
]
Yakir Gibraltar commented on CASSANDRA-9608:
Hi [~snazy] Possible small bug in {{cassandra-env.sh:}}
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-e}}q will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
> Support Java 11
> ---
>
> Key: CASSANDRA-9608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9608
> Project: Cassandra
> Issue Type: Task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Low
> Labels: Java11
> Fix For: 4.0, 4.0-alpha1
>
> Attachments: jdk_9_10.patch
>
>
> This ticket is intended to group all issues found to support Java 9 in the
> future.
> From what I've found out so far:
> * Maven dependency {{com.sun:tools:jar:0}} via cobertura cannot be resolved.
> It can be easily solved using this patch:
> {code}
> - artifactId="cobertura"/>
> + artifactId="cobertura">
> +
> +
> {code}
> * Another issue is that {{sun.misc.Unsafe}} no longer contains the methods
> {{monitorEnter}} + {{monitorExit}}. These methods are used by
> {{o.a.c.utils.concurrent.Locks}} which is only used by
> {{o.a.c.db.AtomicBTreeColumns}}.
> I don't mind to start working on this yet since Java 9 is in a too early
> development phase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-9608) Support Java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233556#comment-17233556
]
Yakir Gibraltar edited comment on CASSANDRA-9608 at 11/17/20, 12:41 PM:
Hi [~snazy] Possible small bug in {{cassandra-env.sh:}}
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^\-\[X\]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "\-\[X\]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-eq}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
was (Author: yakir.g):
Hi [~snazy] Possible small bug in {{cassandra-env.sh:}}
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-e}}q will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
> Support Java 11
> ---
>
> Key: CASSANDRA-9608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9608
> Project: Cassandra
> Issue Type: Task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Low
> Labels: Java11
> Fix For: 4.0, 4.0-alpha1
>
> Attachments: jdk_9_10.patch
>
>
> This ticket is intended to group all issues found to support Java 9 in the
> future.
> From what I've found out so far:
> * Maven dependency {{com.sun:tools:jar:0}} via cobertura cannot be resolved.
> It can be easily solved using this patch:
> {code}
> - artifactId="cobertura"/>
> + artifactId="cobertura">
> +
> +
> {code}
> * Another issue is that {{sun.misc.Unsafe}} no longer contains the methods
> {{monitorEnter}} + {{monitorExit}}. These methods are used by
> {{o.a.c.utils.concurrent.Locks}} which is only used by
> {{o.a.c.db.AtomicBTreeColumns}}.
> I don't mind to start working on this yet since Java 9 is in a too early
> development phase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-9608) Support Java 11
[
https://issues.apache.org/jira/browse/CASSANDRA-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233556#comment-17233556
]
Yakir Gibraltar edited comment on CASSANDRA-9608 at 11/17/20, 12:41 PM:
Hi [~snazy] [~benedict] [~jasobrown] Possible small bug in {{cassandra-env.sh:}}
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-eq}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
was (Author: yakir.g):
Hi [~snazy] Possible small bug in {{cassandra-env.sh:}}
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^\-\[X\]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "\-\[X\]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-eq}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
> Support Java 11
> ---
>
> Key: CASSANDRA-9608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9608
> Project: Cassandra
> Issue Type: Task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Low
> Labels: Java11
> Fix For: 4.0, 4.0-alpha1
>
> Attachments: jdk_9_10.patch
>
>
> This ticket is intended to group all issues found to support Java 9 in the
> future.
> From what I've found out so far:
> * Maven dependency {{com.sun:tools:jar:0}} via cobertura cannot be resolved.
> It can be easily solved using this patch:
> {code}
> - artifactId="cobertura"/>
> + artifactId="cobertura">
> +
> +
> {code}
> * Another issue is that {{sun.misc.Unsafe}} no longer contains the methods
> {{monitorEnter}} + {{monitorExit}}. These methods are used by
> {{o.a.c.utils.concurrent.Locks}} which is only used by
> {{o.a.c.db.AtomicBTreeColumns}}.
> I don't mind to start working on this yet since Java 9 is in a too early
> development phase.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
Yakir Gibraltar created CASSANDRA-16279:
---
Summary: Incorrect check for -Xlog in cassandra-env.sh
Key: CASSANDRA-16279
URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
Project: Cassandra
Issue Type: Bug
Reporter: Yakir Gibraltar
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16279:
Bug Category: Parent values: Packaging(13660)Level 1 values: All(13663)
Complexity: Low Hanging Fruit
Component/s: Packaging
Discovered By: User Report
Severity: Normal
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Priority: Normal
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
> Should be:
> {{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paulo Motta updated CASSANDRA-16279:
Fix Version/s: 4.0-beta
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
> Should be:
> {{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yakir Gibraltar updated CASSANDRA-16279:
Description:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^\-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "\-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{\-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
was:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {{echo "$JVM_OPTS" | grep -q "^\-[X]log:gc"}}
> Should be:
> {{echo "$JVM_OPTS" | grep -qe "\-[X]log:gc"}}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{\-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yakir Gibraltar updated CASSANDRA-16279:
Description:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
Right now, let's say that i added
was:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^\-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "\-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{\-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
> Should be:
> {{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
>
> Right now, let's say that i added
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yakir Gibraltar updated CASSANDRA-16279:
Description:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
Right now, jvm11-server.options with:
{code:java}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M{code}
Will generate process of:
{code:java}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
{code}
With fix it will generate correct gc option of :
{code}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
{code}
was:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
Right now, let's say that i added
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
> Should be:
> {{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
>
> Right now, jvm11-server.options with:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M{code}
> Will generate process of:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
>
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
> {code}
> With fix it will generate correct gc option of :
> {code}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16213) Cannot replace_address /X because it doesn't exist in gossip
[ https://issues.apache.org/jira/browse/CASSANDRA-16213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233571#comment-17233571 ] Paulo Motta commented on CASSANDRA-16213: - Thanks! I'll take another look this week. > Cannot replace_address /X because it doesn't exist in gossip > > > Key: CASSANDRA-16213 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16213 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip, Cluster/Membership >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We see this exception around nodes crashing and trying to do a host > replacement; this error appears to be correlated around multiple node > failures. > A simplified case to trigger this is the following > *) Have a N node cluster > *) Shutdown all N nodes > *) Bring up N-1 nodes (at least 1 seed, else replace seed) > *) Host replace the N-1th node -> this will fail with the above > The reason this happens is that the N-1th node isn’t gossiping anymore, and > the existing nodes do not have its details in gossip (but have the details in > the peers table), so the host replacement fails as the node isn’t known in > gossip. > This affects all versions (tested 3.0 and trunk, assume 2.2 as well) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16279) Incorrect check for -Xlog in cassandra-env.sh
[
https://issues.apache.org/jira/browse/CASSANDRA-16279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yakir Gibraltar updated CASSANDRA-16279:
Description:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{code}
echo "$JVM_OPTS" | grep -q "^-[X]log:gc"
{code}
Should be:
{code}
echo "$JVM_OPTS" | grep -qe "-[X]log:gc"
{code}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
Right now, jvm11-server.options with:
{code:java}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M{code}
Will generate process of:
{code:java}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
{code}
With fix it will generate correct gc option of :
{code}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
{code}
was:
Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
[https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
The following command incorrect:
{{echo "$JVM_OPTS" | grep -q "^-[X]log:gc"}}
Should be:
{{echo "$JVM_OPTS" | grep -qe "-[X]log:gc"}}
The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always return
1, remove {{^}} and add {{-qe}} will solve the problem.
It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
{{jvm11-server.options}} and {{jvm8-server.options}}
Right now, jvm11-server.options with:
{code:java}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M{code}
Will generate process of:
{code:java}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
{code}
With fix it will generate correct gc option of :
{code}
-Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
{code}
> Incorrect check for -Xlog in cassandra-env.sh
> --
>
> Key: CASSANDRA-16279
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16279
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Yakir Gibraltar
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Possible small bug in {{cassandra-env.sh}} of Cassandra 4:
> [https://github.com/apache/cassandra/blob/trunk/conf/cassandra-env.sh#L98]
> The following command incorrect:
> {code}
> echo "$JVM_OPTS" | grep -q "^-[X]log:gc"
> {code}
> Should be:
> {code}
> echo "$JVM_OPTS" | grep -qe "-[X]log:gc"
> {code}
> The variable {{$JVM_OPTS}} not starting with {{-Xlog..}} , and always
> return 1, remove {{^}} and add {{-qe}} will solve the problem.
> It's causing that {{cassandra-env.sh}} ignoring variable of {{-Xlog}} in
> {{jvm11-server.options}} and {{jvm8-server.options}}
>
> Right now, jvm11-server.options with:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M{code}
> Will generate process of:
> {code:java}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
>
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=10485760
> {code}
> With fix it will generate correct gc option of :
> {code}
> -Xlog:gc=info,heap*=trace,age*=debug,safepoint=info,promotion*=trace:file=/var/log/cassandra/gc.log:time,uptime,pid,tid,level:filecount=10,filesize=100M
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.or
[jira] [Commented] (CASSANDRA-16173) Update "Getting Started" document for Windows users
[ https://issues.apache.org/jira/browse/CASSANDRA-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233576#comment-17233576 ] Berenguer Blasi commented on CASSANDRA-16173: - I thought this one was about removing any doc references to windows scripts that have been removed, rather than how to use C* on Windows nowadays as CASSANDRA-15887. So imo this one is mandatory for 4.0 but not the other one. If that makes sense ofc :-) > Update "Getting Started" document for Windows users > --- > > Key: CASSANDRA-16173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16173 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Yuki Morishita >Priority: Normal > Fix For: 4.0 > > > This is a documentation follow up to CASSANDRA-16171. > Since we are removing support for Windows, we should update ["Getting > Started" > guide|https://cassandra.apache.org/doc/latest/getting_started/index.html] to > include how-to's for Windows users for setting up Cassandra for dev > evnironment. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[ https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233591#comment-17233591 ] Michael Semb Wever commented on CASSANDRA-14477: +1 on PRs. CI runs for [3.0|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/210/], [3.11|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/211/], and [trunk|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/212/pipeline]. > The check of num_tokens against the length of inital_token in the yaml > triggers unexpectedly > > > Key: CASSANDRA-14477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14477 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Vincent White >Assignee: Stefan Miklosovic >Priority: Low > Time Spent: 40m > Remaining Estimate: 0h > > In CASSANDRA-10120 we added a check that compares num_tokens against the > number of tokens supplied in the yaml via initial_token. From my reading of > CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained > contradictory values for num_tokens and initial_tokens which should help > prevent misconfiguration via human error. The current behaviour appears to > differ slightly in that it performs this comparison regardless of whether > num_tokens is included in the yaml or not. Below are proposed patches to only > perform the check if both options are present in the yaml. > ||Branch|| > |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]| > |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16173) Update "Getting Started" document for Windows users
[ https://issues.apache.org/jira/browse/CASSANDRA-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233594#comment-17233594 ] Yuki Morishita commented on CASSANDRA-16173: I meant this ticket to be adding the doc for Windows users how to run Cassandra on Windows after we removed all the scripts. So in that sense, this is a dupe for CASSANDRA-15887. And in my opinion, it is mandatory for 4.0 to add how to run on windows using WSL2/Docker, which is CASSANDRA-15887. > Update "Getting Started" document for Windows users > --- > > Key: CASSANDRA-16173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16173 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Yuki Morishita >Priority: Normal > Fix For: 4.0 > > > This is a documentation follow up to CASSANDRA-16171. > Since we are removing support for Windows, we should update ["Getting > Started" > guide|https://cassandra.apache.org/doc/latest/getting_started/index.html] to > include how-to's for Windows users for setting up Cassandra for dev > evnironment. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15833) Unresolvable false digest mismatch during upgrade due to CASSANDRA-10657
[ https://issues.apache.org/jira/browse/CASSANDRA-15833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15833: --- Fix Version/s: (was: 4.0-beta) (was: 3.11.x) 3.11.9 4.0-beta3 > Unresolvable false digest mismatch during upgrade due to CASSANDRA-10657 > > > Key: CASSANDRA-15833 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15833 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 3.11.9, 4.0-beta3 > > Attachments: CASSANDRA-15833-3.11.patch, CASSANDRA-15833-4.0.patch > > > CASSANDRA-10657 introduced changes in how the ColumnFilter is interpreted. > This results in digest mismatch when querying incomplete set of columns from > a table with consistency that requires reaching instances running pre > CASSANDRA-10657 from nodes that include CASSANDRA-10657 (it was introduced in > Cassandra 3.4). > The fix is to bring back the previous behaviour until there are no instances > running pre CASSANDRA-10657 version. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
Alexander Dejanovski created CASSANDRA-16280:
Summary: SSTableLoader will fail if encryption parameters are used
due to CASSANDRA-16144
Key: CASSANDRA-16280
URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
Project: Cassandra
Issue Type: Bug
Reporter: Alexander Dejanovski
Assignee: Alexander Dejanovski
CASSANDRA-16144 recently introduced [repeated calls
|https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
_clientEncOptions.applyConfig()_ for each encryption parameter passed to the
sstableloader command line.
This consistently fails because _applyConfig()_ can be called only once due to
the _ensureConfigNotApplied()_ check at the beginning of the method.
This call is not necessary since the _with...()_ methods will invoke
_applyConfig()_ each time:
{code:java}
public EncryptionOptions withTrustStore(String truststore)
{
return new EncryptionOptions(keystore, keystore_password, truststore,
truststore_password, cipher_suites,
protocol, algorithm, store_type,
require_client_auth, require_endpoint_verification,
enabled, optional).applyConfig();
}
{code}
I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Dejanovski updated CASSANDRA-16280:
-
Component/s: Tool/bulk load
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15164) Overflowed Partition Cell Histograms Can Prevent Compactions from Executing
[ https://issues.apache.org/jira/browse/CASSANDRA-15164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15164: --- Fix Version/s: (was: 4.0-beta) 4.0-beta3 > Overflowed Partition Cell Histograms Can Prevent Compactions from Executing > --- > > Key: CASSANDRA-15164 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15164 > Project: Cassandra > Issue Type: Bug > Components: CQL/Interpreter >Reporter: Ankur Jha >Assignee: Caleb Rackliffe >Priority: Urgent > Labels: compaction, partition > Fix For: 3.0.23, 3.11.9, 4.0-beta3 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > Hi, we are running 6 node Cassandra cluster in production with 3 seed node > but from last night one of our seed nodes is continuously throwing an error > like this;- > cassandra.protocol.ServerError: message="java.lang.IllegalStateException: Unable to compute ceiling for max > when histogram overflowed"> > For a cluster to be up and running I Drained this node. > Can somebody help me out with this? > > Any help or lead would be appreciated > > Note : We are using Cassandra version 3.7 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Dejanovski updated CASSANDRA-16280:
-
Fix Version/s: 4.0-beta
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Dejanovski updated CASSANDRA-16280:
-
Bug Category: Parent values: Availability(12983)Level 1 values: Process
Crash(12992)
Complexity: Normal
Discovered By: User Report
Severity: Critical
Status: Open (was: Triage Needed)
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Dejanovski updated CASSANDRA-16280:
-
Test and Documentation Plan:
Regression test added under {{LoaderOptionsTest.testEncryptionSettings}},
invoking {{LoaderOptions.builder().parseArgs()}} with all the encryption
options.
Failure with the current trunk:
{code:java}
test:
[echo] Number of test runners: 3
[mkdir] Created dir:
/Users/adejanovski/projets/cassandra/thelastpickle/cassandra/build/test/cassandra
[mkdir] Created dir:
/Users/adejanovski/projets/cassandra/thelastpickle/cassandra/build/test/output
[junit-timeout] Testsuite: org.apache.cassandra.tools.LoaderOptionsTest
[junit-timeout] Testsuite: org.apache.cassandra.tools.LoaderOptionsTest Tests
run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0,824 sec
[junit-timeout]
[junit-timeout] Testcase:
testEncryptionSettings(org.apache.cassandra.tools.LoaderOptionsTest): Caused an
ERROR
[junit-timeout] EncryptionOptions cannot be changed after configuration applied
[junit-timeout] java.lang.IllegalStateException: EncryptionOptions cannot be
changed after configuration applied
[junit-timeout] at
org.apache.cassandra.config.EncryptionOptions.ensureConfigNotApplied(EncryptionOptions.java:162)
[junit-timeout] at
org.apache.cassandra.config.EncryptionOptions.applyConfig(EncryptionOptions.java:130)
[junit-timeout] at
org.apache.cassandra.tools.LoaderOptions$Builder.parseArgs(LoaderOptions.java:478)
[junit-timeout] at
org.apache.cassandra.tools.LoaderOptionsTest.testEncryptionSettings(LoaderOptionsTest.java:55)
[junit-timeout]
[junit-timeout]
[junit-timeout] Test org.apache.cassandra.tools.LoaderOptionsTest FAILED
{code}
The test passes with the patch:
{code:java}
test:
[echo] Number of test runners: 3
[junit-timeout] Testsuite: org.apache.cassandra.tools.LoaderOptionsTest
[junit-timeout] Testsuite: org.apache.cassandra.tools.LoaderOptionsTest Tests
run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0,5 sec
BUILD SUCCESSFUL
{code}
Status: Patch Available (was: In Progress)
Here's the patch:
* [branch|https://github.com/thelastpickle/cassandra/tree/CASSANDRA-16280]
*
[commit|https://github.com/thelastpickle/cassandra/commit/dbce40a06d89c415cbe172e4726b6c4bb38fe4c9]
I'm waiting for the build to go through in CircleCI.
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15584) 4.0 quality testing: Tooling - External Ecosystem
[
https://issues.apache.org/jira/browse/CASSANDRA-15584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233676#comment-17233676
]
Alexander Dejanovski commented on CASSANDRA-15584:
--
I found two problems when investigating the failing tests with Medusa:
* pre-4.0 Cassandra was apparently more permissive with missing ciphers when
sstableloader was invoked. I've added the path to cassandra.yaml in the
sstableloader call which fixed the issue (PR pending merge).
* [CASSANDRA-16144|https://issues.apache.org/jira/browse/CASSANDRA-16144]
recently introduced a bug in parsing loader options with encryption args. I've
created [CASSANDRA-16280|https://issues.apache.org/jira/browse/CASSANDRA-16280]
to track the issue along with a fix.
I'll update the table in this ticket's description once trunk is fixed and the
Medusa PR gets merged.
> 4.0 quality testing: Tooling - External Ecosystem
> -
>
> Key: CASSANDRA-15584
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15584
> Project: Cassandra
> Issue Type: Task
> Components: Tool/external
>Reporter: Josh McKenzie
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Reference [doc from
> NGCC|https://docs.google.com/document/d/1uhUOp7wpE9ZXNDgxoCZHejHt5SO4Qw1dArZqqsJccyQ/edit#]
> for context.
> *Shepherd: Benjamin Lerer*
> Many users of Apache Cassandra employ open source tooling to automate
> Cassandra configuration, runtime management, and repair scheduling. Prior to
> release, we need to confirm that popular third-party tools function properly.
> Current list of tools:
> || Name || Status || Contact ||
> | [Priam|http://netflix.github.io/Priam/] |{color:#00875A} *DONE WITH
> ALPHA*{color} (need to be tested with beta) | [~sumanth.pasupuleti]|
> | [sstabletools|https://github.com/instaclustr/cassandra-sstable-tools] |
> *NOT STARTED* | [~stefan.miklosovic]|
> | [cassandra-exporter|https://github.com/instaclustr/cassandra-exporter]|
> *NOT STARTED* | [~stefan.miklosovic]|
> | [Instaclustr Cassandra
> operator|https://github.com/instaclustr/cassandra-operator]|
> {color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
> | [Instaclustr Esop |
> https://github.com/instaclustr/instaclustr-esop]|{color:#00875A}*DONE*{color}
> | [~stefan.miklosovic]|
> | [Instaclustr Icarus |
> https://github.com/instaclustr/instaclustr-icarus]|{color:#00875A}*DONE*{color}
> | [~stefan.miklosovic]|
> | [Cassandra SSTable generator |
> https://github.com/instaclustr/cassandra-sstable-generator]|{color:#00875A}*DONE*{color}|
> [~stefan.miklosovic]|
> | [Cassandra TTL Remover | https://github.com/instaclustr/TTLRemover] |
> {color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
> | [Cassandra Everywhere Strategy |
> https://github.com/instaclustr/cassandra-everywhere-strategy] |
> {color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
> | [Cassandra LDAP Authenticator |
> https://github.com/instaclustr/cassandra-ldap] | {color:#00875A}*DONE*{color}
> | [~stefan.miklosovic]|
> | [Instaclustr Minotaur |
> https://github.com/instaclustr/instaclustr-minotaur] |
> {color:#00875A}*DONE*{color} | [~stefan.miklosovic]|
> | [Reaper|http://cassandra-reaper.io/]| {color:#00875A}*AUTOMATIC*{color} |
> [~adejanovski]|
> | [Medusa|https://github.com/thelastpickle/cassandra-medusa]| *IN PROGRESS*|
> [~adejanovski]|
> | [Casskop|https://orange-opensource.github.io/casskop/]| *NOT STARTED*|
> Franck Dehay|
> |
> [spark-cassandra-connector|https://github.com/datastax/spark-cassandra-connector]|
> {color:#00875A}*DONE*{color}| [~jtgrabowski]|
> | [cass operator|https://github.com/datastax/cass-operator]|
> {color:#00875A}*DONE*{color}| [~jimdickinson]|
> | [metric
> collector|https://github.com/datastax/metric-collector-for-apache-cassandra]|
> {color:#00875A}*DONE*{color}| [~tjake]|
> | [managment
> API|https://github.com/datastax/management-api-for-apache-cassandra]|
> {color:#00875A}*DONE*{color}| [~tjake]|
> Columns descriptions:
> * *Name*: Name and link to the tool official page
> * *Status*: {{NOT STARTED}}, {{IN PROGRESS}}, {{BLOCKED}} if you hit any
> issue and have to wait for it to be solved, {{DONE}}, {{AUTOMATIC}} if
> testing 4.0 is part of your CI process.
> * *Contact*: The person acting as the contact point for that tool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16277) 'SSLEngine closed already' exception on failed outbound connection
[
https://issues.apache.org/jira/browse/CASSANDRA-16277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233704#comment-17233704
]
Alex Petrov commented on CASSANDRA-16277:
-
LGTM, +1.
> 'SSLEngine closed already' exception on failed outbound connection
> --
>
> Key: CASSANDRA-16277
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16277
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
>
> Occasionally Netty will invoke
> {{OutboundConnectionInitiator#exceptionCaught()}} handler to process an
> exception of the following kind:
> {code}
> io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed:
> Connection reset by peer
> {code}
> When we invoke {{ctx.close()}} later in that method, the listener, set up in
> {{channelActive()}}, might be
> failed with an {{SSLException("SSLEngine closed already”)}} by Netty, and
> {{exceptionCaught()}} will be invoked
> once again, this time to handle the {{SSLException}} triggered by
> {{ctx.close()}}.
> The exception at this stage is benign, and we shouldn't be double-logging the
> failure to connect.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16272) jvm-dtest nodetool assert apis do not include the new stdout and stderr in the failure message
[ https://issues.apache.org/jira/browse/CASSANDRA-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233706#comment-17233706 ] Alex Petrov commented on CASSANDRA-16272: - +1, LGTM! > jvm-dtest nodetool assert apis do not include the new stdout and stderr in > the failure message > -- > > Key: CASSANDRA-16272 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16272 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We added support for nodetool to store stdout and stderr but the failure > message wasn’t updated to include it -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16272) jvm-dtest nodetool assert apis do not include the new stdout and stderr in the failure message
[ https://issues.apache.org/jira/browse/CASSANDRA-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-16272: Reviewers: Alex Petrov Status: Review In Progress (was: Patch Available) > jvm-dtest nodetool assert apis do not include the new stdout and stderr in > the failure message > -- > > Key: CASSANDRA-16272 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16272 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We added support for nodetool to store stdout and stderr but the failure > message wasn’t updated to include it -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16259) tablehistograms cause ArrayIndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/CASSANDRA-16259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233709#comment-17233709
]
Benjamin Lerer commented on CASSANDRA-16259:
Sorry, I am sick since a couple of day and do not have the energy to dig into
the code.
If I am not mistaken the problem can be solve by triggering an index summary
redistribution using JMX ({{IndexSummaryManager.redistributeSummaries()}}). See
CASSANDRA-5519 for a better understanding of the impact.
Hope it help.
> tablehistograms cause ArrayIndexOutOfBoundsException
>
>
> Key: CASSANDRA-16259
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16259
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Metrics
>Reporter: Justin Montgomery
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> After upgrading some nodes in our cluster from 3.11.8 to 3.11.9 an error
> appeared on the upgraded nodes when trying to access *tablehistograms*. The
> same command run on our .8 nodes return as expected, only the upgraded .9
> nodes fail. Not all tables fail when queried, but about 90% of them do.
> We use Datastax MCAC which appears to query histograms every 30 seconds, this
> outputs to the system.log:
> {noformat}
> WARN [insights-3-1] 2020-11-09 01:11:22,331 UnixSocketClient.java:830 -
> Error reporting:
> java.lang.ArrayIndexOutOfBoundsException: 115
> at
> org.apache.cassandra.metrics.TableMetrics.combineHistograms(TableMetrics.java:261)
> ~[apache-cassandra-3.11.9.jar:3.11.9]
> at
> org.apache.cassandra.metrics.TableMetrics.access$000(TableMetrics.java:48)
> ~[apache-cassandra-3.11.9.jar:3.11.9]
> at
> org.apache.cassandra.metrics.TableMetrics$11.getValue(TableMetrics.java:376)
> ~[apache-cassandra-3.11.9.jar:3.11.9]
> at
> org.apache.cassandra.metrics.TableMetrics$11.getValue(TableMetrics.java:373)
> ~[apache-cassandra-3.11.9.jar:3.11.9]
> at
> com.datastax.mcac.UnixSocketClient.writeMetric(UnixSocketClient.java:839)
> [datastax-mcac-agent.jar:na]
> at
> com.datastax.mcac.UnixSocketClient.access$700(UnixSocketClient.java:78)
> [datastax-mcac-agent.jar:na]
> at
> com.datastax.mcac.UnixSocketClient$2.lambda$onGaugeAdded$0(UnixSocketClient.java:626)
> ~[datastax-mcac-agent.jar:na]
> at
> com.datastax.mcac.UnixSocketClient.writeGroup(UnixSocketClient.java:819)
> [datastax-mcac-agent.jar:na]
> at
> com.datastax.mcac.UnixSocketClient.lambda$restartMetricReporting$2(UnixSocketClient.java:798)
> [datastax-mcac-agent.jar:na]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_272]
> at
> io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:126)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:307)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at
> io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
> ~[netty-all-4.0.44.Final.jar:4.0.44.Final]
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_272]{noformat}
> Manually trying a histogram from the CLI:
> {noformat}
> $ nodetool tablehistograms logdata log_height_index
> error: 115
> -- StackTrace --
> java.lang.ArrayIndexOutOfBoundsException: 115
> at
> org.apache.cassandra.metrics.TableMetrics.combineHistograms(TableMetrics.java:261)
> at
> org.apache.cassandra.metrics.TableMetrics.access$000(TableMetrics.java:48)
> at
> org.apache.cassandra.metrics.TableMetrics$11.getValue(TableMetrics.java:376)
> at
> org.apache.cassandra.metrics.TableMetrics$11.getValue(TableMetrics.java:373)
> at
> org.apache.cassandra.metrics.CassandraMetricsRegistry$JmxGauge.getValue(CassandraMetricsRegistry.java:250)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorIm
[jira] [Commented] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233712#comment-17233712
]
Jon Meredith commented on CASSANDRA-16280:
--
+1 from me, thanks for the test and fix. Apologies for the breakage, the code
was left over from an earlier version of the patch but I missed
{{SSTableLoader}} when I refactored the {{applyConfig}}.
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-16226:
Test and Documentation Plan:
The patch includes a series of new tests in {{SSTablesIteratedTest}} that
verify expected numbers of SSTables read for several query types across compact
and non-compact tables. They should serve as reasonable documentation and
guardrails against further regression.
The official [docs on compact
storage|https://cassandra.apache.org/doc/latest/cql/appendices.html#appendix-c-dropping-compact-storage]
will need some rework as well, both to indicate that it will live on in 4.0,
and to take into account the concerns in this issue.
was:
The patch includes a new in-JVM upgrade test that exercises the 2.2 -> 3.0
upgrade path were an existing table uses COMPACT STORAGE and we hit the
particular scenario outlined in the description. Outside that, the interesting
parts of the regression suite are the tests around row deletions and cell
deletions where only a primary key remains.
Performance should also be improved significantly in this scenario, so a smoke
test to verify that would be helpful. (There should be zero impact on
non-compact tables.)
Status: Patch Available (was: In Progress)
I've posted (the 3.0 version of) an alternative solution here, which should
merge up cleanly, given {{TableMetadata#isCQLTable()}} is still present on
trunk after CASSANDRA-16217: https://github.com/apache/cassandra/pull/823
(CircleCI is having difficulty with the unit tests, but there is a partial run
[here|https://app.circleci.com/pipelines/github/maedhroz/cassandra/155/workflows/cc54c092-cac1-4144-b9be-577ea4c8a78e].)
My goals w/ this patch were the following:
1.) Maintain the empty row semantics of tables that have either always been
compact or always been non-compact (see previous comment ^), and preserve as
much of them as possible after dropping compact storage, given the historical
limitations of the compact format.
2.) Either improve/reduce or leave unchanged the number of SSTables read in all
cases for tables that have never been compact.
3.) Fix the regression around the number of SSTables read in the original
description of this issue for compact tables, and avoid having the regression
re-introduced when and if compact storage is dropped.
4.) Avoid creating a new SSTable format or burdening operators with running
{{upgradesstables}} again to make all of this above work.
Although there are some issues w/ the unit tests on CI, the relevant local
tests are looking clean, including {{operations.DeleteTest}} and
{{operations.UpgradeTest}}, which verify item #1 above, and the other points
are by and large satisfied. For point #2, there was actually a case where pure
non-compact tables appeared to be reading too many SSTables w/ updates present
(although that [needs to be
reviewed|https://github.com/apache/cassandra/pull/823/files?file-filters%5B%5D=.java#r524809272]).
One drawback of this approach is that when a compact table with existing
SSTables has compact storage dropped, those SSTables still do not contain
primary key liveness info, and therefore may continue to return zero rows
rather than rows w/ primary keys and null regular columns. (Data written after
the drop will, of course, have liveness info, and so mixed behavior is
possible.) I've discussed a few ideas w/ [~jwest] and [~ifesdjeen] around this,
and I'm not convinced any of them would be valuable. (Keep in mind that
applications written against compact storage tables will already be handling
this case, and good documentation will help even further.)
The first is having enough information, either via a sequence number or a new
piece of metadata, to identify whether an SSTable was created while a table was
still compact. Having this data might be useful in some other way, but it still
does not insert primary key liveness info into pre-drop SSTables. DELETE and
UPDATE operations never have primary key liveness information attached to them,
even for pure non-compact tables, so an insert pre-drop in one SSTable,
followed by a delete in a post-drop SSTable, will still translate to the
absence of a row rather than a row with null regular cells.
The second broad idea is to somehow insert primary key liveness information
into pre-drop SSTables, but this is problematic for at least a couple reasons.
If we make it a prerequisite for running DROP COMPACT STORAGE, it means
physically rewriting SSTables. (To be fair, this may not be a huge additional
burden for those still on 2.x, who would have to do this anyway.) The other
oddity is that there doesn't appear to be any existing procedure for
synthesizing primary key liveness info. In the non-compact world, it is carried
only by INSERT operations, not UPDATE or DELETE operations. It's not c
[jira] [Comment Edited] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233742#comment-17233742
]
Caleb Rackliffe edited comment on CASSANDRA-16226 at 11/17/20, 5:09 PM:
I've posted (the 3.0 version of) an alternative solution here, which should
merge up cleanly, given {{TableMetadata#isCQLTable()}} is still present on
trunk after CASSANDRA-16217: https://github.com/apache/cassandra/pull/823
(CircleCI is having difficulty with the unit tests, but there is a partial run
[here|https://app.circleci.com/pipelines/github/maedhroz/cassandra/155/workflows/cc54c092-cac1-4144-b9be-577ea4c8a78e].)
My goals w/ this patch were the following:
1.) Maintain the empty row semantics of tables that have either always been
compact or always been non-compact (see
[previous|https://issues.apache.org/jira/browse/CASSANDRA-16226?focusedCommentId=17233168&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17233168]
comment), and preserve as much of them as possible after dropping compact
storage, given the historical limitations of the compact format.
2.) Either improve/reduce or leave unchanged the number of SSTables read in all
cases for tables that have never been compact.
3.) Fix the regression around the number of SSTables read in the original
description of this issue for compact tables, and avoid having the regression
re-introduced when and if compact storage is dropped.
4.) Avoid creating a new SSTable format or burdening operators with running
{{upgradesstables}} again to make all of this above work.
Although there are some issues w/ the unit tests on CI, the relevant local
tests are looking clean, including {{operations.DeleteTest}} and
{{operations.UpgradeTest}}, which verify item #1 above, and the other points
are by and large satisfied. For point #2, there was actually a case where pure
non-compact tables appeared to be reading too many SSTables w/ updates present
(although that [needs to be
reviewed|https://github.com/apache/cassandra/pull/823/files?file-filters%5B%5D=.java#r524809272]).
One drawback of this approach is that when a compact table with existing
SSTables has compact storage dropped, those SSTables still do not contain
primary key liveness info, and therefore may continue to return zero rows
rather than rows w/ primary keys and null regular columns. (Data written after
the drop will, of course, have liveness info, and so mixed behavior is
possible.) I've discussed a few ideas w/ [~jwest] and [~ifesdjeen] around this,
and I'm not convinced any of them would be valuable. (Keep in mind that
applications written against compact storage tables will already be handling
this case, and good documentation will help even further.)
The first is having enough information, either via a sequence number or a new
piece of metadata, to identify whether an SSTable was created while a table was
still compact. Having this data might be useful in some other way, but it still
does not insert primary key liveness info into pre-drop SSTables. DELETE and
UPDATE operations never have primary key liveness information attached to them,
even for pure non-compact tables, so an insert pre-drop in one SSTable,
followed by a delete in a post-drop SSTable, will still translate to the
absence of a row rather than a row with null regular cells.
The second broad idea is to somehow insert primary key liveness information
into pre-drop SSTables, but this is problematic for at least a couple reasons.
If we make it a prerequisite for running DROP COMPACT STORAGE, it means
physically rewriting SSTables. (To be fair, this may not be a huge additional
burden for those still on 2.x, who would have to do this anyway.) The other
oddity is that there doesn't appear to be any existing procedure for
synthesizing primary key liveness info. In the non-compact world, it is carried
only by INSERT operations, not UPDATE or DELETE operations. It's not clear to
me how we would determine where to add it during a {{runsstableupgrade}} run,
particularly how we would know the difference between INSERT and UPDATE
starting with just a compact SSTable on disk.
To summarize, I think tables that are purely compact or non-compact should be
handled pretty well w/ this patch (and will strictly git all the goals above,
where applicable). The interesting case is the mixed compact/non-compact
SSTable set caused by dropping compact storage. In this case, I think it makes
reasonable trade-offs and avoids all the major pain points for operators.
Whatever direction we go, the official docs around compact storage are going to
need updating.
was (Author: maedhroz):
I've posted (the 3.0 version of) an alternative solution here, which should
merge up cleanly, given {{TableMetadata#isCQLTable()}} is still present on
trunk after CASSANDRA-16
[jira] [Commented] (CASSANDRA-16212) Cassandra version above 3.11.0 failing for ARM64
[
https://issues.apache.org/jira/browse/CASSANDRA-16212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233778#comment-17233778
]
David Capwell commented on CASSANDRA-16212:
---
[~mck], [~jolynch], and I have been talking in slack, moving the conversation
here.
After this got committed several of the JVM dtests started failing on Circle
CI; there were only a small number of expected test failures, now several are
failing.
Below is a simple reproducer
{code}
./ci-test org.apache.cassandra.distributed.test.SimpleReadWriteTest
...
[junit-timeout] Testcase:
org.apache.cassandra.distributed.test.SimpleReadWriteTest:writeWithSchemaDisagreement2:
Caused an ERROR
[junit-timeout] Forked Java VM exited abnormally. Please note the time in the
report does not reflect the time until the VM exit.
[junit-timeout] junit.framework.AssertionFailedError: Forked Java VM exited
abnormally. Please note the time in the report does not reflect the time until
the VM exit.
[junit-timeout] at java.util.Vector.forEach(Vector.java:1275)
[junit-timeout] at java.util.Vector.forEach(Vector.java:1275)
[junit-timeout] at java.lang.Thread.run(Thread.java:748)
[junit-timeout]
[junit-timeout]
[junit-timeout] Test org.apache.cassandra.distributed.test.SimpleReadWriteTest
FAILED (crashed)BUILD FAILED
/Users/davidcapwell/src/github/apache/cassandra-trunk/build.xml:2035: The
following error occurred while executing this line:
/Users/davidcapwell/src/github/apache/cassandra-trunk/build.xml:1916: Some
test(s) failed.
{code}
{code}
$ cat ci-test
#!/usr/bin/env bash
#set -o xtrace
set -o errexit
set -o pipefail
set -o nounset
usage() {
if [[ $# -gt 0 ]]; then
echo "$*" 1>&2
fi
cat < Cassandra version above 3.11.0 failing for ARM64
> -
>
> Key: CASSANDRA-16212
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16212
> Project: Cassandra
> Issue Type: Task
> Components: Local/Other
>Reporter: odidev
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-beta4
>
> Attachments: crash.txt
>
>
> Cassandra versions above 3.11.0 are failing on ARM64 platform with below
> issue:
> java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna3214742498288082263.tmp:
> *Error loading shared library ld-linux-aarch64.so.1: No such file or
> directory (needed by /tmp/jna-3506402/jna3214742498288082263.tmp)*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16212) Cassandra version above 3.11.0 failing for ARM64
[
https://issues.apache.org/jira/browse/CASSANDRA-16212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233784#comment-17233784
]
David Capwell commented on CASSANDRA-16212:
---
Here is what I see in the logs
{code}
$ egrep -r 'Error|Exception' build/test/logs/ | egrep -v 'no libsigar|The JVM
is not configured to stop|expanded subtype'
build/test/logs//org.apache.cassandra.distributed.test.SimpleReadWriteTest/cluster-be7276b7-2abf-457c-ab52-8c927c06fb83/node1/system.log:io.netty.channel.AbstractChannel$AnnotatedConnectException:
Connection refused: /127.0.0.3:7012
build/test/logs//org.apache.cassandra.distributed.test.SimpleReadWriteTest/cluster-be7276b7-2abf-457c-ab52-8c927c06fb83/node1/system.log:Caused
by: java.net.ConnectException: Connection refused
build/test/logs//org.apache.cassandra.distributed.test.SimpleReadWriteTest/cluster-be7276b7-2abf-457c-ab52-8c927c06fb83/node3/system.log:org.apache.cassandra.exceptions.UnknownColumnException:
Unknown column v2 during deserialization
build/test/logs//org.apache.cassandra.distributed.test.SimpleReadWriteTest/cluster-be7276b7-2abf-457c-ab52-8c927c06fb83/node2/system.log:org.apache.cassandra.exceptions.UnknownColumnException:
Unknown column v2 during deserialization
build/test/logs//org.apache.cassandra.distributed.test.SimpleReadWriteTest/cluster-6f55f3c8-856c-4f23-a2e0-82bf9af6d470/node1/system.log:io.netty.channel.AbstractChannel$AnnotatedConnectException:
Connection refused: /127.0.0.3:7012
build/test/logs//org.apache.cassandra.distributed.test.SimpleReadWriteTest/cluster-6f55f3c8-856c-4f23-a2e0-82bf9af6d470/node1/system.log:Caused
by: java.net.ConnectException: Connection refused
{code}
I do not see any jvm error file, nor do I see anything related to OOMing.
> Cassandra version above 3.11.0 failing for ARM64
> -
>
> Key: CASSANDRA-16212
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16212
> Project: Cassandra
> Issue Type: Task
> Components: Local/Other
>Reporter: odidev
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-beta4
>
> Attachments: crash.txt
>
>
> Cassandra versions above 3.11.0 are failing on ARM64 platform with below
> issue:
> java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna3214742498288082263.tmp:
> *Error loading shared library ld-linux-aarch64.so.1: No such file or
> directory (needed by /tmp/jna-3506402/jna3214742498288082263.tmp)*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16212) Cassandra version above 3.11.0 failing for ARM64
[
https://issues.apache.org/jira/browse/CASSANDRA-16212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233791#comment-17233791
]
David Capwell commented on CASSANDRA-16212:
---
Here is the same thing on jdk11
{code}
$ setjdk 11 ; export CASSANDRA_USE_JDK11=true
$ ./ci-test org.apache.cassandra.distributed.test.SimpleReadWriteTest
...
testclasslist:
[testparallelhelper] Warning: Nashorn engine is planned to be removed from a
future JDK release
[echo] Number of test runners: 1
[mkdir] Created dir:
/Users/davidcapwell/src/github/apache/cassandra-trunk/build/test/cassandra
[mkdir] Created dir:
/Users/davidcapwell/src/github/apache/cassandra-trunk/build/test/output
[junit-timeout] Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
[junit-timeout] OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was
deprecated in version 9.0 and will likely be removed in a future release.
[junit-timeout] Testsuite:
org.apache.cassandra.distributed.test.SimpleReadWriteTest
[junit-timeout] Testsuite:
org.apache.cassandra.distributed.test.SimpleReadWriteTest Tests run: 14,
Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 118.15 sec
[junit-timeout]
BUILD SUCCESSFUL
Total time: 2 minutes 4 seconds
{code}
> Cassandra version above 3.11.0 failing for ARM64
> -
>
> Key: CASSANDRA-16212
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16212
> Project: Cassandra
> Issue Type: Task
> Components: Local/Other
>Reporter: odidev
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-beta4
>
> Attachments: crash.txt
>
>
> Cassandra versions above 3.11.0 are failing on ARM64 platform with below
> issue:
> java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna3214742498288082263.tmp:
> *Error loading shared library ld-linux-aarch64.so.1: No such file or
> directory (needed by /tmp/jna-3506402/jna3214742498288082263.tmp)*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16212) Cassandra version above 3.11.0 failing for ARM64
[ https://issues.apache.org/jira/browse/CASSANDRA-16212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233794#comment-17233794 ] David Capwell commented on CASSANDRA-16212: --- If meatspace runs out, we do normally see this in the logs as we don't kill the JVM when this happens. Yet, the tests pass on jdk 11, where the main difference is how meatspace works (not capped due to jvm bug). > Cassandra version above 3.11.0 failing for ARM64 > - > > Key: CASSANDRA-16212 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16212 > Project: Cassandra > Issue Type: Task > Components: Local/Other >Reporter: odidev >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 4.0-beta4 > > Attachments: crash.txt > > > Cassandra versions above 3.11.0 are failing on ARM64 platform with below > issue: > java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna3214742498288082263.tmp: > *Error loading shared library ld-linux-aarch64.so.1: No such file or > directory (needed by /tmp/jna-3506402/jna3214742498288082263.tmp)* -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16212) Cassandra version above 3.11.0 failing for ARM64
[
https://issues.apache.org/jira/browse/CASSANDRA-16212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233778#comment-17233778
]
David Capwell edited comment on CASSANDRA-16212 at 11/17/20, 6:05 PM:
--
[~mck], [~jolynch], and I have been talking in slack, moving the conversation
here.
After this got committed several of the JVM dtests started failing on Circle
CI; there were only a small number of expected test failures, now several are
failing.
Sample JVM dtest (feature branch after rebasing against trunk):
https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=bug%2FCASSANDRA-16213
Below is a simple reproducer
{code:java}
./ci-test org.apache.cassandra.distributed.test.SimpleReadWriteTest
...
[junit-timeout] Testcase:
org.apache.cassandra.distributed.test.SimpleReadWriteTest:writeWithSchemaDisagreement2:
Caused an ERROR
[junit-timeout] Forked Java VM exited abnormally. Please note the time in the
report does not reflect the time until the VM exit.
[junit-timeout] junit.framework.AssertionFailedError: Forked Java VM exited
abnormally. Please note the time in the report does not reflect the time until
the VM exit.
[junit-timeout] at java.util.Vector.forEach(Vector.java:1275)
[junit-timeout] at java.util.Vector.forEach(Vector.java:1275)
[junit-timeout] at java.lang.Thread.run(Thread.java:748)
[junit-timeout]
[junit-timeout]
[junit-timeout] Test org.apache.cassandra.distributed.test.SimpleReadWriteTest
FAILED (crashed)BUILD FAILED
/Users/davidcapwell/src/github/apache/cassandra-trunk/build.xml:2035: The
following error occurred while executing this line:
/Users/davidcapwell/src/github/apache/cassandra-trunk/build.xml:1916: Some
test(s) failed.
{code}
{code:java}
$ cat ci-test
#!/usr/bin/env bash
#set -o xtrace
set -o errexit
set -o pipefail
set -o nounset
usage() {
if [[ $# -gt 0 ]]; then
echo "$*" 1>&2
fi
cat <&2
fi
cat < Cassandra version above 3.11.0 failing for ARM64
> -
>
> Key: CASSANDRA-16212
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16212
> Project: Cassandra
> Issue Type: Task
> Components: Local/Other
>Reporter: odidev
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.0-beta4
>
> Attachments: crash.txt
>
>
> Cassandra versions above 3.11.0 are failing on ARM64 platform with below
> issue:
> java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna3214742498288082263.tmp:
> *Error loading shared library ld-linux-aarch64.so.1: No such file or
> directory (needed by /tmp/jna-3506402/jna3214742498288082263.tmp)*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16272) jvm-dtest nodetool assert apis do not include the new stdout and stderr in the failure message
[ https://issues.apache.org/jira/browse/CASSANDRA-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16272: -- Status: Ready to Commit (was: Review In Progress) > jvm-dtest nodetool assert apis do not include the new stdout and stderr in > the failure message > -- > > Key: CASSANDRA-16272 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16272 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We added support for nodetool to store stdout and stderr but the failure > message wasn’t updated to include it -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16272) jvm-dtest nodetool assert apis do not include the new stdout and stderr in the failure message
[ https://issues.apache.org/jira/browse/CASSANDRA-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16272: -- Source Control Link: https://github.com/apache/cassandra-in-jvm-dtest-api/commit/3d36cd1890dea0429297e70c1d6dd3813b682b4b Resolution: Fixed Status: Resolved (was: Ready to Commit) > jvm-dtest nodetool assert apis do not include the new stdout and stderr in > the failure message > -- > > Key: CASSANDRA-16272 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16272 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java >Reporter: David Capwell >Assignee: David Capwell >Priority: Low > Fix For: NA > > > We added support for nodetool to store stdout and stderr but the failure > message wasn’t updated to include it -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch trunk updated: nodetool assert apis do not include the new stdout and stderr in the failure message (#25)
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git
The following commit(s) were added to refs/heads/trunk by this push:
new 3d36cd1 nodetool assert apis do not include the new stdout and stderr
in the failure message (#25)
3d36cd1 is described below
commit 3d36cd1890dea0429297e70c1d6dd3813b682b4b
Author: dcapwell
AuthorDate: Tue Nov 17 10:07:44 2020 -0800
nodetool assert apis do not include the new stdout and stderr in the
failure message (#25)
patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-16272
---
.../cassandra/distributed/api/NodeToolResult.java | 4 ++
.../distributed/api/NodeToolOutputTest.java| 48 +-
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
index 6cfe4e5..12be426 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
@@ -202,6 +202,10 @@ public class NodeToolResult
{
StringBuilder sb = new StringBuilder();
sb.append("nodetool command
").append(Arrays.toString(commandAndArgs)).append("
").append(message).append("\n");
+if (stdout != null)
+sb.append("stdout:\n").append(stdout).append("\n");
+if (stderr != null)
+sb.append("stderr:\n").append(stderr).append("\n");
sb.append("Notifications:\n");
for (Notification n : notifications)
sb.append(NodeToolResult.toString(n)).append("\n");
diff --git
a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
index 4420485..4fb9210 100644
--- a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
+++ b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
@@ -18,11 +18,13 @@
package org.apache.cassandra.distributed.api;
+import java.util.Arrays;
import java.util.Collections;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.assertj.core.api.Assertions.catchThrowableOfType;
public class NodeToolOutputTest
@@ -67,8 +69,52 @@ public class NodeToolOutputTest
assertThat(exception.getMessage()).contains("Found unexpected
substring");
}
+@Test
+public void testFailContainsLogs()
+{
+for (int rc : Arrays.asList(0, 42))
+{
+assertThatThrownBy(() -> fail(create(rc, null, null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, null, "this is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageContaining("stderr:\nthis is stderr");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", "this
is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageContaining("stderr:\nthis is stderr");
+}
+}
+
+private static void fail(NodeToolResult result)
+{
+if (result.getRc() == 0)
+{
+result.asserts().failure();
+}
+else
+{
+result.asserts().success();
+}
+}
+
private NodeToolResult create(String stdout, String stderr)
{
-return new NodeToolResult(null, 0, Collections.emptyList(), null,
stdout, stderr);
+return create(0, stdout, stderr);
+}
+
+private NodeToolResult create(int rc, String stdout, String stderr)
+{
+return new NodeToolResult(null, rc, Collections.emptyList(), null,
stdout, stderr);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch trunk updated: nodetool assert apis do not include the new stdout and stderr in the failure message (#25)
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git
The following commit(s) were added to refs/heads/trunk by this push:
new 3d36cd1 nodetool assert apis do not include the new stdout and stderr
in the failure message (#25)
3d36cd1 is described below
commit 3d36cd1890dea0429297e70c1d6dd3813b682b4b
Author: dcapwell
AuthorDate: Tue Nov 17 10:07:44 2020 -0800
nodetool assert apis do not include the new stdout and stderr in the
failure message (#25)
patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-16272
---
.../cassandra/distributed/api/NodeToolResult.java | 4 ++
.../distributed/api/NodeToolOutputTest.java| 48 +-
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
index 6cfe4e5..12be426 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
@@ -202,6 +202,10 @@ public class NodeToolResult
{
StringBuilder sb = new StringBuilder();
sb.append("nodetool command
").append(Arrays.toString(commandAndArgs)).append("
").append(message).append("\n");
+if (stdout != null)
+sb.append("stdout:\n").append(stdout).append("\n");
+if (stderr != null)
+sb.append("stderr:\n").append(stderr).append("\n");
sb.append("Notifications:\n");
for (Notification n : notifications)
sb.append(NodeToolResult.toString(n)).append("\n");
diff --git
a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
index 4420485..4fb9210 100644
--- a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
+++ b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
@@ -18,11 +18,13 @@
package org.apache.cassandra.distributed.api;
+import java.util.Arrays;
import java.util.Collections;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.assertj.core.api.Assertions.catchThrowableOfType;
public class NodeToolOutputTest
@@ -67,8 +69,52 @@ public class NodeToolOutputTest
assertThat(exception.getMessage()).contains("Found unexpected
substring");
}
+@Test
+public void testFailContainsLogs()
+{
+for (int rc : Arrays.asList(0, 42))
+{
+assertThatThrownBy(() -> fail(create(rc, null, null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, null, "this is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageContaining("stderr:\nthis is stderr");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", "this
is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageContaining("stderr:\nthis is stderr");
+}
+}
+
+private static void fail(NodeToolResult result)
+{
+if (result.getRc() == 0)
+{
+result.asserts().failure();
+}
+else
+{
+result.asserts().success();
+}
+}
+
private NodeToolResult create(String stdout, String stderr)
{
-return new NodeToolResult(null, 0, Collections.emptyList(), null,
stdout, stderr);
+return create(0, stdout, stderr);
+}
+
+private NodeToolResult create(int rc, String stdout, String stderr)
+{
+return new NodeToolResult(null, rc, Collections.emptyList(), null,
stdout, stderr);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch trunk updated: nodetool assert apis do not include the new stdout and stderr in the failure message (#25)
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git
The following commit(s) were added to refs/heads/trunk by this push:
new 3d36cd1 nodetool assert apis do not include the new stdout and stderr
in the failure message (#25)
3d36cd1 is described below
commit 3d36cd1890dea0429297e70c1d6dd3813b682b4b
Author: dcapwell
AuthorDate: Tue Nov 17 10:07:44 2020 -0800
nodetool assert apis do not include the new stdout and stderr in the
failure message (#25)
patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-16272
---
.../cassandra/distributed/api/NodeToolResult.java | 4 ++
.../distributed/api/NodeToolOutputTest.java| 48 +-
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
index 6cfe4e5..12be426 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
@@ -202,6 +202,10 @@ public class NodeToolResult
{
StringBuilder sb = new StringBuilder();
sb.append("nodetool command
").append(Arrays.toString(commandAndArgs)).append("
").append(message).append("\n");
+if (stdout != null)
+sb.append("stdout:\n").append(stdout).append("\n");
+if (stderr != null)
+sb.append("stderr:\n").append(stderr).append("\n");
sb.append("Notifications:\n");
for (Notification n : notifications)
sb.append(NodeToolResult.toString(n)).append("\n");
diff --git
a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
index 4420485..4fb9210 100644
--- a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
+++ b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
@@ -18,11 +18,13 @@
package org.apache.cassandra.distributed.api;
+import java.util.Arrays;
import java.util.Collections;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.assertj.core.api.Assertions.catchThrowableOfType;
public class NodeToolOutputTest
@@ -67,8 +69,52 @@ public class NodeToolOutputTest
assertThat(exception.getMessage()).contains("Found unexpected
substring");
}
+@Test
+public void testFailContainsLogs()
+{
+for (int rc : Arrays.asList(0, 42))
+{
+assertThatThrownBy(() -> fail(create(rc, null, null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, null, "this is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageContaining("stderr:\nthis is stderr");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", "this
is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageContaining("stderr:\nthis is stderr");
+}
+}
+
+private static void fail(NodeToolResult result)
+{
+if (result.getRc() == 0)
+{
+result.asserts().failure();
+}
+else
+{
+result.asserts().success();
+}
+}
+
private NodeToolResult create(String stdout, String stderr)
{
-return new NodeToolResult(null, 0, Collections.emptyList(), null,
stdout, stderr);
+return create(0, stdout, stderr);
+}
+
+private NodeToolResult create(int rc, String stdout, String stderr)
+{
+return new NodeToolResult(null, rc, Collections.emptyList(), null,
stdout, stderr);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-in-jvm-dtest-api] branch trunk updated: nodetool assert apis do not include the new stdout and stderr in the failure message (#25)
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-in-jvm-dtest-api.git
The following commit(s) were added to refs/heads/trunk by this push:
new 3d36cd1 nodetool assert apis do not include the new stdout and stderr
in the failure message (#25)
3d36cd1 is described below
commit 3d36cd1890dea0429297e70c1d6dd3813b682b4b
Author: dcapwell
AuthorDate: Tue Nov 17 10:07:44 2020 -0800
nodetool assert apis do not include the new stdout and stderr in the
failure message (#25)
patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-16272
---
.../cassandra/distributed/api/NodeToolResult.java | 4 ++
.../distributed/api/NodeToolOutputTest.java| 48 +-
2 files changed, 51 insertions(+), 1 deletion(-)
diff --git
a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
index 6cfe4e5..12be426 100644
--- a/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
+++ b/src/main/java/org/apache/cassandra/distributed/api/NodeToolResult.java
@@ -202,6 +202,10 @@ public class NodeToolResult
{
StringBuilder sb = new StringBuilder();
sb.append("nodetool command
").append(Arrays.toString(commandAndArgs)).append("
").append(message).append("\n");
+if (stdout != null)
+sb.append("stdout:\n").append(stdout).append("\n");
+if (stderr != null)
+sb.append("stderr:\n").append(stderr).append("\n");
sb.append("Notifications:\n");
for (Notification n : notifications)
sb.append(NodeToolResult.toString(n)).append("\n");
diff --git
a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
index 4420485..4fb9210 100644
--- a/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
+++ b/src/test/java/org/apache/cassandra/distributed/api/NodeToolOutputTest.java
@@ -18,11 +18,13 @@
package org.apache.cassandra.distributed.api;
+import java.util.Arrays;
import java.util.Collections;
import org.junit.jupiter.api.Test;
import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
import static org.assertj.core.api.Assertions.catchThrowableOfType;
public class NodeToolOutputTest
@@ -67,8 +69,52 @@ public class NodeToolOutputTest
assertThat(exception.getMessage()).contains("Found unexpected
substring");
}
+@Test
+public void testFailContainsLogs()
+{
+for (int rc : Arrays.asList(0, 42))
+{
+assertThatThrownBy(() -> fail(create(rc, null, null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", null)))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageNotContaining("stderr:");
+
+assertThatThrownBy(() -> fail(create(rc, null, "this is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageNotContaining("stdout:")
+.hasMessageContaining("stderr:\nthis is stderr");
+
+assertThatThrownBy(() -> fail(create(rc, "this is stdout", "this
is stderr")))
+.isInstanceOf(AssertionError.class)
+.hasMessageContaining("stdout:\nthis is stdout")
+.hasMessageContaining("stderr:\nthis is stderr");
+}
+}
+
+private static void fail(NodeToolResult result)
+{
+if (result.getRc() == 0)
+{
+result.asserts().failure();
+}
+else
+{
+result.asserts().success();
+}
+}
+
private NodeToolResult create(String stdout, String stderr)
{
-return new NodeToolResult(null, 0, Collections.emptyList(), null,
stdout, stderr);
+return create(0, stdout, stderr);
+}
+
+private NodeToolResult create(int rc, String stdout, String stderr)
+{
+return new NodeToolResult(null, rc, Collections.emptyList(), null,
stdout, stderr);
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16280:
--
Reviewers: David Capwell, Jon Meredith, David Capwell (was: David Capwell,
Jon Meredith)
David Capwell, Jon Meredith, David Capwell
Status: Review In Progress (was: Patch Available)
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233803#comment-17233803
]
David Capwell commented on CASSANDRA-16280:
---
LGTM +1
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16280:
--
Status: Ready to Commit (was: Review In Progress)
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233807#comment-17233807
]
David Capwell commented on CASSANDRA-16280:
---
Starting commit
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|trunk|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-16280-trunk-058DDE2C-FE3C-45AE-8FA4-7EB872209E88]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-16280-trunk-058DDE2C-FE3C-45AE-8FA4-7EB872209E88]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/214/]|
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233803#comment-17233803
]
David Capwell edited comment on CASSANDRA-16280 at 11/17/20, 6:23 PM:
--
LGTM +1
I reverted your changes to src/java than ran the test added
{code}
testclasslist:
[echo] Number of test runners: 1
[mkdir] Created dir:
/Users/davidcapwell/src/github/apache/cassandra-oss-commit/build/test/cassandra
[mkdir] Created dir:
/Users/davidcapwell/src/github/apache/cassandra-oss-commit/build/test/output
[junit-timeout] Picked up _JAVA_OPTIONS: -Djava.net.preferIPv4Stack=true
[junit-timeout] Testsuite: org.apache.cassandra.tools.LoaderOptionsTest
[junit-timeout] Testsuite: org.apache.cassandra.tools.LoaderOptionsTest Tests
run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.761 sec
[junit-timeout]
[junit-timeout] Testcase:
testEncryptionSettings(org.apache.cassandra.tools.LoaderOptionsTest): Caused an
ERROR
[junit-timeout] EncryptionOptions cannot be changed after configuration applied
[junit-timeout] java.lang.IllegalStateException: EncryptionOptions cannot be
changed after configuration applied
[junit-timeout] at
org.apache.cassandra.config.EncryptionOptions.ensureConfigNotApplied(EncryptionOptions.java:162)
[junit-timeout] at
org.apache.cassandra.config.EncryptionOptions.applyConfig(EncryptionOptions.java:130)
[junit-timeout] at
org.apache.cassandra.tools.LoaderOptions$Builder.parseArgs(LoaderOptions.java:478)
[junit-timeout] at
org.apache.cassandra.tools.LoaderOptionsTest.testEncryptionSettings(LoaderOptionsTest.java:63)
[junit-timeout]
[junit-timeout]
[junit-timeout] Test org.apache.cassandra.tools.LoaderOptionsTest FAILED
BUILD FAILED
/Users/davidcapwell/src/github/apache/cassandra-oss-commit/build.xml:2035: The
following error occurred while executing this line:
/Users/davidcapwell/src/github/apache/cassandra-oss-commit/build.xml:1916: Some
test(s) failed.
{code}
Tests are able to reproduce the issue!
was (Author: dcapwell):
LGTM +1
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 67f0c34 SSTableLoader will fail if encryption parameters are used due
to CASSANDRA-16144
67f0c34 is described below
commit 67f0c3491a4a1966603ace4fe110b7cfc5b64e9c
Author: Alexander Dejanovski
AuthorDate: Tue Nov 17 10:18:22 2020 -0800
SSTableLoader will fail if encryption parameters are used due to
CASSANDRA-16144
patch by Alexander Dejanovski; reviewed by David Capwell, Jon Meredith for
CASSANDRA-16280
---
.../org/apache/cassandra/tools/LoaderOptions.java | 8
.../apache/cassandra/tools/LoaderOptionsTest.java | 23 +-
2 files changed, 22 insertions(+), 9 deletions(-)
diff --git a/src/java/org/apache/cassandra/tools/LoaderOptions.java
b/src/java/org/apache/cassandra/tools/LoaderOptions.java
index 070ffa8..ca1bd40 100644
--- a/src/java/org/apache/cassandra/tools/LoaderOptions.java
+++ b/src/java/org/apache/cassandra/tools/LoaderOptions.java
@@ -475,13 +475,11 @@ public class LoaderOptions
if (cmd.hasOption(SSL_TRUSTSTORE))
{
clientEncOptions =
clientEncOptions.withTrustStore(cmd.getOptionValue(SSL_TRUSTSTORE));
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(SSL_TRUSTSTORE_PW))
{
clientEncOptions =
clientEncOptions.withTrustStorePassword(cmd.getOptionValue(SSL_TRUSTSTORE_PW));
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(SSL_KEYSTORE))
@@ -489,37 +487,31 @@ public class LoaderOptions
// if a keystore was provided, lets assume we'll need to
use
clientEncOptions =
clientEncOptions.withKeyStore(cmd.getOptionValue(SSL_KEYSTORE))
.withRequireClientAuth(true);
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(SSL_KEYSTORE_PW))
{
clientEncOptions =
clientEncOptions.withKeyStorePassword(cmd.getOptionValue(SSL_KEYSTORE_PW));
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(SSL_PROTOCOL))
{
clientEncOptions =
clientEncOptions.withProtocol(cmd.getOptionValue(SSL_PROTOCOL));
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(SSL_ALGORITHM))
{
clientEncOptions =
clientEncOptions.withAlgorithm(cmd.getOptionValue(SSL_ALGORITHM));
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(SSL_STORE_TYPE))
{
clientEncOptions =
clientEncOptions.withStoreType(cmd.getOptionValue(SSL_STORE_TYPE));
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(SSL_CIPHER_SUITES))
{
clientEncOptions =
clientEncOptions.withCipherSuites(cmd.getOptionValue(SSL_CIPHER_SUITES).split(","));
-clientEncOptions.applyConfig();
}
if (cmd.hasOption(TARGET_KEYSPACE))
diff --git a/test/unit/org/apache/cassandra/tools/LoaderOptionsTest.java
b/test/unit/org/apache/cassandra/tools/LoaderOptionsTest.java
index 6377c59..b0d3c68 100644
--- a/test/unit/org/apache/cassandra/tools/LoaderOptionsTest.java
+++ b/test/unit/org/apache/cassandra/tools/LoaderOptionsTest.java
@@ -44,4 +44,25 @@ public class LoaderOptionsTest
options = LoaderOptions.builder().parseArgs(args2).build();
assertEquals(9142, options.nativePort);
}
-}
\ No newline at end of file
+
+/**
+ * Regression testing for CASSANDRA-16280
+ *
+ * Check that providing encryption parameters to the loader (such as
keystore and truststore) won't break loading
+ * the options.
+ *
+ * @throws Exception
+ */
+@Test
+public void testEncryptionSettings() throws Exception
+{
+String[] args = { "-d", "127.9.9.1", "-ts", "test.jks", "-tspw",
"truststorePass1", "-ks", "test.jks", "-kspw",
+"testdata1", "--ssl-ciphers", "TLS_RSA_WITH_AES_256_CBC_SHA",
+"--ssl-alg", "SunX509", "--store-type", "JKS",
"--ssl-protocol", "TLS",
+sstableDirName("legacy_sstables", "legacy_ma_simple") };
+LoaderOptions options =
LoaderOptions.builder().parseArgs(args).build();
+options = LoaderOptions.builder().parseArgs(args).build();
+assertEquals("test.jks", options.clientEncOptions.keystore);
+}
+}
+
[jira] [Updated] (CASSANDRA-16280) SSTableLoader will fail if encryption parameters are used due to CASSANDRA-16144
[
https://issues.apache.org/jira/browse/CASSANDRA-16280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-16280:
--
Fix Version/s: (was: 4.0-beta)
4.0-beta4
Since Version: 4.0-beta3
Source Control Link:
https://github.com/apache/cassandra/commit/d031c445f6fe9f3639537d69fef1538e09d18410
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> SSTableLoader will fail if encryption parameters are used due to
> CASSANDRA-16144
>
>
> Key: CASSANDRA-16280
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16280
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Alexander Dejanovski
>Assignee: Alexander Dejanovski
>Priority: Normal
> Fix For: 4.0-beta4
>
>
> CASSANDRA-16144 recently introduced [repeated calls
> |https://github.com/apache/cassandra/compare/trunk...dcapwell:commit_remote_branch/CASSANDRA-16144-trunk-209E2350-3A50-457E-A466-F2661CD0D4D1#diff-b87acacbdc34464d327446f7a7e64718dbf843d70f5fbc9e5ddcd1bafca0f441R478]to
> _clientEncOptions.applyConfig()_ for each encryption parameter passed to the
> sstableloader command line.
> This consistently fails because _applyConfig()_ can be called only once due
> to the _ensureConfigNotApplied()_ check at the beginning of the method.
> This call is not necessary since the _with...()_ methods will invoke
> _applyConfig()_ each time:
> {code:java}
> public EncryptionOptions withTrustStore(String truststore)
> {
> return new EncryptionOptions(keystore, keystore_password, truststore,
> truststore_password, cipher_suites,
> protocol, algorithm, store_type,
> require_client_auth, require_endpoint_verification,
> enabled, optional).applyConfig();
> }
> {code}
> I'll build a patch for this with the appropriate unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16217) Minimal 4.0 COMPACT STORAGE backport
[ https://issues.apache.org/jira/browse/CASSANDRA-16217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233871#comment-17233871 ] Ekaterina Dimitrova commented on CASSANDRA-16217: - [~ifesdjeen] Also, I think we can remove this from NEWS.txt: * Cassandra 4.0 removed support for COMPACT STORAGE tables. All Compact Tables have to be migrated using 'ALTER ... DROP COMPACT STORAGE' statement in 3.0/3.11. Cassandra starting 4.0 will not start if flags indicate that the table is non-CQL. Syntax for creating compact tables is also deprecated Should I open a new follow up ticket for test fixes, harry tests, and documentation to be updated? Or even a few of them? > Minimal 4.0 COMPACT STORAGE backport > > > Key: CASSANDRA-16217 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16217 > Project: Cassandra > Issue Type: Bug > Components: Legacy/CQL >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Fix For: 4.0-beta4 > > > There are several behavioural changes related to compact storage, and these > differences are larger than most of us have anticipated: we first thought > there’ll be that “appearing column”, but there’s implicit nulls in > clusterings thing, and row vs column deletion. > Some of the recent issues on the subject are: CASSANDRA-16048, which allows > to ignore these differences. The other one was trying to improve user > experience of anyone still using compact storage: CASSANDRA-15811. > Easily reproducible differernces are: > (1) hidden columns show up, which breaks SELECT * queries > (2) DELETE v and UPDATE v WITH TTL would result into row removals in > non-dense compact tables (CASSANDRA-16069) > (3) INSERT allows skipping clusterings, which are filled with nulls by > default. > Some of these are tricky to support, as 15811 has shown. Anyone on OSS side > who might want to upgrade to 4.0 while still using compact storage might be > affected by being forced into one of these behaviours. > Possible solutions are to document these behaviours, or to bring back a > minimal set of COMPACT STORAGE to keep supporting these. > It looks like it is possible to leave some of the functionality related to > DENSE flag and allow it to be present in 4.0, but only for these three (and > potential related, however not direrclty visible) cases. > [~e.dimitrova] since you were working on removal on compact storage, wanted > to reassure that this is not a revert of your patch. On contrary: your patch > was instrumental in identifying the right places. > cc [~slebresne] [~aleksey] [~benedict] [~marcuse] > |[patch|https://github.com/apache/cassandra/pull/785]|[ci|https://app.circleci.com/pipelines/github/ifesdjeen/cassandra?branch=13994-followup]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233875#comment-17233875 ] Ekaterina Dimitrova commented on CASSANDRA-15588: - Hi [~dcapwell] and [~jmckenzie], Are the upgrade tests going to be covered under this ticket or should we open a separate one for them to be fixed? > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[ https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-14477: --- Status: Patch Available (was: In Progress) > The check of num_tokens against the length of inital_token in the yaml > triggers unexpectedly > > > Key: CASSANDRA-14477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14477 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Vincent White >Assignee: Stefan Miklosovic >Priority: Low > Time Spent: 40m > Remaining Estimate: 0h > > In CASSANDRA-10120 we added a check that compares num_tokens against the > number of tokens supplied in the yaml via initial_token. From my reading of > CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained > contradictory values for num_tokens and initial_tokens which should help > prevent misconfiguration via human error. The current behaviour appears to > differ slightly in that it performs this comparison regardless of whether > num_tokens is included in the yaml or not. Below are proposed patches to only > perform the check if both options are present in the yaml. > ||Branch|| > |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]| > |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[ https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-14477: --- Fix Version/s: 4.0.x 3.11.x 3.0.x > The check of num_tokens against the length of inital_token in the yaml > triggers unexpectedly > > > Key: CASSANDRA-14477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14477 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Vincent White >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 3.0.x, 3.11.x, 4.0.x > > Time Spent: 40m > Remaining Estimate: 0h > > In CASSANDRA-10120 we added a check that compares num_tokens against the > number of tokens supplied in the yaml via initial_token. From my reading of > CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained > contradictory values for num_tokens and initial_tokens which should help > prevent misconfiguration via human error. The current behaviour appears to > differ slightly in that it performs this comparison regardless of whether > num_tokens is included in the yaml or not. Below are proposed patches to only > perform the check if both options are present in the yaml. > ||Branch|| > |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]| > |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[ https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-14477: --- Reviewers: Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Michael Semb Wever, Michael Semb Wever (was: Michael Semb Wever) Status: Review In Progress (was: Patch Available) > The check of num_tokens against the length of inital_token in the yaml > triggers unexpectedly > > > Key: CASSANDRA-14477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14477 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Vincent White >Assignee: Stefan Miklosovic >Priority: Low > Time Spent: 40m > Remaining Estimate: 0h > > In CASSANDRA-10120 we added a check that compares num_tokens against the > number of tokens supplied in the yaml via initial_token. From my reading of > CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained > contradictory values for num_tokens and initial_tokens which should help > prevent misconfiguration via human error. The current behaviour appears to > differ slightly in that it performs this comparison regardless of whether > num_tokens is included in the yaml or not. Below are proposed patches to only > perform the check if both options are present in the yaml. > ||Branch|| > |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]| > |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[ https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-14477: --- Status: Ready to Commit (was: Review In Progress) > The check of num_tokens against the length of inital_token in the yaml > triggers unexpectedly > > > Key: CASSANDRA-14477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14477 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Vincent White >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 3.0.x, 3.11.x, 4.0.x > > Time Spent: 40m > Remaining Estimate: 0h > > In CASSANDRA-10120 we added a check that compares num_tokens against the > number of tokens supplied in the yaml via initial_token. From my reading of > CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained > contradictory values for num_tokens and initial_tokens which should help > prevent misconfiguration via human error. The current behaviour appears to > differ slightly in that it performs this comparison regardless of whether > num_tokens is included in the yaml or not. Below are proposed patches to only > perform the check if both options are present in the yaml. > ||Branch|| > |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]| > |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13325) Bring back the accepted encryption protocols list as configurable option
[
https://issues.apache.org/jira/browse/CASSANDRA-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233896#comment-17233896
]
Jon Meredith commented on CASSANDRA-13325:
--
Thanks for the review, pushed up a commit picking up the SSLFactory
simplification and added the missing {{@Test}}s.
> Bring back the accepted encryption protocols list as configurable option
>
>
> Key: CASSANDRA-13325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13325
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Nachiket Patil
>Assignee: Jon Meredith
>Priority: Low
> Fix For: 4.0-beta
>
> Attachments: trunk.diff
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> With CASSANDRA-10508, the hard coded list of accepted encryption protocols
> was eliminated. For some use cases, it is necessary to restrict the
> encryption protocols used for communication between client and server.
> Default JVM way of negotiations allows the best encryption protocol that
> client can use.
> e.g. I have set Cassandra to use encryption. Ideally client and server
> negotiate to use best protocol (TLSv1.2). But a malicious client might force
> TLSv1.0 which is susceptible to POODLE attacks.
> At the moment only way to restrict the encryption protocol is using the
> {{jdk.tls.client.protocols}} systems property. If I dont have enough access
> to modify this property, I dont have any way of restricting the encryption
> protocols.
> I am proposing bring back the accepted_protocols property but make it
> configurable. If not specified, let the JVM take care of the TLS negotiations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: sstablescrub unit test hardening and docs
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new a04ccf3 sstablescrub unit test hardening and docs a04ccf3 is described below commit a04ccf3297839febed68c314704db4d920b64413 Author: Brandon Williams AuthorDate: Tue Nov 17 14:01:28 2020 -0600 sstablescrub unit test hardening and docs Patch by Berenguer Blasi, reviewed by brandonwilliams for CASSANDRA-16013 --- doc/source/tools/sstable/sstablescrub.rst | 7 + .../apache/cassandra/tools/StandaloneScrubber.java | 40 +++-- src/java/org/apache/cassandra/tools/Util.java | 2 +- .../cql3/validation/operations/TTLTest.java| 101 test/unit/org/apache/cassandra/db/ScrubTest.java | 176 - .../cassandra/tools/StandaloneScrubberTest.java| 6 + 6 files changed, 284 insertions(+), 48 deletions(-) diff --git a/doc/source/tools/sstable/sstablescrub.rst b/doc/source/tools/sstable/sstablescrub.rst index 0bbda9f..a8529e3 100644 --- a/doc/source/tools/sstable/sstablescrub.rst +++ b/doc/source/tools/sstable/sstablescrub.rst @@ -29,6 +29,13 @@ sstablescrub === --debug display stack traces +-e,--header-fixOption whether and how to perform a check of the sstable serialization-headers and fix , fixable issues. +Possible argument values: + - validate-only: validate the serialization-headers, but do not fix those. Do not continue with scrub - i.e. only validate the header (dry-run of fix-only). + - validate: (default) validate the serialization-headers, but do not fix those and only continue with scrub if no error were detected. + - fix-only: validate and fix the serialization-headers, don't continue with scrub. + - fix: validate and fix the serialization-headers, do not fix and do not continue with scrub if the serialization-header check encountered errors. + - off: don't perform the serialization-header checks. -h,--help display this help message -m,--manifest-check only check and repair the leveled manifest, without actually scrubbing the sstables -n,--no-validatedo not validate columns using column validator diff --git a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java b/src/java/org/apache/cassandra/tools/StandaloneScrubber.java index e2fedad..bd71c64 100644 --- a/src/java/org/apache/cassandra/tools/StandaloneScrubber.java +++ b/src/java/org/apache/cassandra/tools/StandaloneScrubber.java @@ -20,29 +20,41 @@ package org.apache.cassandra.tools; import java.io.File; import java.nio.file.Paths; -import java.util.*; +import java.util.ArrayList; +import java.util.Collection; +import java.util.List; +import java.util.Map; +import java.util.Set; import java.util.concurrent.TimeUnit; -import com.google.common.base.Predicate; -import com.google.common.base.Predicates; -import com.google.common.collect.Iterables; -import com.google.common.collect.Lists; -import org.apache.commons.cli.*; +import org.apache.commons.cli.CommandLine; +import org.apache.commons.cli.CommandLineParser; +import org.apache.commons.cli.GnuParser; +import org.apache.commons.cli.HelpFormatter; +import org.apache.commons.cli.ParseException; -import org.apache.cassandra.schema.Schema; +import org.apache.cassandra.config.DatabaseDescriptor; import org.apache.cassandra.db.ColumnFamilyStore; import org.apache.cassandra.db.Directories; import org.apache.cassandra.db.Keyspace; -import org.apache.cassandra.db.compaction.*; +import org.apache.cassandra.db.compaction.AbstractStrategyHolder; +import org.apache.cassandra.db.compaction.CompactionManager; +import org.apache.cassandra.db.compaction.CompactionStrategyManager; +import org.apache.cassandra.db.compaction.LeveledCompactionStrategy; +import org.apache.cassandra.db.compaction.LeveledManifest; +import org.apache.cassandra.db.compaction.OperationType; +import org.apache.cassandra.db.compaction.Scrubber; import org.apache.cassandra.db.lifecycle.LifecycleTransaction; +import org.apache.cassandra.io.sstable.Component; +import org.apache.cassandra.io.sstable.Descriptor; +import org.apache.cassandra.io.sstable.SSTableHeaderFix; import org.apache.cassandra.io.sstable.format.SSTableReader; -import org.apache.cassandra.io.sstable.*; +import org.apache.cassandra.schema.Schema; +import org.apache.cassandra.tools.BulkLoader.CmdLineOptions; import org.apache.cassandr
[jira] [Updated] (CASSANDRA-16013) sstablescrub unit test hardening and docs improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-16013:
-
Status: Ready to Commit (was: Review In Progress)
> sstablescrub unit test hardening and docs improvements
> --
>
> Key: CASSANDRA-16013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16013
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}. Also docs need updating to reflect
> the latest options available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16271) Writes timeout instead of failing on cluster with CL-1 replicas available during replace
[
https://issues.apache.org/jira/browse/CASSANDRA-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233907#comment-17233907
]
Krishna Vadali commented on CASSANDRA-16271:
[~samt] is. there any update on this?
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace
>
>
> Key: CASSANDRA-16271
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16271
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Krishna Vadali
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Attachments: sleep_before_replace.diff
>
>
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace node operation.
> With Consistency Level ALL, we are observing Timeout exceptions during writes
> when (RF - 1) nodes are available in the cluster with one replace-node
> operation running. The coordinator is expecting RF + 1 responses, while there
> are only RF nodes (RF-1 nodes in UN and 1 node in UJ) are available in the
> cluster, hence timing out.
> The same problem happens on a keyspace with RF=1, CL=ONE and one replica
> being replaced. Also RF=3, CL=QUORUM, one replica down and another being
> replaced.
> I believe the expected behavior is that the write should fail with
> UnavailableException since there are not enough NORMAL replicas to fulfill
> the request.
> h4. *Steps to reproduce:*
> Run a 3 node test cluster (call the nodes node1 (127.0.0.1), node2
> (127.0.0.2), node3 (127.0.0.3)):
> {code:java}
> ccm create test -v 3.11.3 -n 3 -s
> {code}
> Create test keyspaces with RF = 3 and RF = 1 respectively:
> {code:java}
> create keyspace rf3 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create keyspace rf1 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 1};
> {code}
> Create a table test in both the keyspaces:
> {code:java}
> create table rf3.test ( pk int primary KEY, value int);
> create table rf1.test ( pk int primary KEY, value int);
> {code}
> Stop node node2:
> {code:java}
> ccm node2 stop
> {code}
> Create node node4:
> {code:java}
> ccm add node4 -i 127.0.0.4
> {code}
> Enable auto_bootstrap
> {code:java}
> ccm node4 updateconf 'auto_bootstrap: true'
> {code}
> Ensure node4 does not have itself in its seeds list.
> Run a replace node to replace node2 (address 127.0.0.2 corresponds to node
> node2)
> {code:java}
> ccm node4 start --jvm_arg="-Dcassandra.replace_address=127.0.0.2"
> {code}
> When the replace node is running, perform write/reads with CONSISTENCY ALL,
> we observed TimeoutException.
> {code:java}
> SET CONSISTENCY ALL:SET CONSISTENCY ALL:
> cqlsh> insert into rf3.test (pk, value) values (16, 7);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 3 responses." info=\{'received_responses': 3, 'required_responses': 4,
> 'consistency': 'ALL'}{code}
> {code:java}
> cqlsh> CONSISTENCY ONE;
> cqlsh> insert into rf1.test (pk, value) VALUES(5, 1);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 1 responses." info=\{'received_responses': 1, 'required_responses': 2,
> 'consistency': 'ONE'}
> {code}
> Cluster State:
> {code:java}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.1 70.45 KiB 1100.0%
> 4f652b22-045b-493b-8722-fb5f7e1723ce rack1
> UN 127.0.0.3 70.43 KiB 1100.0%
> a0dcd677-bdb3-4947-b9a7-14f3686a709f rack1
> UJ 127.0.0.4 137.47 KiB 1?
> e3d794f1-081e-4aba-94f2-31950c713846 rack1
> {code}
> Note:
> We introduced sleep during replace operation in order to simulate do our
> experiments. We attached code diff that does it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16013) sstablescrub unit test hardening and docs improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-16013:
-
Since Version: NA
Source Control Link:
https://github.com/apache/cassandra/commit/a04ccf3297839febed68c314704db4d920b64413
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks, committed w/minor rebase.
> sstablescrub unit test hardening and docs improvements
> --
>
> Key: CASSANDRA-16013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16013
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}. Also docs need updating to reflect
> the latest options available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16013) sstablescrub unit test hardening and docs improvements
[
https://issues.apache.org/jira/browse/CASSANDRA-16013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-16013:
-
Fix Version/s: (was: 4.0)
4.0-beta4
4.0-beta
> sstablescrub unit test hardening and docs improvements
> --
>
> Key: CASSANDRA-16013
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16013
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta, 4.0-beta4
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> During CASSANDRA-15883 / CASSANDRA-15991 it was detected unit test coverage
> for this tool is minimal. There is a unit test to enhance upon under
> {{test/unit/org/apache/cassandra/tools}}. Also docs need updating to reflect
> the latest options available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16271) Writes timeout instead of failing on cluster with CL-1 replicas available during replace
[
https://issues.apache.org/jira/browse/CASSANDRA-16271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233907#comment-17233907
]
Krishna Vadali edited comment on CASSANDRA-16271 at 11/17/20, 8:09 PM:
---
[~samt] is there any update on this?
was (Author: tejavadali):
[~samt] is. there any update on this?
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace
>
>
> Key: CASSANDRA-16271
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16271
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Krishna Vadali
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Attachments: sleep_before_replace.diff
>
>
> Writes timeout instead of failing on cluster with CL-1 replicas available
> during replace node operation.
> With Consistency Level ALL, we are observing Timeout exceptions during writes
> when (RF - 1) nodes are available in the cluster with one replace-node
> operation running. The coordinator is expecting RF + 1 responses, while there
> are only RF nodes (RF-1 nodes in UN and 1 node in UJ) are available in the
> cluster, hence timing out.
> The same problem happens on a keyspace with RF=1, CL=ONE and one replica
> being replaced. Also RF=3, CL=QUORUM, one replica down and another being
> replaced.
> I believe the expected behavior is that the write should fail with
> UnavailableException since there are not enough NORMAL replicas to fulfill
> the request.
> h4. *Steps to reproduce:*
> Run a 3 node test cluster (call the nodes node1 (127.0.0.1), node2
> (127.0.0.2), node3 (127.0.0.3)):
> {code:java}
> ccm create test -v 3.11.3 -n 3 -s
> {code}
> Create test keyspaces with RF = 3 and RF = 1 respectively:
> {code:java}
> create keyspace rf3 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 3};
> create keyspace rf1 with replication = \{'class': 'SimpleStrategy',
> 'replication_factor': 1};
> {code}
> Create a table test in both the keyspaces:
> {code:java}
> create table rf3.test ( pk int primary KEY, value int);
> create table rf1.test ( pk int primary KEY, value int);
> {code}
> Stop node node2:
> {code:java}
> ccm node2 stop
> {code}
> Create node node4:
> {code:java}
> ccm add node4 -i 127.0.0.4
> {code}
> Enable auto_bootstrap
> {code:java}
> ccm node4 updateconf 'auto_bootstrap: true'
> {code}
> Ensure node4 does not have itself in its seeds list.
> Run a replace node to replace node2 (address 127.0.0.2 corresponds to node
> node2)
> {code:java}
> ccm node4 start --jvm_arg="-Dcassandra.replace_address=127.0.0.2"
> {code}
> When the replace node is running, perform write/reads with CONSISTENCY ALL,
> we observed TimeoutException.
> {code:java}
> SET CONSISTENCY ALL:SET CONSISTENCY ALL:
> cqlsh> insert into rf3.test (pk, value) values (16, 7);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 3 responses." info=\{'received_responses': 3, 'required_responses': 4,
> 'consistency': 'ALL'}{code}
> {code:java}
> cqlsh> CONSISTENCY ONE;
> cqlsh> insert into rf1.test (pk, value) VALUES(5, 1);
> WriteTimeout: Error from server: code=1100 [Coordinator node timed out
> waiting for replica nodes' responses] message="Operation timed out - received
> only 1 responses." info=\{'received_responses': 1, 'required_responses': 2,
> 'consistency': 'ONE'}
> {code}
> Cluster State:
> {code:java}
> Datacenter: datacenter1
> ===
> Status=Up/Down
> |/ State=Normal/Leaving/Joining/Moving
> -- AddressLoad Tokens Owns (effective) Host ID
> Rack
> UN 127.0.0.1 70.45 KiB 1100.0%
> 4f652b22-045b-493b-8722-fb5f7e1723ce rack1
> UN 127.0.0.3 70.43 KiB 1100.0%
> a0dcd677-bdb3-4947-b9a7-14f3686a709f rack1
> UJ 127.0.0.4 137.47 KiB 1?
> e3d794f1-081e-4aba-94f2-31950c713846 rack1
> {code}
> Note:
> We introduced sleep during replace operation in order to simulate do our
> experiments. We attached code diff that does it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233934#comment-17233934 ] David Capwell commented on CASSANDRA-15588: --- thanks for pointing that out; 10% yes =). I plan to really start breaking down this work after thanksgiving so will try to drive generating different tickets to help define "done", where one of them is "python dtest upgrade tests green" (currently we have 600 failures in Circle CI). > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233742#comment-17233742
]
Caleb Rackliffe edited comment on CASSANDRA-16226 at 11/17/20, 9:07 PM:
I've posted (the 3.0 version of) an alternative solution here, which should
merge up cleanly, given {{TableMetadata#isCQLTable()}} is still present on
trunk after CASSANDRA-16217: https://github.com/apache/cassandra/pull/823
(CircleCI is having difficulty with the unit tests, but there is an ex-unit
test run
[here|https://app.circleci.com/pipelines/github/maedhroz/cassandra/155/workflows/cc54c092-cac1-4144-b9be-577ea4c8a78e]
and a default config unit test
[here|https://app.circleci.com/pipelines/github/maedhroz/cassandra/158/workflows/12cd707a-2fdd-4b9f-b512-b5f060a63dbf].)
My goals w/ this patch were the following:
1.) Maintain the empty row semantics of tables that have either always been
compact or always been non-compact (see
[previous|https://issues.apache.org/jira/browse/CASSANDRA-16226?focusedCommentId=17233168&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17233168]
comment), and preserve as much of them as possible after dropping compact
storage, given the historical limitations of the compact format.
2.) Either improve/reduce or leave unchanged the number of SSTables read in all
cases for tables that have never been compact.
3.) Fix the regression around the number of SSTables read in the original
description of this issue for compact tables, and avoid having the regression
re-introduced when and if compact storage is dropped.
4.) Avoid creating a new SSTable format or burdening operators with running
{{upgradesstables}} again to make all of this above work.
Although there are some issues w/ the unit tests on CI, the relevant local
tests are looking clean, including {{operations.DeleteTest}} and
{{operations.UpgradeTest}}, which verify item #1 above, and the other points
are by and large satisfied. For point #2, there was actually a case where pure
non-compact tables appeared to be reading too many SSTables w/ updates present
(although that [needs to be
reviewed|https://github.com/apache/cassandra/pull/823/files?file-filters%5B%5D=.java#r524809272]).
One drawback of this approach is that when a compact table with existing
SSTables has compact storage dropped, those SSTables still do not contain
primary key liveness info, and therefore may continue to return zero rows
rather than rows w/ primary keys and null regular columns. (Data written after
the drop will, of course, have liveness info, and so mixed behavior is
possible.) I've discussed a few ideas w/ [~jwest] and [~ifesdjeen] around this,
and I'm not convinced any of them would be valuable. (Keep in mind that
applications written against compact storage tables will already be handling
this case, and good documentation will help even further.)
The first is having enough information, either via a sequence number or a new
piece of metadata, to identify whether an SSTable was created while a table was
still compact. Having this data might be useful in some other way, but it still
does not insert primary key liveness info into pre-drop SSTables. DELETE and
UPDATE operations never have primary key liveness information attached to them,
even for pure non-compact tables, so an insert pre-drop in one SSTable,
followed by a delete in a post-drop SSTable, will still translate to the
absence of a row rather than a row with null regular cells.
The second broad idea is to somehow insert primary key liveness information
into pre-drop SSTables, but this is problematic for at least a couple reasons.
If we make it a prerequisite for running DROP COMPACT STORAGE, it means
physically rewriting SSTables. (To be fair, this may not be a huge additional
burden for those still on 2.x, who would have to do this anyway.) The other
oddity is that there doesn't appear to be any existing procedure for
synthesizing primary key liveness info. In the non-compact world, it is carried
only by INSERT operations, not UPDATE or DELETE operations. It's not clear to
me how we would determine where to add it during a {{runsstableupgrade}} run,
particularly how we would know the difference between INSERT and UPDATE
starting with just a compact SSTable on disk.
To summarize, I think tables that are purely compact or non-compact should be
handled pretty well w/ this patch (and will strictly git all the goals above,
where applicable). The interesting case is the mixed compact/non-compact
SSTable set caused by dropping compact storage. In this case, I think it makes
reasonable trade-offs and avoids all the major pain points for operators.
Whatever direction we go, the official docs around compact storage are going to
need updating.
was (Author: maedhroz):
I've posted (the 3
[jira] [Commented] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233989#comment-17233989 ] Ekaterina Dimitrova commented on CASSANDRA-15588: - I think most of the Circle failures are because of resource issues. BUT [~mck] made it possible to run all of them in Jenkins where they are still a lot. I am actually try to look at them again. There were two big groups of failures. While looking into one of them, it disappeared (probably someone fixed something) :D But I have to check what was the other one if I can easily fix it. > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233989#comment-17233989 ] Ekaterina Dimitrova edited comment on CASSANDRA-15588 at 11/17/20, 9:17 PM: I think most of the Circle failures are because of resource issues. BUT [~mck] made it possible to run all of them in Jenkins where they are still a lot. I will actually try to look at them again. There were two big groups of failures. While looking into one of them, it disappeared (probably someone fixed something) :D But I have to check what was the other one if I can easily fix it and reduce the scope. was (Author: e.dimitrova): I think most of the Circle failures are because of resource issues. BUT [~mck] made it possible to run all of them in Jenkins where they are still a lot. I will actually try to look at them again. There were two big groups of failures. While looking into one of them, it disappeared (probably someone fixed something) :D But I have to check what was the other one if I can easily fix it. > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233989#comment-17233989 ] Ekaterina Dimitrova edited comment on CASSANDRA-15588 at 11/17/20, 9:17 PM: I think most of the Circle failures are because of resource issues. BUT [~mck] made it possible to run all of them in Jenkins where they are still a lot. I will actually try to look at them again. There were two big groups of failures. While looking into one of them, it disappeared (probably someone fixed something) :D But I have to check what was the other one if I can easily fix it. was (Author: e.dimitrova): I think most of the Circle failures are because of resource issues. BUT [~mck] made it possible to run all of them in Jenkins where they are still a lot. I am actually try to look at them again. There were two big groups of failures. While looking into one of them, it disappeared (probably someone fixed something) :D But I have to check what was the other one if I can easily fix it. > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233989#comment-17233989 ] Ekaterina Dimitrova edited comment on CASSANDRA-15588 at 11/17/20, 9:18 PM: I think most of the Circle failures are because of resource issues. BUT [~mck] made it possible to run all of them in Jenkins where the failures are still a lot. I will actually try to look at them again. There were two big groups of failures. While looking into one of them, it disappeared (probably someone fixed something) :D But I have to check what was the other one if I can easily fix it and reduce the scope. was (Author: e.dimitrova): I think most of the Circle failures are because of resource issues. BUT [~mck] made it possible to run all of them in Jenkins where they are still a lot. I will actually try to look at them again. There were two big groups of failures. While looking into one of them, it disappeared (probably someone fixed something) :D But I have to check what was the other one if I can easily fix it and reduce the scope. > 4.0 quality testing: Cluster Upgrade > > > Key: CASSANDRA-15588 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15588 > Project: Cassandra > Issue Type: Task > Components: Test/dtest/java, Test/dtest/python >Reporter: Josh McKenzie >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > We've historically had numerous bugs concerning upgrading clusters from one > version to the other. Let's establish the supported upgrade path and ensure > that users can safely perform the upgrades in production. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (50d8245 -> d9e1af8)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 50d8245 Merge branch 'cassandra-3.0' into cassandra-3.11 new 8ef5a88 Improved check of num_tokens against initial_token in the cassandra.yaml new d9e1af8 Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + NEWS.txt | 8 +++ src/java/org/apache/cassandra/config/Config.java | 2 +- .../cassandra/config/DatabaseDescriptor.java | 27 ++-- .../config/DatabaseDescriptorRefTest.java | 2 +- .../cassandra/config/DatabaseDescriptorTest.java | 78 ++ 6 files changed, 112 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated: Improved check of num_tokens against initial_token in the cassandra.yaml
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 8ef5a88 Improved check of num_tokens against initial_token in the
cassandra.yaml
8ef5a88 is described below
commit 8ef5a886312e20f09cd4b0358c71018908341796
Author: Stefan Miklosovic
AuthorDate: Thu Oct 29 17:14:22 2020 +0100
Improved check of num_tokens against initial_token in the cassandra.yaml
patch by Stefan Miklosovic; reviewed by Mick Semb Wever for CASSANDRA-14477
---
CHANGES.txt| 1 +
NEWS.txt | 7 +-
src/java/org/apache/cassandra/config/Config.java | 2 +-
.../cassandra/config/DatabaseDescriptor.java | 47 +---
.../cassandra/config/DatabaseDescriptorTest.java | 134 -
5 files changed, 171 insertions(+), 20 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 41daa6a..7aecefa 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.24:
+ * Improved check of num_tokens against the length of initial_token
(CASSANDRA-14477)
* Fix a race condition on ColumnFamilyStore and TableMetrics (CASSANDRA-16228)
* Remove the SEPExecutor blocking behavior (CASSANDRA-16186)
* Wait for schema agreement when bootstrapping (CASSANDRA-15158)
diff --git a/NEWS.txt b/NEWS.txt
index 5a2ef51..42fbf63 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -42,13 +42,14 @@ restore snapshots created with the previous major version
using the
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
-3.0.21
+3.0.24
==
Upgrading
-
-- Nothing specific to this release, but please see previous upgrading
sections,
- especially if you are upgrading from 2.2.
+- In cassandra.yaml, num_tokens must be defined if initial_token is
defined.
+ If it is not defined, or not equal to the numbers of tokens defined in
initial_tokens,
+ the node will not start. See CASSANDRA-14477 for details.
3.0.20
==
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index 2218ee2..277a68a 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -79,7 +79,7 @@ public class Config
/* initial token in the ring */
public String initial_token;
-public Integer num_tokens = 1;
+public Integer num_tokens;
/** Triggers automatic allocation of tokens if set, using the replication
strategy of the referenced keyspace */
public String allocate_tokens_for_keyspace = null;
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 9369229..04293fb 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -289,6 +289,37 @@ public class DatabaseDescriptor
}
}
+@VisibleForTesting
+static void applyTokensConfig(Config config) throws ConfigurationException
+{
+if (config.num_tokens != null && config.num_tokens > MAX_NUM_TOKENS)
+throw new ConfigurationException(String.format("A maximum number
of %d tokens per node is supported", MAX_NUM_TOKENS), false);
+
+if (config.initial_token != null)
+{
+Collection tokens = tokensFromString(config.initial_token);
+if (config.num_tokens == null)
+{
+throw new ConfigurationException("initial_token was set but
num_tokens is not!", false);
+}
+
+if (tokens.size() != config.num_tokens)
+{
+throw new ConfigurationException(String.format("The number of
initial tokens (by initial_token) specified (%s) is different from num_tokens
value (%s)",
+ tokens.size(),
+
config.num_tokens),
+ false);
+}
+
+for (String token : tokens)
+partitioner.getTokenFactory().validate(token);
+}
+else if (config.num_tokens == null)
+{
+config.num_tokens = 1;
+}
+}
+
public static void applyConfig(Config config) throws ConfigurationException
{
conf = config;
@@ -655,21 +686,7 @@ public class DatabaseDescriptor
if (conf.concurrent_compactors <= 0)
throw new ConfigurationException("concurrent_compactors should be
strictly greater than 0, but was " + conf.concurrent_compactors, false);
-if (conf.num_tokens == null)
-conf.num_tokens = 1;
-
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 1122fcfeaa127de40a3ff4dbfd0e74a4e8783d52 Merge: a04ccf3 d9e1af8 Author: Mick Semb Wever AuthorDate: Tue Nov 17 22:46:04 2020 +0100 Merge branch 'cassandra-3.11' into trunk CHANGES.txt| 1 + NEWS.txt | 3 + src/java/org/apache/cassandra/config/Config.java | 2 +- .../cassandra/config/DatabaseDescriptor.java | 27 ++-- .../config/DatabaseDescriptorRefTest.java | 2 +- .../cassandra/config/DatabaseDescriptorTest.java | 78 ++ 6 files changed, 107 insertions(+), 6 deletions(-) diff --cc CHANGES.txt index 83b8a61,80b1532..69ccf55 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,47 -1,13 +1,48 @@@ -3.11.10 +4.0-beta4 + * Upgrade JNA to 5.6.0, dropping support for <=glibc-2.6 systems (CASSANDRA-16212) + * Add saved Host IDs to TokenMetadata at startup (CASSANDRA-16246) + * Ensure that CacheMetrics.requests is picked up by the metric reporter (CASSANDRA-16228) + * Add a ratelimiter to snapshot creation and deletion (CASSANDRA-13019) + * Produce consistent tombstone for reads to avoid digest mistmatch (CASSANDRA-15369) + * Fix SSTableloader issue when restoring a table named backups (CASSANDRA-16235) + * Invalid serialized size for responses caused by increasing message time by 1ms which caused extra bytes in size calculation (CASSANDRA-16103) + * Throw BufferOverflowException from DataOutputBuffer for better visibility (CASSANDRA-16214) + * TLS connections to the storage port on a node without server encryption configured causes java.io.IOException accessing missing keystore (CASSANDRA-16144) +Merged from 3.11: Merged from 3.0: + * Improved check of num_tokens against the length of initial_token (CASSANDRA-14477) * Fix a race condition on ColumnFamilyStore and TableMetrics (CASSANDRA-16228) * Remove the SEPExecutor blocking behavior (CASSANDRA-16186) - * Fix invalid cell value skipping when reading from disk (CASSANDRA-16223) + * Wait for schema agreement when bootstrapping (CASSANDRA-15158) * Prevent invoking enable/disable gossip when not in NORMAL (CASSANDRA-16146) -3.11.9 - * Synchronize Keyspace instance store/clear (CASSANDRA-16210) +4.0-beta3 + * Segregate Network and Chunk Cache BufferPools and Recirculate Partially Freed Chunks (CASSANDRA-15229) + * Fail truncation requests when they fail on a replica (CASSANDRA-16208) + * Move compact storage validation earlier in startup process (CASSANDRA-16063) + * Fix ByteBufferAccessor cast exceptions are thrown when trying to query a virtual table (CASSANDRA-16155) + * Consolidate node liveness check for forced repair (CASSANDRA-16113) + * Use unsigned short in ValueAccessor.sliceWithShortLength (CASSANDRA-16147) + * Abort repairs when getting a truncation request (CASSANDRA-15854) + * Remove bad assert when getting active compactions for an sstable (CASSANDRA-15457) + * Avoid failing compactions with very large partitions (CASSANDRA-15164) + * Prevent NPE in StreamMessage in type lookup (CASSANDRA-16131) + * Avoid invalid state transition exception during incremental repair (CASSANDRA-16067) + * Allow zero padding in timestamp serialization (CASSANDRA-16105) + * Add byte array backed cells (CASSANDRA-15393) + * Correctly handle pending ranges with adjacent range movements (CASSANDRA-14801) + * Avoid adding locahost when streaming trivial ranges (CASSANDRA-16099) + * Add nodetool getfullquerylog (CASSANDRA-15988) + * Fix yaml format and alignment in tpstats (CASSANDRA-11402) + * Avoid trying to keep track of RTs for endpoints we won't write to during read repair (CASSANDRA-16084) + * When compaction gets interrupted, the exception should include the compactionId (CASSANDRA-15954) + * Make Table/Keyspace Metric Names Consistent With Each Other (CASSANDRA-15909) + * Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure (CASSANDRA-15861) + * NPE thrown while updating speculative execution time if keyspace is removed during task execution (CASSANDRA-15949) + * Show the progress of data streaming and index build (CASSANDRA-15406) + * Add flag to disable chunk cache and disable by default (CASSANDRA-16036) + * Upgrade to snakeyaml >= 1.26 version for CVE-2017-18640 fix (CASSANDRA-16150) +Merged from 3.11: * Fix ColumnFilter to avoid querying cells of unselected complex columns (CASSANDRA-15977) * Fix memory leak in CompressedChunkReader (CASSANDRA-15880) * Don't attempt value skipping with mixed version cluster (CASSANDRA-15833) diff --cc NEWS.txt index de472c4,d16bcce..cf33df9 --- a/NEWS.txt +++ b/NEWS.txt @@@ -33,249 -42,29 +33,252 @@@ restore snapshots created with the prev 'sstableloader' tool. You can upgrade the file fo
[cassandra] branch trunk updated (a04ccf3 -> 1122fcf)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from a04ccf3 sstablescrub unit test hardening and docs new 8ef5a88 Improved check of num_tokens against initial_token in the cassandra.yaml new d9e1af8 Merge branch 'cassandra-3.0' into cassandra-3.11 new 1122fcf Merge branch 'cassandra-3.11' into trunk The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + NEWS.txt | 3 + src/java/org/apache/cassandra/config/Config.java | 2 +- .../cassandra/config/DatabaseDescriptor.java | 27 ++-- .../config/DatabaseDescriptorRefTest.java | 2 +- .../cassandra/config/DatabaseDescriptorTest.java | 78 ++ 6 files changed, 107 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit d9e1af8824f6914a6eff99a6345575b93188c51a
Merge: 50d8245 8ef5a88
Author: Mick Semb Wever
AuthorDate: Tue Nov 17 22:42:06 2020 +0100
Merge branch 'cassandra-3.0' into cassandra-3.11
CHANGES.txt| 1 +
NEWS.txt | 8 +++
src/java/org/apache/cassandra/config/Config.java | 2 +-
.../cassandra/config/DatabaseDescriptor.java | 27 ++--
.../config/DatabaseDescriptorRefTest.java | 2 +-
.../cassandra/config/DatabaseDescriptorTest.java | 78 ++
6 files changed, 112 insertions(+), 6 deletions(-)
diff --cc CHANGES.txt
index 1083096,7aecefa..80b1532
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,7 -1,8 +1,8 @@@
-3.0.24:
+3.11.10
+Merged from 3.0:
+ * Improved check of num_tokens against the length of initial_token
(CASSANDRA-14477)
* Fix a race condition on ColumnFamilyStore and TableMetrics
(CASSANDRA-16228)
* Remove the SEPExecutor blocking behavior (CASSANDRA-16186)
- * Wait for schema agreement when bootstrapping (CASSANDRA-15158)
* Fix invalid cell value skipping when reading from disk (CASSANDRA-16223)
* Prevent invoking enable/disable gossip when not in NORMAL (CASSANDRA-16146)
diff --cc NEWS.txt
index b3246b5,42fbf63..d16bcce
--- a/NEWS.txt
+++ b/NEWS.txt
@@@ -42,15 -42,16 +42,23 @@@ restore snapshots created with the prev
'sstableloader' tool. You can upgrade the file format of your snapshots
using the provided 'sstableupgrade' tool.
-3.0.24
-==
-
++3.11.10
++=
+ Upgrading
+ -
+ - In cassandra.yaml, num_tokens must be defined if initial_token is
defined.
+ If it is not defined, or not equal to the numbers of tokens defined in
initial_tokens,
+ the node will not start. See CASSANDRA-14477 for details.
+
-3.0.20
+3.11.9
+==
+Upgrading
+-
+ - Custom compaction strategies must handle getting sstables added/removed
notifications for
+ sstables already added/removed - see CASSANDRA-14103 for details. This
has been a requirement
+ for correct operation since 3.11.0 due to an issue in
CompactionStrategyManager.
+
+3.11.7
==
Upgrading
diff --cc src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index fe7291b,04293fb..cbf42b9
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@@ -314,34 -193,136 +314,34 @@@ public class DatabaseDescripto
}
}
-@VisibleForTesting
-public static void applyAddressConfig(Config config) throws
ConfigurationException
+private static void setConfig(Config config)
{
-listenAddress = null;
-rpcAddress = null;
-broadcastAddress = null;
-broadcastRpcAddress = null;
-
-/* Local IP, hostname or interface to bind services to */
-if (config.listen_address != null && config.listen_interface != null)
-{
-throw new ConfigurationException("Set listen_address OR
listen_interface, not both", false);
-}
-else if (config.listen_address != null)
-{
-try
-{
-listenAddress = InetAddress.getByName(config.listen_address);
-}
-catch (UnknownHostException e)
-{
-throw new ConfigurationException("Unknown listen_address '" +
config.listen_address + "'", false);
-}
-
-if (listenAddress.isAnyLocalAddress())
-throw new ConfigurationException("listen_address cannot be a
wildcard address (" + config.listen_address + ")!", false);
-}
-else if (config.listen_interface != null)
-{
-listenAddress =
getNetworkInterfaceAddress(config.listen_interface, "listen_interface",
config.listen_interface_prefer_ipv6);
-}
+conf = config;
+}
-/* Gossip Address to broadcast */
-if (config.broadcast_address != null)
-{
-try
-{
-broadcastAddress =
InetAddress.getByName(config.broadcast_address);
-}
-catch (UnknownHostException e)
-{
-throw new ConfigurationException("Unknown broadcast_address
'" + config.broadcast_address + "'", false);
-}
+private static void applyAll() throws ConfigurationException
+{
+applySimpleConfig();
-if (broadcastAddress.isAnyLocalAddress())
-throw new ConfigurationException("broadcast_address cannot be
a wildcard address (" + config.broadcast_address + ")!", false);
-}
+applyPartitioner();
-/* Local IP, hostname or interface to bin
[jira] [Updated] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[ https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-14477: --- Fix Version/s: (was: 4.0.x) (was: 3.11.x) (was: 3.0.x) 3.0.23 3.11.9 4.0-beta4 Since Version: 3.0.6 Source Control Link: https://github.com/apache/cassandra/commit/8ef5a886312e20f09cd4b0358c71018908341796 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [8ef5a886312e20f09cd4b0358c71018908341796|https://github.com/apache/cassandra/commit/8ef5a886312e20f09cd4b0358c71018908341796]. > The check of num_tokens against the length of inital_token in the yaml > triggers unexpectedly > > > Key: CASSANDRA-14477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14477 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Vincent White >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 4.0-beta4, 3.11.9, 3.0.23 > > Time Spent: 40m > Remaining Estimate: 0h > > In CASSANDRA-10120 we added a check that compares num_tokens against the > number of tokens supplied in the yaml via initial_token. From my reading of > CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained > contradictory values for num_tokens and initial_tokens which should help > prevent misconfiguration via human error. The current behaviour appears to > differ slightly in that it performs this comparison regardless of whether > num_tokens is included in the yaml or not. Below are proposed patches to only > perform the check if both options are present in the yaml. > ||Branch|| > |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]| > |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16281) Update Harry Usage Instructions
Ekaterina Dimitrova created CASSANDRA-16281: --- Summary: Update Harry Usage Instructions Key: CASSANDRA-16281 URL: https://issues.apache.org/jira/browse/CASSANDRA-16281 Project: Cassandra Issue Type: Bug Reporter: Ekaterina Dimitrova -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16281) Update Harry Usage Instructions
[ https://issues.apache.org/jira/browse/CASSANDRA-16281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16281: Bug Category: Parent values: Correctness(12982) Complexity: Low Hanging Fruit Component/s: Test/fuzz Discovered By: User Report Severity: Low Status: Open (was: Triage Needed) > Update Harry Usage Instructions > --- > > Key: CASSANDRA-16281 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16281 > Project: Cassandra > Issue Type: Bug > Components: Test/fuzz >Reporter: Ekaterina Dimitrova >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16281) Update Harry Usage Instructions
[ https://issues.apache.org/jira/browse/CASSANDRA-16281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova reassigned CASSANDRA-16281: --- Assignee: Ekaterina Dimitrova > Update Harry Usage Instructions > --- > > Key: CASSANDRA-16281 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16281 > Project: Cassandra > Issue Type: Bug > Components: Test/fuzz >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-11479) BatchlogManager unit tests failing on truncate race condition
[ https://issues.apache.org/jira/browse/CASSANDRA-11479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-11479: --- Reviewers: Joel Knighton (was: Joel Knighton, Michael Semb Wever) > BatchlogManager unit tests failing on truncate race condition > - > > Key: CASSANDRA-11479 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11479 > Project: Cassandra > Issue Type: Bug > Components: Local/Compaction >Reporter: Joel Knighton >Assignee: Yuki Morishita >Priority: Normal > Fix For: 2.2.x, 3.0.x, 3.11.x > > Attachments: > TEST-org.apache.cassandra.batchlog.BatchlogManagerTest.log > > > Example on CI > [here|http://cassci.datastax.com/job/trunk_testall/818/testReport/junit/org.apache.cassandra.batchlog/BatchlogManagerTest/testLegacyReplay_compression/]. > This seems to have only started happening relatively recently (within the > last month or two). > As far as I can tell, this is only showing up on BatchlogManagerTests purely > because it is an aggressive user of truncate. The assertion is hit in the > setUp method, so it can happen before any of the test methods. The assertion > occurs because a compaction is happening when truncate wants to discard > SSTables; trace level logs suggest that this compaction is submitted after > the pause on the CompactionStrategyManager. > This should be reproducible by running BatchlogManagerTest in a loop - it > takes up to half an hour in my experience. A trace-level log from such a run > is attached - grep for my added log message "SSTABLES COMPACTING WHEN > DISCARDING" to find when the assert is hit. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15588) 4.0 quality testing: Cluster Upgrade
[
https://issues.apache.org/jira/browse/CASSANDRA-15588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234054#comment-17234054
]
Michael Semb Wever commented on CASSANDRA-15588:
bq. BUT Michael Semb Wever made it possible to run all of them in Jenkins where
the failures are still a lot.
Many of those ~700 failures, currently, are the same thing {{'ProtocolError
returned from server while using explicitly set client protocol_version 4'}}.
> 4.0 quality testing: Cluster Upgrade
>
>
> Key: CASSANDRA-15588
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15588
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest/java, Test/dtest/python
>Reporter: Josh McKenzie
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We've historically had numerous bugs concerning upgrading clusters from one
> version to the other. Let's establish the supported upgrade path and ensure
> that users can safely perform the upgrades in production.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-16226:
Reviewers: Alex Petrov
> COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by
> timestamp due to missing primary key liveness info
> ---
>
> Key: CASSANDRA-16226
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16226
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Labels: perfomance, upgrade
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> This was discovered while tracking down a spike in the number of SSTables
> per read for a COMPACT STORAGE table after a 2.1 -> 3.0 upgrade. Before 3.0,
> there is no direct analog of 3.0's primary key liveness info. When we upgrade
> 2.1 COMPACT STORAGE SSTables to the mf format, we simply don't write row
> timestamps, even if the original mutations were INSERTs. On read, when we
> look at SSTables in order from newest to oldest max timestamp, we expect to
> have this primary key liveness information to determine whether we can skip
> older SSTables after finding completely populated rows.
> ex. I have three SSTables in a COMPACT STORAGE table with max timestamps
> 1000, 2000, and 3000. There are many rows in a particular partition, making
> filtering on the min and max clustering effectively a no-op. All data is
> inserted, and there are no partial updates. A fully specified row with
> timestamp 2500 exists in the SSTable with a max timestamp of 3000. With a
> proper row timestamp in hand, we can easily ignore the SSTables w/ max
> timestamps of 1000 and 2000. Without it, we read 3 SSTables instead of 1,
> which likely means a significant performance regression.
> The following test illustrates this difference in behavior between 2.1 and
> 3.0:
> https://github.com/maedhroz/cassandra/commit/84ce9242bedd735ca79d4f06007d127de6a82800
> A solution here might be as simple as having
> {{SinglePartitionReadCommand#canRemoveRow()}} only inspect primary key
> liveness information for non-compact/CQL tables. Tombstones seem to be
> handled at a level above that anyway. (One potential problem with that is
> whether or not the distinction will continue to exist in 4.0, and dropping
> compact storage from a table doesn't magically make pk liveness information
> appear.)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16226) COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
[
https://issues.apache.org/jira/browse/CASSANDRA-16226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17233742#comment-17233742
]
Caleb Rackliffe edited comment on CASSANDRA-16226 at 11/17/20, 10:37 PM:
-
I've posted (the 3.0 version of) an alternative solution here, which should
merge up cleanly, given {{TableMetadata#isCQLTable()}} is still present on
trunk after CASSANDRA-16217: https://github.com/apache/cassandra/pull/823
(CircleCI is having difficulty with the unit tests, but there is an ex-unit
test run
[here|https://app.circleci.com/pipelines/github/maedhroz/cassandra/155/workflows/cc54c092-cac1-4144-b9be-577ea4c8a78e]
and a default config unit test run
[here|https://app.circleci.com/pipelines/github/maedhroz/cassandra/158/workflows/12cd707a-2fdd-4b9f-b512-b5f060a63dbf].)
My goals w/ this patch were the following:
1.) Maintain the empty row semantics of tables that have either always been
compact or always been non-compact (see
[previous|https://issues.apache.org/jira/browse/CASSANDRA-16226?focusedCommentId=17233168&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17233168]
comment), and preserve as much of them as possible after dropping compact
storage, given the historical limitations of the compact format.
2.) Either improve/reduce or leave unchanged the number of SSTables read in all
cases for tables that have never been compact.
3.) Fix the regression around the number of SSTables read in the original
description of this issue for compact tables, and avoid having the regression
re-introduced when and if compact storage is dropped.
4.) Avoid creating a new SSTable format or burdening operators with running
{{upgradesstables}} again to make all of this above work.
Although there are some issues w/ the unit tests on CI, the relevant local
tests are looking clean, including {{operations.DeleteTest}} and
{{operations.UpgradeTest}}, which verify item #1 above, and the other points
are by and large satisfied. For point #2, there was actually a case where pure
non-compact tables appeared to be reading too many SSTables w/ updates present
(although that [needs to be
reviewed|https://github.com/apache/cassandra/pull/823/files?file-filters%5B%5D=.java#r524809272]).
One drawback of this approach is that when a compact table with existing
SSTables has compact storage dropped, those SSTables still do not contain
primary key liveness info, and therefore may continue to return zero rows
rather than rows w/ primary keys and null regular columns. (Data written after
the drop will, of course, have liveness info, and so mixed behavior is
possible.) I've discussed a few ideas w/ [~jwest] and [~ifesdjeen] around this,
and I'm not convinced any of them would be valuable. (Keep in mind that
applications written against compact storage tables will already be handling
this case, and good documentation will help even further.)
The first is having enough information, either via a sequence number or a new
piece of metadata, to identify whether an SSTable was created while a table was
still compact. Having this data might be useful in some other way, but it still
does not insert primary key liveness info into pre-drop SSTables. DELETE and
UPDATE operations never have primary key liveness information attached to them,
even for pure non-compact tables, so an insert pre-drop in one SSTable,
followed by a delete in a post-drop SSTable, will still translate to the
absence of a row rather than a row with null regular cells.
The second broad idea is to somehow insert primary key liveness information
into pre-drop SSTables, but this is problematic for at least a couple reasons.
If we make it a prerequisite for running DROP COMPACT STORAGE, it means
physically rewriting SSTables. (To be fair, this may not be a huge additional
burden for those still on 2.x, who would have to do this anyway.) The other
oddity is that there doesn't appear to be any existing procedure for
synthesizing primary key liveness info. In the non-compact world, it is carried
only by INSERT operations, not UPDATE or DELETE operations. It's not clear to
me how we would determine where to add it during a {{runsstableupgrade}} run,
particularly how we would know the difference between INSERT and UPDATE
starting with just a compact SSTable on disk.
To summarize, I think tables that are purely compact or non-compact should be
handled pretty well w/ this patch (and will strictly git all the goals above,
where applicable). The interesting case is the mixed compact/non-compact
SSTable set caused by dropping compact storage. In this case, I think it makes
reasonable trade-offs and avoids all the major pain points for operators.
Whatever direction we go, the official docs around compact storage are going to
need updating.
was (Author: maedhroz):
I've posted
[jira] [Commented] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[
https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234080#comment-17234080
]
Michael Semb Wever commented on CASSANDRA-14477:
This broke all the {{dtest-novnode}} dtests :(
I suspect…
- the check for num_tokens being defined should be skipped if initial_tokens
defines only one token (as it unlikely to be a typo, folk would rarely be
configuring two tokens),
- {{dtest-novnode}} are run with {{--num-tokens 1}}, defined
[here|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-dtest-pytest.sh#L73],
- the
[Cassandra-devbranch|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/]
jenkins pipeline should be including {{dtest-novnode}}
> The check of num_tokens against the length of inital_token in the yaml
> triggers unexpectedly
>
>
> Key: CASSANDRA-14477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14477
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Vincent White
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 3.0.23, 3.11.9, 4.0-beta4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In CASSANDRA-10120 we added a check that compares num_tokens against the
> number of tokens supplied in the yaml via initial_token. From my reading of
> CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained
> contradictory values for num_tokens and initial_tokens which should help
> prevent misconfiguration via human error. The current behaviour appears to
> differ slightly in that it performs this comparison regardless of whether
> num_tokens is included in the yaml or not. Below are proposed patches to only
> perform the check if both options are present in the yaml.
> ||Branch||
> |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]|
> |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16222) Spark-Cassandra Bulk Reader
[ https://issues.apache.org/jira/browse/CASSANDRA-16222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234082#comment-17234082 ] Erick Ramirez commented on CASSANDRA-16222: --- [~jberragan] really nice work! Can we interest you in drafting a blog post for the site that we can feature in a future newsletter? (y) > Spark-Cassandra Bulk Reader > --- > > Key: CASSANDRA-16222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16222 > Project: Cassandra > Issue Type: New Feature > Components: Tool/external >Reporter: James Berragan >Priority: Normal > Fix For: NA > > Attachments: sparkbulkreader.patch > > > *Description:* > This ticket introduces the Spark-Cassandra Bulk Reader: a Spark library able > to read and compact Cassandra raw sstables into SparkSQL along the principles > of streaming compaction. The full code is attached as a patch file and will > be submitted to a GitHub repo. > *Motivation* > For analytics or "select *" use cases at scale, the performance is > prohibitively expensive to read via the normal CQL read path - either using > the Java driver or the Open Source Spark connector. By reading the raw > sstables, the bulk reader is able to read with near-zero impact to a > production cluster at speeds many orders of magnitudes faster than > alternatives. We have seen very good performance results, exporting a 32TB > table (~46bn CQL rows) to HDFS in Parquet format in around 1h10m; a 20x > reduction compared to previous solutions. By reading from multiple replicas > and ‘compacting’ duplicate data together, it can achieve consistency at a > user defined level (i.e. ONE, TWO, LOCAL_QUORUM etc). > *Overview* > This library provides the core code for reading a set of SSTables into > SparkSQL through a DataLayer abstraction. The role of the DataLayer is to: > * return a SchemaStruct, mapping the Cassandra CQL table schema to the > SparkSQL schema. > * a list of sstables available for reading. > * a method to open an InputStream for any file component of an sstable (e.g. > data, compression, summary etc). > The PartitionedDataLayer abstraction builds on the DataLayer interface for > partitioning Spark workers across a Cassandra token ring - allowing the Spark > job to scale linearly - and reading from sufficient Cassandra replicas to > achieve a user-specified consistency level. > A simple example LocalDataLayer implementation is included for reading from a > local file system. Users of the library can build their own implementations > to read from wherever they wish e.g. reading from a backup in a cloud storage > system, or reading from the snapshot directory of a live Cassandra cluster. > > At the core, the bulk reader uses the Apache Cassandra CompactionIterator to > perform the streaming compaction. As it iterates through the > CompactionIterator it deserializes the ByteBuffers, converts into the > appropriate SparkSQL data type and pivots each cell into a SparkSQL row. > > Supporting this library is a robust property-based test framework for > writing Cassandra sstables with arbitrary schemas using the CQLSSTableWriter, > and reading back through SparkSQL to verify the library achieves both > consistency and correctness. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[
https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234080#comment-17234080
]
Michael Semb Wever edited comment on CASSANDRA-14477 at 11/17/20, 11:16 PM:
This broke all the {{dtest-novnode}} dtests :(
I suspect…
- the check for num_tokens being defined should be skipped if initial_tokens
defines only one token (as it unlikely to be a typo, folk would rarely be
configuring two tokens),
- {{dtest-novnode}} are run with {{--num-tokens 1}}, defined
[here|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-dtest-pytest.sh#L73]
and in the circleci configs,
- the
[Cassandra-devbranch|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/]
jenkins pipeline should be including {{dtest-novnode}}
was (Author: michaelsembwever):
This broke all the {{dtest-novnode}} dtests :(
I suspect…
- the check for num_tokens being defined should be skipped if initial_tokens
defines only one token (as it unlikely to be a typo, folk would rarely be
configuring two tokens),
- {{dtest-novnode}} are run with {{--num-tokens 1}}, defined
[here|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-dtest-pytest.sh#L73],
- the
[Cassandra-devbranch|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/]
jenkins pipeline should be including {{dtest-novnode}}
> The check of num_tokens against the length of inital_token in the yaml
> triggers unexpectedly
>
>
> Key: CASSANDRA-14477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14477
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Vincent White
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 3.0.23, 3.11.9, 4.0-beta4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In CASSANDRA-10120 we added a check that compares num_tokens against the
> number of tokens supplied in the yaml via initial_token. From my reading of
> CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained
> contradictory values for num_tokens and initial_tokens which should help
> prevent misconfiguration via human error. The current behaviour appears to
> differ slightly in that it performs this comparison regardless of whether
> num_tokens is included in the yaml or not. Below are proposed patches to only
> perform the check if both options are present in the yaml.
> ||Branch||
> |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]|
> |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[ https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-14477: --- Resolution: (was: Fixed) Status: Open (was: Resolved) > The check of num_tokens against the length of inital_token in the yaml > triggers unexpectedly > > > Key: CASSANDRA-14477 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14477 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Vincent White >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 3.0.23, 3.11.9, 4.0-beta4 > > Time Spent: 40m > Remaining Estimate: 0h > > In CASSANDRA-10120 we added a check that compares num_tokens against the > number of tokens supplied in the yaml via initial_token. From my reading of > CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained > contradictory values for num_tokens and initial_tokens which should help > prevent misconfiguration via human error. The current behaviour appears to > differ slightly in that it performs this comparison regardless of whether > num_tokens is included in the yaml or not. Below are proposed patches to only > perform the check if both options are present in the yaml. > ||Branch|| > |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]| > |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]| -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[
https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234080#comment-17234080
]
Michael Semb Wever edited comment on CASSANDRA-14477 at 11/17/20, 11:24 PM:
This broke all the {{dtest-novnode}} dtests :(
I suspect…
- the check for num_tokens being defined should be skipped if initial_tokens
defines only one token (as it unlikely to be a typo, folk would rarely be
configuring two tokens),
- dtests define {{num_tokens=1}} for non-nvodes,
[here|https://github.com/apache/cassandra-dtest/blob/d61eff9b89a2f4bbe1791e1cd71a5e3456d6cc94/dtest_setup.py#L426],
- the
[Cassandra-devbranch|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/]
jenkins pipeline should be including {{dtest-novnode}}
was (Author: michaelsembwever):
This broke all the {{dtest-novnode}} dtests :(
I suspect…
- the check for num_tokens being defined should be skipped if initial_tokens
defines only one token (as it unlikely to be a typo, folk would rarely be
configuring two tokens),
- dtests define {{num_tokens=1}} for non-nvode,
[here|https://github.com/apache/cassandra-dtest/blob/d61eff9b89a2f4bbe1791e1cd71a5e3456d6cc94/dtest_setup.py#L426],
- the
[Cassandra-devbranch|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/]
jenkins pipeline should be including {{dtest-novnode}}
> The check of num_tokens against the length of inital_token in the yaml
> triggers unexpectedly
>
>
> Key: CASSANDRA-14477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14477
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Vincent White
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 3.0.23, 3.11.9, 4.0-beta4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In CASSANDRA-10120 we added a check that compares num_tokens against the
> number of tokens supplied in the yaml via initial_token. From my reading of
> CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained
> contradictory values for num_tokens and initial_tokens which should help
> prevent misconfiguration via human error. The current behaviour appears to
> differ slightly in that it performs this comparison regardless of whether
> num_tokens is included in the yaml or not. Below are proposed patches to only
> perform the check if both options are present in the yaml.
> ||Branch||
> |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]|
> |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14477) The check of num_tokens against the length of inital_token in the yaml triggers unexpectedly
[
https://issues.apache.org/jira/browse/CASSANDRA-14477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234080#comment-17234080
]
Michael Semb Wever edited comment on CASSANDRA-14477 at 11/17/20, 11:24 PM:
This broke all the {{dtest-novnode}} dtests :(
I suspect…
- the check for num_tokens being defined should be skipped if initial_tokens
defines only one token (as it unlikely to be a typo, folk would rarely be
configuring two tokens),
- dtests define {{num_tokens=1}} for non-nvode,
[here|https://github.com/apache/cassandra-dtest/blob/d61eff9b89a2f4bbe1791e1cd71a5e3456d6cc94/dtest_setup.py#L426],
- the
[Cassandra-devbranch|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/]
jenkins pipeline should be including {{dtest-novnode}}
was (Author: michaelsembwever):
This broke all the {{dtest-novnode}} dtests :(
I suspect…
- the check for num_tokens being defined should be skipped if initial_tokens
defines only one token (as it unlikely to be a typo, folk would rarely be
configuring two tokens),
- {{dtest-novnode}} are run with {{--num-tokens 1}}, defined
[here|https://github.com/apache/cassandra-builds/blob/trunk/build-scripts/cassandra-dtest-pytest.sh#L73]
and in the circleci configs,
- the
[Cassandra-devbranch|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/]
jenkins pipeline should be including {{dtest-novnode}}
> The check of num_tokens against the length of inital_token in the yaml
> triggers unexpectedly
>
>
> Key: CASSANDRA-14477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14477
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Vincent White
>Assignee: Stefan Miklosovic
>Priority: Low
> Fix For: 3.0.23, 3.11.9, 4.0-beta4
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> In CASSANDRA-10120 we added a check that compares num_tokens against the
> number of tokens supplied in the yaml via initial_token. From my reading of
> CASSANDRA-10120 it was to prevent cassandra starting if the yaml contained
> contradictory values for num_tokens and initial_tokens which should help
> prevent misconfiguration via human error. The current behaviour appears to
> differ slightly in that it performs this comparison regardless of whether
> num_tokens is included in the yaml or not. Below are proposed patches to only
> perform the check if both options are present in the yaml.
> ||Branch||
> |[3.0.x|https://github.com/apache/cassandra/compare/cassandra-3.0...vincewhite:num_tokens_30]|
> |[3.x|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:num_tokens_test_1_311]|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: Explicitly define num_tokens=1 when not using vnodes
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 845a82a Explicitly define num_tokens=1 when not using vnodes
845a82a is described below
commit 845a82a38d3c4bb6d14c0bdb3341bfdc36ebbb02
Author: Brandon Williams
AuthorDate: Tue Nov 17 17:33:13 2020 -0600
Explicitly define num_tokens=1 when not using vnodes
---
dtest_setup.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/dtest_setup.py b/dtest_setup.py
index 8cae7d1..23d7531 100644
--- a/dtest_setup.py
+++ b/dtest_setup.py
@@ -423,7 +423,7 @@ class DTestSetup(object):
self.cluster.set_configuration_options(
values={'initial_token': None, 'num_tokens':
self.dtest_config.num_tokens})
else:
-self.cluster.set_configuration_options(values={'num_tokens': None})
+self.cluster.set_configuration_options(values={'num_tokens': 1})
if self.dtest_config.use_off_heap_memtables:
self.cluster.set_configuration_options(values={'memtable_allocation_type':
'offheap_objects'})
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16222) Spark-Cassandra Bulk Reader
[ https://issues.apache.org/jira/browse/CASSANDRA-16222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234108#comment-17234108 ] James Berragan commented on CASSANDRA-16222: Hi [~flightc] - sure. What would you like to see in the blog post? > Spark-Cassandra Bulk Reader > --- > > Key: CASSANDRA-16222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16222 > Project: Cassandra > Issue Type: New Feature > Components: Tool/external >Reporter: James Berragan >Priority: Normal > Fix For: NA > > Attachments: sparkbulkreader.patch > > > *Description:* > This ticket introduces the Spark-Cassandra Bulk Reader: a Spark library able > to read and compact Cassandra raw sstables into SparkSQL along the principles > of streaming compaction. The full code is attached as a patch file and will > be submitted to a GitHub repo. > *Motivation* > For analytics or "select *" use cases at scale, the performance is > prohibitively expensive to read via the normal CQL read path - either using > the Java driver or the Open Source Spark connector. By reading the raw > sstables, the bulk reader is able to read with near-zero impact to a > production cluster at speeds many orders of magnitudes faster than > alternatives. We have seen very good performance results, exporting a 32TB > table (~46bn CQL rows) to HDFS in Parquet format in around 1h10m; a 20x > reduction compared to previous solutions. By reading from multiple replicas > and ‘compacting’ duplicate data together, it can achieve consistency at a > user defined level (i.e. ONE, TWO, LOCAL_QUORUM etc). > *Overview* > This library provides the core code for reading a set of SSTables into > SparkSQL through a DataLayer abstraction. The role of the DataLayer is to: > * return a SchemaStruct, mapping the Cassandra CQL table schema to the > SparkSQL schema. > * a list of sstables available for reading. > * a method to open an InputStream for any file component of an sstable (e.g. > data, compression, summary etc). > The PartitionedDataLayer abstraction builds on the DataLayer interface for > partitioning Spark workers across a Cassandra token ring - allowing the Spark > job to scale linearly - and reading from sufficient Cassandra replicas to > achieve a user-specified consistency level. > A simple example LocalDataLayer implementation is included for reading from a > local file system. Users of the library can build their own implementations > to read from wherever they wish e.g. reading from a backup in a cloud storage > system, or reading from the snapshot directory of a live Cassandra cluster. > > At the core, the bulk reader uses the Apache Cassandra CompactionIterator to > perform the streaming compaction. As it iterates through the > CompactionIterator it deserializes the ByteBuffers, converts into the > appropriate SparkSQL data type and pivots each cell into a SparkSQL row. > > Supporting this library is a robust property-based test framework for > writing Cassandra sstables with arbitrary schemas using the CQLSSTableWriter, > and reading back through SparkSQL to verify the library achieves both > consistency and correctness. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16222) Spark-Cassandra Bulk Reader
[ https://issues.apache.org/jira/browse/CASSANDRA-16222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234154#comment-17234154 ] Erick Ramirez commented on CASSANDRA-16222: --- I think users would be interested to hear about: * the motivation for creating the tool * use cases * examples of how to use it > Spark-Cassandra Bulk Reader > --- > > Key: CASSANDRA-16222 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16222 > Project: Cassandra > Issue Type: New Feature > Components: Tool/external >Reporter: James Berragan >Priority: Normal > Fix For: NA > > Attachments: sparkbulkreader.patch > > > *Description:* > This ticket introduces the Spark-Cassandra Bulk Reader: a Spark library able > to read and compact Cassandra raw sstables into SparkSQL along the principles > of streaming compaction. The full code is attached as a patch file and will > be submitted to a GitHub repo. > *Motivation* > For analytics or "select *" use cases at scale, the performance is > prohibitively expensive to read via the normal CQL read path - either using > the Java driver or the Open Source Spark connector. By reading the raw > sstables, the bulk reader is able to read with near-zero impact to a > production cluster at speeds many orders of magnitudes faster than > alternatives. We have seen very good performance results, exporting a 32TB > table (~46bn CQL rows) to HDFS in Parquet format in around 1h10m; a 20x > reduction compared to previous solutions. By reading from multiple replicas > and ‘compacting’ duplicate data together, it can achieve consistency at a > user defined level (i.e. ONE, TWO, LOCAL_QUORUM etc). > *Overview* > This library provides the core code for reading a set of SSTables into > SparkSQL through a DataLayer abstraction. The role of the DataLayer is to: > * return a SchemaStruct, mapping the Cassandra CQL table schema to the > SparkSQL schema. > * a list of sstables available for reading. > * a method to open an InputStream for any file component of an sstable (e.g. > data, compression, summary etc). > The PartitionedDataLayer abstraction builds on the DataLayer interface for > partitioning Spark workers across a Cassandra token ring - allowing the Spark > job to scale linearly - and reading from sufficient Cassandra replicas to > achieve a user-specified consistency level. > A simple example LocalDataLayer implementation is included for reading from a > local file system. Users of the library can build their own implementations > to read from wherever they wish e.g. reading from a backup in a cloud storage > system, or reading from the snapshot directory of a live Cassandra cluster. > > At the core, the bulk reader uses the Apache Cassandra CompactionIterator to > perform the streaming compaction. As it iterates through the > CompactionIterator it deserializes the ByteBuffers, converts into the > appropriate SparkSQL data type and pivots each cell into a SparkSQL row. > > Supporting this library is a robust property-based test framework for > writing Cassandra sstables with arbitrary schemas using the CQLSSTableWriter, > and reading back through SparkSQL to verify the library achieves both > consistency and correctness. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16173) Update "Getting Started" document for Windows users
[ https://issues.apache.org/jira/browse/CASSANDRA-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234215#comment-17234215 ] Berenguer Blasi commented on CASSANDRA-16173: - Ok so marking this as a dupe then > Update "Getting Started" document for Windows users > --- > > Key: CASSANDRA-16173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16173 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Yuki Morishita >Priority: Normal > Fix For: 4.0 > > > This is a documentation follow up to CASSANDRA-16171. > Since we are removing support for Windows, we should update ["Getting > Started" > guide|https://cassandra.apache.org/doc/latest/getting_started/index.html] to > include how-to's for Windows users for setting up Cassandra for dev > evnironment. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16173) Update "Getting Started" document for Windows users
[ https://issues.apache.org/jira/browse/CASSANDRA-16173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16173: Resolution: Duplicate Status: Resolved (was: Open) > Update "Getting Started" document for Windows users > --- > > Key: CASSANDRA-16173 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16173 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Yuki Morishita >Priority: Normal > Fix For: 4.0 > > > This is a documentation follow up to CASSANDRA-16171. > Since we are removing support for Windows, we should update ["Getting > Started" > guide|https://cassandra.apache.org/doc/latest/getting_started/index.html] to > include how-to's for Windows users for setting up Cassandra for dev > evnironment. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16143) Streaming fails when s SSTable writer finish() exceeds internode_tcp_user_timeout
[
https://issues.apache.org/jira/browse/CASSANDRA-16143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-16143:
--
Test and Documentation Plan: ci, unit test, jvm dtest, adhoc test
Status: Patch Available (was: Open)
PR: https://github.com/apache/cassandra/pull/824
CI:
https://app.circleci.com/pipelines/github/yifan-c/cassandra/164/workflows/a1fee16c-b4cd-4e67-b52f-f2e649ea33da
The patch
# adds a dedicated TCP user timeout (defaults to 0) for streaming.
# collects the processing time for the received incoming stream messages.
# logs warning when the processing time exceeds the tcp user timeout.
I chose to not add "a separate (unbounded) excecutor service" for
FileChannel.force and "periodically call force".
In the code base, the incoming stream messages are consumed in a dedicated
thread that polls from {{AsyncStreamingInputPlus}}. It does not block the netty
event loop. The {{AsyncStreamingInputPlus}} intentionally disables auto read
from netty channel to avoid OOM. If we starts an executor service to consume
the bytes and FileChannel.force clogs, we end up with keeping more deserialized
messages in memory with the risk of OOM. The timeout is actually related to the
auto read being disabled. The receiver is blocked at flushing and not able to
issue newer channel read.
I am also concerned about calling force periodically. According to the java doc
of FileChannel.force (quoted below), do so may increase the I/O ops that are
not necessary.
{quote} Invoking this method may cause an I/O operation to occur even
if the
channel was only opened for reading. Some operating systems, for
example, maintain a last-access time as part of a file's metadata, and
this time is updated whenever the file is read. Whether or not this is
actually done is system-dependent and is therefore unspecified.{quote}
> Streaming fails when s SSTable writer finish() exceeds
> internode_tcp_user_timeout
> -
>
> Key: CASSANDRA-16143
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16143
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Jon Meredith
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> tl;dr The internode TCP user timeout that provides more responsive detection
> of dead nodes for internode message will cause streaming to fail if system
> calls to fsync/fdatasync exceed the timeout (default 30s).
> To workaround, explicitly set internode_tcp_user_timeout to longer than
> fsync/fdatasync, or to zero to revert to the operating system default.
> Details:
> While bootstrapping a replacement 4.0beta3 node in an existing cluster,
> bootstrap streaming repeatedly failed with the streaming follower logging
> {code:java}
> ERROR 2020-09-10T14:29:34,711 [NettyStreaming-Outbound-1.1.1.1.7000:1]
> org.apache.cassandra.streaming.StreamSession:693 - [Stream
> #7cb67c00-f3ac-11ea-b940-f7836f164528] Streaming error occurred on session
> with peer 1.1.1.1:7000
> org.apache.cassandra.net.AsyncChannelOutputPlus$FlushException: The channel
> this output stream was writing to has been closed
>at
> org.apache.cassandra.net.AsyncChannelOutputPlus.propagateFailedFlush(AsyncChannelOutputPlus.java:200)
>at
> org.apache.cassandra.net.AsyncChannelOutputPlus.waitUntilFlushed(AsyncChannelOutputPlus.java:158)
>at
> org.apache.cassandra.net.AsyncChannelOutputPlus.waitForSpace(AsyncChannelOutputPlus.java:140)
>at
> org.apache.cassandra.net.AsyncChannelOutputPlus.beginFlush(AsyncChannelOutputPlus.java:97)
>at
> org.apache.cassandra.net.AsyncStreamingOutputPlus.lambda$writeToChannel$0(AsyncStreamingOutputPlus.java:142)
>at
> org.apache.cassandra.db.streaming.CassandraCompressedStreamWriter.lambda$write$0(CassandraCompressedStreamWriter.java:90)
>at
> org.apache.cassandra.net.AsyncStreamingOutputPlus.writeToChannel(AsyncStreamingOutputPlus.java:138)
>at
> org.apache.cassandra.db.streaming.CassandraCompressedStreamWriter.write(CassandraCompressedStreamWriter.java:89)
>at
> org.apache.cassandra.db.streaming.CassandraOutgoingFile.write(CassandraOutgoingFile.java:180)
>at
> org.apache.cassandra.streaming.messages.OutgoingStreamMessage.serialize(OutgoingStreamMessage.java:87)
>at
> org.apache.cassandra.streaming.messages.OutgoingStreamMessage$1.serialize(OutgoingStreamMessage.java:45)
>at
> org.apache.cassandra.streaming.messages.OutgoingStreamMessage$1.serialize(OutgoingStreamMessage.java:34)
>at
> org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:40)
>
[jira] [Commented] (CASSANDRA-13325) Bring back the accepted encryption protocols list as configurable option
[
https://issues.apache.org/jira/browse/CASSANDRA-13325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17234220#comment-17234220
]
Berenguer Blasi commented on CASSANDRA-13325:
-
+1 from me.
> Bring back the accepted encryption protocols list as configurable option
>
>
> Key: CASSANDRA-13325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13325
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Nachiket Patil
>Assignee: Jon Meredith
>Priority: Low
> Fix For: 4.0-beta
>
> Attachments: trunk.diff
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> With CASSANDRA-10508, the hard coded list of accepted encryption protocols
> was eliminated. For some use cases, it is necessary to restrict the
> encryption protocols used for communication between client and server.
> Default JVM way of negotiations allows the best encryption protocol that
> client can use.
> e.g. I have set Cassandra to use encryption. Ideally client and server
> negotiate to use best protocol (TLSv1.2). But a malicious client might force
> TLSv1.0 which is susceptible to POODLE attacks.
> At the moment only way to restrict the encryption protocol is using the
> {{jdk.tls.client.protocols}} systems property. If I dont have enough access
> to modify this property, I dont have any way of restricting the encryption
> protocols.
> I am proposing bring back the accepted_protocols property but make it
> configurable. If not specified, let the JVM take care of the TLS negotiations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
