[jira] [Updated] (CASSANDRA-14344) Support filtering using IN restrictions
[
https://issues.apache.org/jira/browse/CASSANDRA-14344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anonymous updated CASSANDRA-14344:
--
Status: Ready to Commit (was: Patch Available)
> Support filtering using IN restrictions
> ---
>
> Key: CASSANDRA-14344
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14344
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Dikang Gu
>Assignee: Venkata Harikrishna Nukala
>Priority: Major
> Attachments: 14344-trunk.txt
>
>
> Support IN filter query like this:
>
> CREATE TABLE ks1.t1 (
> key int,
> col1 int,
> col2 int,
> value int,
> PRIMARY KEY (key, col1, col2)
> ) WITH CLUSTERING ORDER BY (col1 ASC, col2 ASC)
>
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1) allow filtering;
>
> key | col1 | col2 | value
> -+--+--+---
> 1 | 1 | 1 | 1
> 1 | 2 | 1 | 3
>
> (2 rows)
> cqlsh:ks1> select * from t1 where key = 1 and col2 in (1, 2) allow filtering;
> *{color:#ff}InvalidRequest: Error from server: code=2200 [Invalid query]
> message="IN restrictions are not supported on indexed columns"{color}*
> cqlsh:ks1>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14298) cqlshlib tests broken on b.a.o
[ https://issues.apache.org/jira/browse/CASSANDRA-14298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488331#comment-16488331 ] Patrick Bannister commented on CASSANDRA-14298: --- I switched to CentOS instead of RHEL, since RHEL requires a license to get access to the packages for many of the C* dependencies. Things look okay for the dtests on CentOS for these modifications using LC_CTYPE=en_US.utf8. That includes the cqlshlib port to Pyfhon 3 too. There are some extensive failures for the cqlshlib unittests in CentOS. cqlshlib/test/run_cqlsh.py::ProcRunner::read_until() can't handle ANSI control characters for colors in the prompt. I was able to modify the regular expression to handle the escape sequences, and with that change, the tests pass. But I won't be happy with that solution until I figure out why these escape characters are showing up in CentOS but not in my usual Ubuntu environment. The inputrc and bashrc files are a bit different, but I haven't figured out yet which difference, if any, is making the CentOS cqlsh prompt include colors. > cqlshlib tests broken on b.a.o > -- > > Key: CASSANDRA-14298 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14298 > Project: Cassandra > Issue Type: Bug > Components: Build, Testing >Reporter: Stefan Podkowinski >Assignee: Patrick Bannister >Priority: Major > Labels: cqlsh, dtest > Attachments: CASSANDRA-14298-old.txt, CASSANDRA-14298.txt, > cqlsh_tests_notes.md > > > It appears that cqlsh-tests on builds.apache.org on all branches stopped > working since we removed nosetests from the system environment. See e.g. > [here|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-trunk-cqlsh-tests/458/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/console]. > Looks like we either have to make nosetests available again or migrate to > pytest as we did with dtests. Giving pytest a quick try resulted in many > errors locally, but I haven't inspected them in detail yet. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13778) Enable Direct I/O for non-system SStables operations
[ https://issues.apache.org/jira/browse/CASSANDRA-13778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488063#comment-16488063 ] Mulugeta Mammo commented on CASSANDRA-13778: We have added a Direct IO support based on JDK 10. You may find the details here: https://issues.apache.org/jira/browse/CASSANDRA-14466. > Enable Direct I/O for non-system SStables operations > > > Key: CASSANDRA-13778 > URL: https://issues.apache.org/jira/browse/CASSANDRA-13778 > Project: Cassandra > Issue Type: Improvement > Components: Local Write-Read Paths >Reporter: Carlos Abad >Priority: Major > Labels: performance > Fix For: 4.x > > > Following the line of other databases (like RocksDB) enable Cassandra to > bypass the Linux Page Cache by using Direct I/O. By enabling this > functionality in the 4 main data paths (read/write un/compressed) our > internal testing at Intel shows that using Direct I/O increased performance > (latency and throughput) considerably. > In this implementation not all disk accesses bypass the OS Page Cache, only > the ones targeting a non-system SStable data files do. Disk access to > SStables metadata files -index, crc, system tables, etc- still benefit from > the OS Page Cache. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14466) Enable Direct I/O
[ https://issues.apache.org/jira/browse/CASSANDRA-14466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated CASSANDRA-14466: --- Description: Hi, JDK 10 introduced a new API for Direct IO that enables applications to bypass the file system cache and potentially improve performance. Details of this feature can be found at [https://bugs.openjdk.java.net/browse/JDK-8164900]. This patch uses the JDK 10 API to enable Direct IO for the Cassandra read path. By default, we have disabled this feature; but it can be enabled using a new configuration parameter, enable_direct_io_for_read_path. We have conducted a Cassandra read-only stress test and measured a throughput gain of up to 60% on flash drives. The patch requires JDK 10 Cassandra Support - https://issues.apache.org/jira/browse/CASSANDRA-9608 Please review the patch and let us know your feedback. Thanks, [^direct_io.patch] was: Hi, JDK 10 introduced a new API for Direct IO that enables applications to bypass the file system cache and potentially improve performance. Details of this feature can be found [here|[https://bugs.openjdk.java.net/browse/JDK-8164900]] This patch uses the JDK 10 API to enable Direct IO for the Cassandra read path. By default, we have disabled this feature; but it can be enabled using a new configuration parameter, enable_direct_io_for_read_path. We have conducted a Cassandra read-only stress test and measured a throughput gain of up to 60% on flash drives. The patch requires JDK 10 Cassandra Support. Please review the patch and let us know your feedback. Thanks, [^direct_io.patch] > Enable Direct I/O > -- > > Key: CASSANDRA-14466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14466 > Project: Cassandra > Issue Type: New Feature > Components: Local Write-Read Paths >Reporter: Mulugeta Mammo >Priority: Major > Attachments: direct_io.patch > > > Hi, > JDK 10 introduced a new API for Direct IO that enables applications to bypass > the file system cache and potentially improve performance. Details of this > feature can be found at [https://bugs.openjdk.java.net/browse/JDK-8164900]. > This patch uses the JDK 10 API to enable Direct IO for the Cassandra read > path. By default, we have disabled this feature; but it can be enabled using > a new configuration parameter, enable_direct_io_for_read_path. We have > conducted a Cassandra read-only stress test and measured a throughput gain of > up to 60% on flash drives. > The patch requires JDK 10 Cassandra Support - > https://issues.apache.org/jira/browse/CASSANDRA-9608 > Please review the patch and let us know your feedback. > Thanks, > [^direct_io.patch] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14466) Enable Direct I/O
[ https://issues.apache.org/jira/browse/CASSANDRA-14466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated CASSANDRA-14466: --- Description: Hi, JDK 10 introduced a new API for Direct IO that enables applications to bypass the file system cache and potentially improve performance. Details of this feature can be found [here|[https://bugs.openjdk.java.net/browse/JDK-8164900]] This patch uses the JDK 10 API to enable Direct IO for the Cassandra read path. By default, we have disabled this feature; but it can be enabled using a new configuration parameter, enable_direct_io_for_read_path. We have conducted a Cassandra read-only stress test and measured a throughput gain of up to 60% on flash drives. The patch requires JDK 10 Cassandra Support. Please review the patch and let us know your feedback. Thanks, [^direct_io.patch] was: Hi, JDK 10 introduced a new API for Direct IO that enables applications to bypass the file system cache and potentially improve performance. Details of this feature can be found [here|[https://bugs.openjdk.java.net/browse/JDK-8164900]|https://bugs.openjdk.java.net/browse/JDK-8164900].] This patch uses the JDK 10 API to enable Direct IO for the Cassandra read path. By default, we have disabled this feature; but it can be enabled using a new configuration parameter, enable_direct_io_for_read_path. We have conducted a Cassandra read-only stress test and measured a throughput gain of up to 60% on flash drives. The patch requires JDK 10 Cassandra support. Please review the patch and let us know your feedback. Thanks, [^direct_io.patch] > Enable Direct I/O > -- > > Key: CASSANDRA-14466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14466 > Project: Cassandra > Issue Type: New Feature > Components: Local Write-Read Paths >Reporter: Mulugeta Mammo >Priority: Major > Attachments: direct_io.patch > > > Hi, > JDK 10 introduced a new API for Direct IO that enables applications to bypass > the file system cache and potentially improve performance. Details of this > feature can be found > [here|[https://bugs.openjdk.java.net/browse/JDK-8164900]] > This patch uses the JDK 10 API to enable Direct IO for the Cassandra read > path. By default, we have disabled this feature; but it can be enabled using > a new configuration parameter, enable_direct_io_for_read_path. We have > conducted a Cassandra read-only stress test and measured a throughput gain of > up to 60% on flash drives. > The patch requires JDK 10 Cassandra Support. > Please review the patch and let us know your feedback. > Thanks, > [^direct_io.patch] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14466) Enable Direct I/O
[ https://issues.apache.org/jira/browse/CASSANDRA-14466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mulugeta Mammo updated CASSANDRA-14466: --- Description: Hi, JDK 10 introduced a new API for Direct IO that enables applications to bypass the file system cache and potentially improve performance. Details of this feature can be found [here|[https://bugs.openjdk.java.net/browse/JDK-8164900]|https://bugs.openjdk.java.net/browse/JDK-8164900].] This patch uses the JDK 10 API to enable Direct IO for the Cassandra read path. By default, we have disabled this feature; but it can be enabled using a new configuration parameter, enable_direct_io_for_read_path. We have conducted a Cassandra read-only stress test and measured a throughput gain of up to 60% on flash drives. The patch requires JDK 10 Cassandra support. Please review the patch and let us know your feedback. Thanks, [^direct_io.patch] was: Hi, JDK 10 introduced a new API for Direct IO that enables applications to bypass the file system cache and potentially improve performance. Details of this feature can be found here. This patch uses the JDK 10 API to enable Direct IO for the Cassandra read path. By default, we have disabled this feature; but it can be enabled using a new configuration parameter, enable_direct_io_for_read_path. We have conducted a Cassandra read-only stress test and measured a throughput gain of up to 60% on flash drives. The patch requires JDK 10 Cassandra support. Please review the patch and let us know your feedback. Thanks, [^direct_io.patch] > Enable Direct I/O > -- > > Key: CASSANDRA-14466 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14466 > Project: Cassandra > Issue Type: New Feature > Components: Local Write-Read Paths >Reporter: Mulugeta Mammo >Priority: Major > Attachments: direct_io.patch > > > Hi, > JDK 10 introduced a new API for Direct IO that enables applications to bypass > the file system cache and potentially improve performance. Details of this > feature can be found > [here|[https://bugs.openjdk.java.net/browse/JDK-8164900]|https://bugs.openjdk.java.net/browse/JDK-8164900].] > > This patch uses the JDK 10 API to enable Direct IO for the Cassandra read > path. By default, we have disabled this feature; but it can be enabled using > a new configuration parameter, enable_direct_io_for_read_path. We have > conducted a Cassandra read-only stress test and measured a throughput gain of > up to 60% on flash drives. > The patch requires JDK 10 Cassandra support. > Please review the patch and let us know your feedback. > Thanks, > [^direct_io.patch] > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14466) Enable Direct I/O
Mulugeta Mammo created CASSANDRA-14466: -- Summary: Enable Direct I/O Key: CASSANDRA-14466 URL: https://issues.apache.org/jira/browse/CASSANDRA-14466 Project: Cassandra Issue Type: New Feature Components: Local Write-Read Paths Reporter: Mulugeta Mammo Attachments: direct_io.patch Hi, JDK 10 introduced a new API for Direct IO that enables applications to bypass the file system cache and potentially improve performance. Details of this feature can be found here. This patch uses the JDK 10 API to enable Direct IO for the Cassandra read path. By default, we have disabled this feature; but it can be enabled using a new configuration parameter, enable_direct_io_for_read_path. We have conducted a Cassandra read-only stress test and measured a throughput gain of up to 60% on flash drives. The patch requires JDK 10 Cassandra support. Please review the patch and let us know your feedback. Thanks, [^direct_io.patch] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9608) Support Java 9
[
https://issues.apache.org/jira/browse/CASSANDRA-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488021#comment-16488021
]
Mulugeta Mammo commented on CASSANDRA-9608:
---
Hi [~snazy],
We have resolved the merge issues
[here|https://github.com/apache/cassandra/compare/trunk...snazy:9608-trunk] and
we can now run the trunk version of Cassandra with both JDK 9 and 10. Please
review the attached patch and let us know what is missing, if any.
[^jdk_9_10.patch]
Thanks,
> Support Java 9
> --
>
> Key: CASSANDRA-9608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9608
> Project: Cassandra
> Issue Type: Task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Attachments: jdk_9_10.patch
>
>
> This ticket is intended to group all issues found to support Java 9 in the
> future.
> From what I've found out so far:
> * Maven dependency {{com.sun:tools:jar:0}} via cobertura cannot be resolved.
> It can be easily solved using this patch:
> {code}
> - artifactId="cobertura"/>
> + artifactId="cobertura">
> +
> +
> {code}
> * Another issue is that {{sun.misc.Unsafe}} no longer contains the methods
> {{monitorEnter}} + {{monitorExit}}. These methods are used by
> {{o.a.c.utils.concurrent.Locks}} which is only used by
> {{o.a.c.db.AtomicBTreeColumns}}.
> I don't mind to start working on this yet since Java 9 is in a too early
> development phase.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-9608) Support Java 9
[
https://issues.apache.org/jira/browse/CASSANDRA-9608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mulugeta Mammo updated CASSANDRA-9608:
--
Attachment: jdk_9_10.patch
> Support Java 9
> --
>
> Key: CASSANDRA-9608
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9608
> Project: Cassandra
> Issue Type: Task
>Reporter: Robert Stupp
>Assignee: Robert Stupp
>Priority: Minor
> Attachments: jdk_9_10.patch
>
>
> This ticket is intended to group all issues found to support Java 9 in the
> future.
> From what I've found out so far:
> * Maven dependency {{com.sun:tools:jar:0}} via cobertura cannot be resolved.
> It can be easily solved using this patch:
> {code}
> - artifactId="cobertura"/>
> + artifactId="cobertura">
> +
> +
> {code}
> * Another issue is that {{sun.misc.Unsafe}} no longer contains the methods
> {{monitorEnter}} + {{monitorExit}}. These methods are used by
> {{o.a.c.utils.concurrent.Locks}} which is only used by
> {{o.a.c.db.AtomicBTreeColumns}}.
> I don't mind to start working on this yet since Java 9 is in a too early
> development phase.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14443) Improvements for running dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487988#comment-16487988
]
Jordan West edited comment on CASSANDRA-14443 at 5/23/18 8:37 PM:
--
[~KurtG] these changes look like a good improvement, especially not running the
regular test suite twice. A few comments below:
* How about exiting instead of logging a warning when insufficient resources
are available for running resource intensive tests (conftest.py#L460), unless
{{--force-resource-intensive-tests}} is set? Behaviour would be unchanged
otherwise and users who want to force run only the resource intensive tests on
insufficient hardware can use {{-force-resource-intensive-tests
--resource-intensive-tests-only}}.
* run_dtests.py: some things I noticed that may be worth cleaning up while
we’re here
1. original_raw_cmd_args is only used in one place now. Consider removing it.
2. comment on line 120 is stale
was (Author: jrwest):
[~KurtG] these changes look like a good improvement, especially not running the
regular test suite twice. A few comments below:
* How about exiting instead of logging a warning when insufficient resources
are available for running resource intensive tests (conftest.py#L460), unless
{{--force-resource-intensive-tests-}} is set? Behaviour would be unchanged
otherwise and users who want to force run only the resource intensive tests on
insufficient hardware can use {{-force-resource-intensive-tests
--resource-intensive-tests-only}}.
* run_dtests.py: some things I noticed that may be worth cleaning up while
we’re here
1. original_raw_cmd_args is only used in one place now. Consider removing it.
2. comment on line 120 is stale
> Improvements for running dtests
> ---
>
> Key: CASSANDRA-14443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14443
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Labels: dtest
>
> We currently hardcode a requirement that you need at least 27gb of memory to
> run the resource intensive tests. This is rather annoying as there isn't
> really a strict hardware requirement and tests can run on smaller machines in
> a lot of cases (especially if you mess around with HEAP).
> We've already got the command line argument
> {{--force-resource-intensive-tests}}, we don't need additional restrictions
> in place to stop people who shouldn't be running the tests from running them.
> We also don't have a way to run _only_ the resource-intensive dtests or
> _only_ the upgrade tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14465) Consider logging prepared statements bound values in Audit Log
[ https://issues.apache.org/jira/browse/CASSANDRA-14465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay Chella updated CASSANDRA-14465: - Description: The Goal of this ticket is to determine the best way to implement audit logging of actual bound values from prepared statement execution. The current default implementation does not log bound values Here are the options I see 1. Log bound values of prepared statements 2. Let a custom implementation of IAuditLogger decide what to do *Context*: Option #1: Works for teams which expects bind values to be logged in audit log without any security or compliance concerns Option #2: Allows teams make the best choice for themselves Note that the efforts of securing C* clusters by certs, authentication, and audit logging can go in vain when log rotation and log aggregation systems are not equally secure enough since logging bind values allow someone to replay the database events and expose sensitive data. [~spo...@gmail.com] [~jasobrown] was: The Goal of this ticket is to determine the best way to implement audit logging of actual bound values from prepared statement execution. The current default implementation does not log bound values Here are the options I see 1. Log bound values of prepared statements 2. Let a custom implementation of IAuditLogger decide what to do *Context*: Option #1: Works for teams which expects bind values to be logged in audit log without any security or compliance concerns Option #2: Allows teams make the best choice for themselves Note that the efforts of securing C* clusters by certs, authentication, and audit logging can go in vain when log rotation and log aggregation systems are not equally secure enough since logging bind values allow someone to reply the database events and exposes sensitive data. [~spo...@gmail.com] [~jasobrown] > Consider logging prepared statements bound values in Audit Log > -- > > Key: CASSANDRA-14465 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14465 > Project: Cassandra > Issue Type: Improvement >Reporter: Vinay Chella >Priority: Minor > > The Goal of this ticket is to determine the best way to implement audit > logging of actual bound values from prepared statement execution. The current > default implementation does not log bound values > Here are the options I see > 1. Log bound values of prepared statements > 2. Let a custom implementation of IAuditLogger decide what to do > *Context*: > Option #1: Works for teams which expects bind values to be logged in audit > log without any security or compliance concerns > Option #2: Allows teams make the best choice for themselves > Note that the efforts of securing C* clusters by certs, authentication, and > audit logging can go in vain when log rotation and log aggregation systems > are not equally secure enough since logging bind values allow someone to > replay the database events and expose sensitive data. > [~spo...@gmail.com] [~jasobrown] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14443) Improvements for running dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487988#comment-16487988
]
Jordan West edited comment on CASSANDRA-14443 at 5/23/18 8:36 PM:
--
[~KurtG] these changes look like a good improvement, especially not running the
regular test suite twice. A few comments below:
* How about exiting instead of logging a warning when insufficient resources
are available for running resource intensive tests (conftest.py#L460), unless
{{--force-resource-intensive-tests-}} is set? Behaviour would be unchanged
otherwise and users who want to force run only the resource intensive tests on
insufficient hardware can use {{-force-resource-intensive-tests
--resource-intensive-tests-only}}.
* run_dtests.py: some things I noticed that may be worth cleaning up while
we’re here
1. original_raw_cmd_args is only used in one place now. Consider removing it.
2. comment on line 120 is stale
was (Author: jrwest):
[~KurtG] these changes look like a good improvement, especially not running the
test regular suite twice. A few comments below:
* How about exiting instead of logging a warning when insufficient resources
are available for running resource intensive tests (conftest.py#L460), unless
{{--force-resource-intensive-tests}} is set? Behaviour would be unchanged
otherwise and users who want to force run only the resource intensive tests on
insufficient hardware can use {{--force-resource-intensive-tests
--resource-intensive-tests-only}}.
* run_dtests.py: some things I noticed that may be worth cleaning up while
we’re here
1. original_raw_cmd_args is only used in one place now. Consider removing it.
2. comment on line 120 is stale
> Improvements for running dtests
> ---
>
> Key: CASSANDRA-14443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14443
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Labels: dtest
>
> We currently hardcode a requirement that you need at least 27gb of memory to
> run the resource intensive tests. This is rather annoying as there isn't
> really a strict hardware requirement and tests can run on smaller machines in
> a lot of cases (especially if you mess around with HEAP).
> We've already got the command line argument
> {{--force-resource-intensive-tests}}, we don't need additional restrictions
> in place to stop people who shouldn't be running the tests from running them.
> We also don't have a way to run _only_ the resource-intensive dtests or
> _only_ the upgrade tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14443) Improvements for running dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487988#comment-16487988
]
Jordan West commented on CASSANDRA-14443:
-
[~KurtG] these changes look like a good improvement, especially not running the
test regular suite twice. A few comments below:
* How about exiting instead of logging a warning when insufficient resources
are available for running resource intensive tests (conftest.py#L460), unless
{{--force-resource-intensive-tests}} is set? Behaviour would be unchanged
otherwise and users who want to force run only the resource intensive tests on
insufficient hardware can use {{--force-resource-intensive-tests
--resource-intensive-tests-only}}.
* run_dtests.py: some things I noticed that may be worth cleaning up while
we’re here
1. original_raw_cmd_args is only used in one place now. Consider removing it.
2. comment on line 120 is stale
> Improvements for running dtests
> ---
>
> Key: CASSANDRA-14443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14443
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Labels: dtest
>
> We currently hardcode a requirement that you need at least 27gb of memory to
> run the resource intensive tests. This is rather annoying as there isn't
> really a strict hardware requirement and tests can run on smaller machines in
> a lot of cases (especially if you mess around with HEAP).
> We've already got the command line argument
> {{--force-resource-intensive-tests}}, we don't need additional restrictions
> in place to stop people who shouldn't be running the tests from running them.
> We also don't have a way to run _only_ the resource-intensive dtests or
> _only_ the upgrade tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14443) Improvements for running dtests
[
https://issues.apache.org/jira/browse/CASSANDRA-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-14443:
Reviewer: Jordan West
> Improvements for running dtests
> ---
>
> Key: CASSANDRA-14443
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14443
> Project: Cassandra
> Issue Type: Improvement
> Components: Testing
>Reporter: Kurt Greaves
>Assignee: Kurt Greaves
>Priority: Major
> Labels: dtest
>
> We currently hardcode a requirement that you need at least 27gb of memory to
> run the resource intensive tests. This is rather annoying as there isn't
> really a strict hardware requirement and tests can run on smaller machines in
> a lot of cases (especially if you mess around with HEAP).
> We've already got the command line argument
> {{--force-resource-intensive-tests}}, we don't need additional restrictions
> in place to stop people who shouldn't be running the tests from running them.
> We also don't have a way to run _only_ the resource-intensive dtests or
> _only_ the upgrade tests
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14465) Consider logging prepared statements bound values in Audit Log
[ https://issues.apache.org/jira/browse/CASSANDRA-14465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinay Chella updated CASSANDRA-14465: - Description: The Goal of this ticket is to determine the best way to implement audit logging of actual bound values from prepared statement execution. The current default implementation does not log bound values Here are the options I see 1. Log bound values of prepared statements 2. Let a custom implementation of IAuditLogger decide what to do *Context*: Option #1: Works for teams which expects bind values to be logged in audit log without any security or compliance concerns Option #2: Allows teams make the best choice for themselves Note that the efforts of securing C* clusters by certs, authentication, and audit logging can go in vain when log rotation and log aggregation systems are not equally secure enough since logging bind values allow someone to reply the database events and exposes sensitive data. [~spo...@gmail.com] [~jasobrown] was: The Goal of this ticket is to determine the best way to implement audit logging of actual bound values from prepared statement execution. The current default implementation does not log bound values Here are the options I see 1. Log bound values of prepared statements 2. Let a custom implementation of IAuditLogger decide what to do Context: Option #1: Works for teams which expects bind values to be logged in audit log without any security or compliance concerns Option #2: Allows teams make the best choice for themselves Note that the efforts of securing C* clusters by certs, authentication, and audit logging can go in vain when log rotation and log aggregation systems are not equally secure enough since logging bind values allow someone to reply the database events and exposes sensitive data. [~spo...@gmail.com] [~jasobrown] > Consider logging prepared statements bound values in Audit Log > -- > > Key: CASSANDRA-14465 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14465 > Project: Cassandra > Issue Type: Improvement >Reporter: Vinay Chella >Priority: Minor > > The Goal of this ticket is to determine the best way to implement audit > logging of actual bound values from prepared statement execution. The current > default implementation does not log bound values > Here are the options I see > 1. Log bound values of prepared statements > 2. Let a custom implementation of IAuditLogger decide what to do > *Context*: > Option #1: Works for teams which expects bind values to be logged in audit > log without any security or compliance concerns > Option #2: Allows teams make the best choice for themselves > Note that the efforts of securing C* clusters by certs, authentication, and > audit logging can go in vain when log rotation and log aggregation systems > are not equally secure enough since logging bind values allow someone to > reply the database events and exposes sensitive data. > [~spo...@gmail.com] [~jasobrown] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14465) Consider logging prepared statements bound values in Audit Log
Vinay Chella created CASSANDRA-14465: Summary: Consider logging prepared statements bound values in Audit Log Key: CASSANDRA-14465 URL: https://issues.apache.org/jira/browse/CASSANDRA-14465 Project: Cassandra Issue Type: Improvement Reporter: Vinay Chella The Goal of this ticket is to determine the best way to implement audit logging of actual bound values from prepared statement execution. The current default implementation does not log bound values Here are the options I see 1. Log bound values of prepared statements 2. Let a custom implementation of IAuditLogger decide what to do Context: Option #1: Works for teams which expects bind values to be logged in audit log without any security or compliance concerns Option #2: Allows teams make the best choice for themselves Note that the efforts of securing C* clusters by certs, authentication, and audit logging can go in vain when log rotation and log aggregation systems are not equally secure enough since logging bind values allow someone to reply the database events and exposes sensitive data. [~spo...@gmail.com] [~jasobrown] -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14459) DynamicEndpointSnitch should never prefer latent nodes
[
https://issues.apache.org/jira/browse/CASSANDRA-14459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487678#comment-16487678
]
Sumanth Pasupuleti commented on CASSANDRA-14459:
lgtm. Couple of comments
# Wondering what makes more sense for retaining a latency for a host upon
reset - min latency, or the most recent latency in the reservoir.
# Minor: Based on this code, MAYBE latency is taken into account if there is
zero or one latency record for that host
{quote}if (sample == null || sample.size() <= 1)
{quote}
while the definition of MAYBE enum value indicates
{quote}// If no other measurements are available, use this
{quote}
You may want to update the definition maybe?
> DynamicEndpointSnitch should never prefer latent nodes
> --
>
> Key: CASSANDRA-14459
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14459
> Project: Cassandra
> Issue Type: Improvement
> Components: Coordination
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
>
> The DynamicEndpointSnitch has two unfortunate behaviors that allow it to
> provide latent hosts as replicas:
> # Loses all latency information when Cassandra restarts
> # Clears latency information entirely every ten minutes (by default),
> allowing global queries to be routed to _other datacenters_ (and local
> queries cross racks/azs)
> This means that the first few queries after restart/reset could be quite slow
> compared to average latencies. I propose we solve this by resetting to the
> minimum observed latency instead of completely clearing the samples and
> extending the {{isLatencyForSnitch}} idea to a three state variable instead
> of two, in particular {{YES}}, {{NO}}, {{MAYBE}}. This extension allows
> {{EchoMessages}} and {{PingMessages}} to send {{MAYBE}} indicating that the
> DS should use those measurements if it only has one or fewer samples for a
> host. This fixes both problems because on process restart we send out
> {{PingMessages}} / {{EchoMessages}} as part of startup, and we would reset to
> effectively the RTT of the hosts (also at that point normal gossip
> {{EchoMessages}} have an opportunity to add an additional latency
> measurement).
> This strategy also nicely deals with the "a host got slow but now it's fine"
> problem that the DS resets were (afaik) designed to stop because the
> {{EchoMessage}} ping latency will count only after the reset for that host.
> Ping latency is a more reasonable lower bound on host latency (as opposed to
> status quo of zero).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14297) Optional startup delay for peers should wait for count rather than percentage
[
https://issues.apache.org/jira/browse/CASSANDRA-14297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16487530#comment-16487530
]
Sumanth Pasupuleti commented on CASSANDRA-14297:
LGTM. +1
> Optional startup delay for peers should wait for count rather than percentage
> -
>
> Key: CASSANDRA-14297
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14297
> Project: Cassandra
> Issue Type: Improvement
> Components: Lifecycle
>Reporter: Joseph Lynch
>Assignee: Joseph Lynch
>Priority: Minor
> Labels: PatchAvailable
>
> As I commented in CASSANDRA-13993, the current wait for functionality is a
> great step in the right direction, but I don't think that the current setting
> (70% of nodes in the cluster) is the right configuration option. First I
> think this because 70% will not protect against errors as if you wait for 70%
> of the cluster you could still very easily have {{UnavailableException}} or
> {{ReadTimeoutException}} exceptions. This is because if you have even two
> nodes down in different racks in a Cassandra cluster these exceptions are
> possible (or with the default {{num_tokens}} setting of 256 it is basically
> guaranteed). Second I think this option is not easy for operators to set, the
> only setting I could think of that would "just work" is 100%.
> I proposed in that ticket instead of having `block_for_peers_percentage`
> defaulting to 70%, we instead have `block_for_peers` as a count of nodes that
> are allowed to be down before the starting node makes itself available as a
> coordinator. Of course, we would still have the timeout to limit startup time
> and deal with really extreme situations (whole datacenters down etc).
> I started working on a patch for this change [on
> github|https://github.com/jasobrown/cassandra/compare/13993...jolynch:13993],
> and am happy to finish it up with unit tests and such if someone can
> review/commit it (maybe [~aweisberg]?).
> I think the short version of my proposal is we replace:
> {noformat}
> block_for_peers_percentage:
> {noformat}
> with either
> {noformat}
> block_for_peers:
> {noformat}
> or, if we want to do even better imo and enable advanced operators to finely
> tune this behavior (while still having good defaults that work for almost
> everyone):
> {noformat}
> block_for_peers_local_dc:
> block_for_peers_each_dc:
> block_for_peers_all_dcs:
> {noformat}
> For example if an operator knows that they must be available at
> {{LOCAL_QUORUM}} they would set {{block_for_peers_local_dc=1}}, if they use
> {{EACH_QUOURM}} they would set {{block_for_peers_local_dc=1}}, if they use
> {{QUORUM}} (RF=3, dcs=2) they would set {{block_for_peers_all_dcs=2}}.
> Naturally everything would of course have a timeout to prevent startup taking
> too long.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14464) stop-server.bat -p ../pid.txt -f command not working on windows 2016
[ https://issues.apache.org/jira/browse/CASSANDRA-14464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anonymous updated CASSANDRA-14464: -- Reproduced In: 3.11.2 Since Version: 3.11.2 Status: Awaiting Feedback (was: Open) > stop-server.bat -p ../pid.txt -f command not working on windows 2016 > > > Key: CASSANDRA-14464 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14464 > Project: Cassandra > Issue Type: Bug >Reporter: Shyam Phirke >Priority: Critical > > Steps to reproduce: > 1. Copy and extract cassandra binaries on windows 2016 machine > 2. Start cassandra in non-legacy mode > 3. Check pid of cassandra in task manager and compare it with in pid.txt > 4. Now stop cassandra using command stop-server.bat -p ../pid.txt -f > Expected: > After executing \bin:\> stop-server.bat -p > ../pid.txt -f > cassandra process as in pid.txt should get killed. > > Actual: > After executing above stop command, the cassandra process as in pid.txt gets > killed but a new process gets created with new pid. Also the pid.txt not > updated with new pid. > This new process should not get created. > > Please comment on this issue if more details required. > I am using cassandra 3.11.2. > > This issue impacting me much since because of this new process getting > created my application uninstallation getting impacted. > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-14464) stop-server.bat -p ../pid.txt -f command not working on windows 2016
Shyam Phirke created CASSANDRA-14464: Summary: stop-server.bat -p ../pid.txt -f command not working on windows 2016 Key: CASSANDRA-14464 URL: https://issues.apache.org/jira/browse/CASSANDRA-14464 Project: Cassandra Issue Type: Bug Reporter: Shyam Phirke Steps to reproduce: 1. Copy and extract cassandra binaries on windows 2016 machine 2. Start cassandra in non-legacy mode 3. Check pid of cassandra in task manager and compare it with in pid.txt 4. Now stop cassandra using command stop-server.bat -p ../pid.txt -f Expected: After executing \bin:\> stop-server.bat -p ../pid.txt -f cassandra process as in pid.txt should get killed. Actual: After executing above stop command, the cassandra process as in pid.txt gets killed but a new process gets created with new pid. Also the pid.txt not updated with new pid. This new process should not get created. Please comment on this issue if more details required. I am using cassandra 3.11.2. This issue impacting me much since because of this new process getting created my application uninstallation getting impacted. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
