[jira] [Updated] (CASSANDRA-15539) Extract in-jvm API and tests out of Cassandra and into a separate repository
[ https://issues.apache.org/jira/browse/CASSANDRA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15539: -- Status: Changes Suggested (was: Review In Progress) > Extract in-jvm API and tests out of Cassandra and into a separate repository > > > Key: CASSANDRA-15539 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15539 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Extract in-jvm DTest _API_ and tests into a separate repository that is > shared between Cassandra branches. Tests themselves should be buildable using > just API, which is not the case now, since cluster creation relies on impl > package, since we do not have factories / constructors in API. > Main goals we’re trying to achieve: > 1. We should be able to fail a build on API incompatibility between versions > 2. Make it as easy as possible to detect break APIs between versions. > 3. Make development of _tests_ based on in-jvm framework simpler > 4. Reduce surface area of impact when making modifications to tests > Potentially, we’d also like to use a plugin to detect API incompatibilities > between in-jvm DTest API and in-branch implementations, and start running > tests using shared in-jvm test repository with each existing implementation > in the branch. This entails both running tests for all branches whenever > there’s a change in tests jar and running tests for a specific branch > whenever the branch has changed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15539) Extract in-jvm API and tests out of Cassandra and into a separate repository
[ https://issues.apache.org/jira/browse/CASSANDRA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062932#comment-17062932 ] David Capwell commented on CASSANDRA-15539: --- Right now I am only reviewing trunk as I assume all branches are the same, once trunk is good ill look at the rest. Reviewed again, but non of my comments were addressed, only looks like you rebased to pick up the latest changes for repair. > Extract in-jvm API and tests out of Cassandra and into a separate repository > > > Key: CASSANDRA-15539 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15539 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Time Spent: 2h 20m > Remaining Estimate: 0h > > Extract in-jvm DTest _API_ and tests into a separate repository that is > shared between Cassandra branches. Tests themselves should be buildable using > just API, which is not the case now, since cluster creation relies on impl > package, since we do not have factories / constructors in API. > Main goals we’re trying to achieve: > 1. We should be able to fail a build on API incompatibility between versions > 2. Make it as easy as possible to detect break APIs between versions. > 3. Make development of _tests_ based on in-jvm framework simpler > 4. Reduce surface area of impact when making modifications to tests > Potentially, we’d also like to use a plugin to detect API incompatibilities > between in-jvm DTest API and in-branch implementations, and start running > tests using shared in-jvm test repository with each existing implementation > in the branch. This entails both running tests for all branches whenever > there’s a change in tests jar and running tests for a specific branch > whenever the branch has changed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15539) Extract in-jvm API and tests out of Cassandra and into a separate repository
[ https://issues.apache.org/jira/browse/CASSANDRA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15539: -- Reviewers: David Capwell, Dinesh Joshi, David Capwell (was: David Capwell, Dinesh Joshi) David Capwell, Dinesh Joshi, David Capwell (was: David Capwell, Dinesh Joshi) Status: Review In Progress (was: Patch Available) > Extract in-jvm API and tests out of Cassandra and into a separate repository > > > Key: CASSANDRA-15539 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15539 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Extract in-jvm DTest _API_ and tests into a separate repository that is > shared between Cassandra branches. Tests themselves should be buildable using > just API, which is not the case now, since cluster creation relies on impl > package, since we do not have factories / constructors in API. > Main goals we’re trying to achieve: > 1. We should be able to fail a build on API incompatibility between versions > 2. Make it as easy as possible to detect break APIs between versions. > 3. Make development of _tests_ based on in-jvm framework simpler > 4. Reduce surface area of impact when making modifications to tests > Potentially, we’d also like to use a plugin to detect API incompatibilities > between in-jvm DTest API and in-branch implementations, and start running > tests using shared in-jvm test repository with each existing implementation > in the branch. This entails both running tests for all branches whenever > there’s a change in tests jar and running tests for a specific branch > whenever the branch has changed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15399) Add ability to track state in repair
[
https://issues.apache.org/jira/browse/CASSANDRA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062910#comment-17062910
]
David Capwell commented on CASSANDRA-15399:
---
bq. "Reliable" message delivery alone in 15566 should make a large improvement..
How I wanted to do that was to dedupe requests on the recipients side, which
requires knowing what's going on (which is what this JIRA gave). You can do it
without this JIRA, but you would still need to do a lot of the heavy lifting of
this JIRA, which is why I liked them coupled.
bq. It requires all participants to be up and responsive through repair task
lifetime. It'd be nice to allow partial repair where some participants repair
successfully and it records failed(crashed/slow/message dropped) participants.
was planning to look into that after the transport and coordination was
reliable. There is some logic for if the the participants are down at the
start of the repair, but don't see anything about failing in the middle (other
than failing the repair); I know IR less than full but I assume both are the
same given the IR commit logic.
I see other open issues with repair that could make it more operational, stuff
like large ranges causing OOM, lack of cluster coordinator requiring external
coordinator (and everyone has their own), etc. My hope is to incrementally
improve this across several releases (I don't like giant patches).
> Add ability to track state in repair
>
>
> Key: CASSANDRA-15399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15399
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> To enhance the visibility in repair, we should expose internal state via
> virtual tables; the state should include coordinator as well as participant
> state (validation, sync, etc.)
> I propose the following tables:
> repairs - high level summary of the global state of repair; this should be
> called on the coordinator.
> {code:sql}
> CREATE TABLE repairs (
> id uuid,
> keyspace_name text,
> table_names frozen>,
> ranges frozen>,
> coordinator text,
> participants frozen>,
> state text,
> progress_percentage float,
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id) )
> )
> {code}
> repair_tasks - represents RepairJob and participants state. This will show
> if validations are running on participants and the progress they are making;
> this should be called on the coordinator.
> {code:sql}
> CREATE TABLE repair_tasks (
> id uuid,
> session_id uuid,
> keyspace_name text,
> table_name text,
> ranges frozen>,
> coordinator text,
> participant text,
> state text,
> state_description text,
> progress_percentage float, -- between 0.0 and 100.0
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id), session_id, table_name, participant )
> )
> {code}
> repair_validations - shows the state of the validation task and updated
> periodically while validation is running; this should be called on the
> participants.
> {code:sql}
> CREATE TABLE repair_validations (
> id uuid,
> session_id uuid,
> ranges frozen>,
> keyspace_name text,
> table_name text,
> initiator text,
> state text,
> progress_percentage float,
> queue_duration_ms bigint,
> runtime_duration_ms bigint,
> total_duration_ms bigint,
> estimated_partitions bigint,
> partitions_processed bigint,
> estimated_total_bytes bigint,
> failure_cause text,
> PRIMARY KEY ( (id), session_id, table_name )
> )
> {code}
> The main reason for exposing virtual tables rather than exposing through
> durable tables is to make sure what is exposed is accurate. In cases of
> write failures or node failures, the durable tables could become in-accurate
> and could add edge cases where the repair is not running but the tables say
> it is; by relying on repair's internal in-memory bookkeeping, these problems
> go away.
> This jira does not try to solve the following:
> 1) repair resiliency - there are edge cases where repair hits an error and
> runs forever (at least from nodetool's perspective).
> 2) repair stream tracking - I have not learned the streaming side yet and
> what I see is multiple implementations exist, so seems like high scope. My
> hope is to punt from this jira and tackle separately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For
[jira] [Updated] (CASSANDRA-15527) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testCrossSSTableQueries
[
https://issues.apache.org/jira/browse/CASSANDRA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15527:
Resolution: Fixed
Status: Resolved (was: Open)
Fixed in 15526 and 15528
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testCrossSSTableQueries
> ---
>
> Key: CASSANDRA-15527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15527
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15527.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key13, key2977, key2978, key2979,
> key2980, key2982, key2983, key2984, key2985, key6]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:340)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:286)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15527) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testCrossSSTableQueries
[
https://issues.apache.org/jira/browse/CASSANDRA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15527:
Status: Open (was: Patch Available)
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testCrossSSTableQueries
> ---
>
> Key: CASSANDRA-15527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15527
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15527.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key13, key2977, key2978, key2979,
> key2980, key2982, key2983, key2984, key2985, key6]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:340)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:286)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15527) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testCrossSSTableQueries
[
https://issues.apache.org/jira/browse/CASSANDRA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15527:
Test and Documentation Plan: Solved in 15526 and 15528
Status: Patch Available (was: In Progress)
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testCrossSSTableQueries
> ---
>
> Key: CASSANDRA-15527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15527
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15527.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key13, key2977, key2978, key2979,
> key2980, key2982, key2983, key2984, key2985, key6]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:340)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:286)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15528:
-
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/1a735f36b988157f1e7143852c4e3749649ee25d
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Commited, thanks!
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt, 15528-dtests-results.png
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15526) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testConcurrentMemtableReadsAndWrites
[
https://issues.apache.org/jira/browse/CASSANDRA-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15526:
-
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/731dfda99249c9b64fca5fa258c6078bfbe040ac
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed, thanks!
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testConcurrentMemtableReadsAndWrites
>
>
> Key: CASSANDRA-15526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15526
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI, Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15526-20200318-trunk-4.0.txt, 15526-dtests-results.png,
> 15526-trunk-4.0.txt, jvm_15526.zip, unit_tests_15526
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.index.sasi.utils.RangeIterator.(RangeIterator.java:46)
> at
> org.apache.cassandra.index.sasi.memory.KeyRangeIterator.(KeyRangeIterator.java:42)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex$ConcurrentTrie.search(TrieMemIndex.java:150)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex.search(TrieMemIndex.java:102)
> at
> org.apache.cassandra.index.sasi.memory.IndexMemtable.search(IndexMemtable.java:70)
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.searchMemtable(ColumnIndex.java:138)
> at org.apache.cassandra.index.sasi.TermIterator.build(TermIterator.java:91)
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndexes(QueryController.java:145)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:434)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68)
> at
> org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:301)
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:455)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getIndexed(SASIIndexTest.java:2576)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getPaged(SASIIndexTest.java:2537)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testConcurrentMemtableReadsAndWrites(SASIIndexTest.java:1108)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Fixed flakey test in SASIIndexTest by shutting down its ExecutorService
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 1a735f36b988157f1e7143852c4e3749649ee25d
Author: Gianluca Righetto
AuthorDate: Fri Mar 6 03:45:17 2020 -0300
Fixed flakey test in SASIIndexTest by shutting down its ExecutorService
Patch by Gianluca Righetto, reviewed by Ekaterina Dimitrova and Zhao
Yang for CASSANDRA-15528
---
CHANGES.txt| 1 +
.../apache/cassandra/index/sasi/SASIIndexTest.java | 35 --
2 files changed, 21 insertions(+), 15 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 8c8e74b..9621128 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Fixed flakey test in SASIIndexTest by shutting down its ExecutorService
(CASSANDRA-15528)
* Fixed empty check in TrieMemIndex due to potential state inconsistency in
ConcurrentSkipListMap (CASSANDRA-15526)
* Add compaction allocation measurement test (CASSANDRA-15388)
* Added UnleveledSSTables global and table level metric (CASSANDRA-15620)
diff --git a/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java
b/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java
index 0be05b9..c64bec9 100644
--- a/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java
+++ b/test/unit/org/apache/cassandra/index/sasi/SASIIndexTest.java
@@ -1099,25 +1099,30 @@ public class SASIIndexTest
int previousCount = 0;
-do
+try
{
-// this loop figures out if number of search results monotonically
increasing
-// to make sure that concurrent updates don't interfere with
reads, uses first_name and age
-// indexes to test correctness of both Trie and SkipList
ColumnIndex implementations.
+do
+{
+// this loop figures out if number of search results
monotonically increasing
+// to make sure that concurrent updates don't interfere with
reads, uses first_name and age
+// indexes to test correctness of both Trie and SkipList
ColumnIndex implementations.
-Set rows = getPaged(store, 100,
buildExpression(firstName, Operator.LIKE_CONTAINS,
UTF8Type.instance.decompose("a")),
- buildExpression(age,
Operator.EQ, Int32Type.instance.decompose(26)));
+Set rows = getPaged(store, 100,
buildExpression(firstName, Operator.LIKE_CONTAINS,
UTF8Type.instance.decompose("a")),
+ buildExpression(age,
Operator.EQ, Int32Type.instance.decompose(26)));
-Assert.assertTrue(previousCount <= rows.size());
-previousCount = rows.size();
-}
-while (updates.get() < writeCount);
-
-// to make sure that after all of the writes are done we can read all
"count" worth of rows
-Set rows = getPaged(store, 100,
buildExpression(firstName, Operator.LIKE_CONTAINS,
UTF8Type.instance.decompose("a")),
- buildExpression(age,
Operator.EQ, Int32Type.instance.decompose(26)));
+Assert.assertTrue(previousCount <= rows.size());
+previousCount = rows.size();
+}
+while (updates.get() < writeCount);
-Assert.assertEquals(writeCount, rows.size());
+// to make sure that after all of the writes are done we can read
all "count" worth of rows
+Set rows = getPaged(store, 100,
buildExpression(firstName, Operator.LIKE_CONTAINS,
UTF8Type.instance.decompose("a")),
+buildExpression(age, Operator.EQ,
Int32Type.instance.decompose(26)));
+Assert.assertEquals(writeCount, rows.size());
+} finally {
+scheduler.shutdownNow();
+scheduler.awaitTermination(30, TimeUnit.SECONDS);
+}
}
@Test
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (111f1da -> 1a735f3)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 111f1da Merge branch 'CASSANDRA-14904' into trunk new 731dfda Fixed empty check in TrieMemIndex due to potential state inconsistency in ConcurrentSkipListMap new 1a735f3 Fixed flakey test in SASIIndexTest by shutting down its ExecutorService The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 ++ .../index/sasi/memory/KeyRangeIterator.java| 4 +-- .../index/sasi/memory/SkipListMemIndex.java| 9 -- .../cassandra/index/sasi/memory/TrieMemIndex.java | 5 ++-- .../apache/cassandra/index/sasi/SASIIndexTest.java | 35 -- 5 files changed, 33 insertions(+), 22 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Fixed empty check in TrieMemIndex due to potential state inconsistency in ConcurrentSkipListMap
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 731dfda99249c9b64fca5fa258c6078bfbe040ac
Author: Gianluca Righetto
AuthorDate: Wed Mar 18 15:42:44 2020 -0300
Fixed empty check in TrieMemIndex due to potential state inconsistency in
ConcurrentSkipListMap
Patch by Gianluca Righetto, reviewed by Ekaterina Dimitrova and Zhao Yang
for CASSANDRA-15526
---
CHANGES.txt | 1 +
.../org/apache/cassandra/index/sasi/memory/KeyRangeIterator.java | 4 ++--
.../org/apache/cassandra/index/sasi/memory/SkipListMemIndex.java | 9 ++---
.../org/apache/cassandra/index/sasi/memory/TrieMemIndex.java | 5 +++--
4 files changed, 12 insertions(+), 7 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 508ae19..8c8e74b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Fixed empty check in TrieMemIndex due to potential state inconsistency in
ConcurrentSkipListMap (CASSANDRA-15526)
* Add compaction allocation measurement test (CASSANDRA-15388)
* Added UnleveledSSTables global and table level metric (CASSANDRA-15620)
* Added Virtual Table exposing Cassandra relevant system properties
(CASSANDRA-15616, CASSANDRA-15643)
diff --git
a/src/java/org/apache/cassandra/index/sasi/memory/KeyRangeIterator.java
b/src/java/org/apache/cassandra/index/sasi/memory/KeyRangeIterator.java
index 59d1122..0f681b7 100644
--- a/src/java/org/apache/cassandra/index/sasi/memory/KeyRangeIterator.java
+++ b/src/java/org/apache/cassandra/index/sasi/memory/KeyRangeIterator.java
@@ -37,9 +37,9 @@ public class KeyRangeIterator extends RangeIterator
{
private final DKIterator iterator;
-public KeyRangeIterator(ConcurrentSkipListSet keys)
+public KeyRangeIterator(ConcurrentSkipListSet keys, int size)
{
-super((Long) keys.first().getToken().getTokenValue(), (Long)
keys.last().getToken().getTokenValue(), keys.size());
+super((Long) keys.first().getToken().getTokenValue(), (Long)
keys.last().getToken().getTokenValue(), size);
this.iterator = new DKIterator(keys.iterator());
}
diff --git
a/src/java/org/apache/cassandra/index/sasi/memory/SkipListMemIndex.java
b/src/java/org/apache/cassandra/index/sasi/memory/SkipListMemIndex.java
index 69b57d0..9e44344 100644
--- a/src/java/org/apache/cassandra/index/sasi/memory/SkipListMemIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/memory/SkipListMemIndex.java
@@ -88,9 +88,12 @@ public class SkipListMemIndex extends MemIndex
}
RangeUnionIterator.Builder builder =
RangeUnionIterator.builder();
-search.values().stream()
- .filter(keys -> !keys.isEmpty())
- .forEach(keys -> builder.add(new
KeyRangeIterator(keys)));
+
+for (ConcurrentSkipListSet keys : search.values()) {
+int size;
+if ((size = keys.size()) > 0)
+builder.add(new KeyRangeIterator(keys, size));
+}
return builder.build();
}
diff --git a/src/java/org/apache/cassandra/index/sasi/memory/TrieMemIndex.java
b/src/java/org/apache/cassandra/index/sasi/memory/TrieMemIndex.java
index 21cb7ce..cebd68f 100644
--- a/src/java/org/apache/cassandra/index/sasi/memory/TrieMemIndex.java
+++ b/src/java/org/apache/cassandra/index/sasi/memory/TrieMemIndex.java
@@ -146,8 +146,9 @@ public class TrieMemIndex extends MemIndex
RangeUnionIterator.Builder builder =
RangeUnionIterator.builder();
for (ConcurrentSkipListSet keys : search)
{
-if (!keys.isEmpty())
-builder.add(new KeyRangeIterator(keys));
+int size;
+if ((size = keys.size()) > 0)
+builder.add(new KeyRangeIterator(keys, size));
}
return builder.build();
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15527) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testCrossSSTableQueries
[
https://issues.apache.org/jira/browse/CASSANDRA-15527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062890#comment-17062890
]
Ekaterina Dimitrova commented on CASSANDRA-15527:
-
Thank you [~gianluca]
Indeed, it solves this one too.
I will close the ticket as soon as the other two are committed.
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testCrossSSTableQueries
> ---
>
> Key: CASSANDRA-15527
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15527
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15527.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key13, key2977, key2978, key2979,
> key2980, key2982, key2983, key2984, key2985, key6]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:340)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testCrossSSTableQueries(SASIIndexTest.java:286)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15526) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testConcurrentMemtableReadsAndWrites
[
https://issues.apache.org/jira/browse/CASSANDRA-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062876#comment-17062876
]
Ekaterina Dimitrova commented on CASSANDRA-15526:
-
[~gianluca] you might want actually to squash first your commits
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testConcurrentMemtableReadsAndWrites
>
>
> Key: CASSANDRA-15526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15526
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI, Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15526-20200318-trunk-4.0.txt, 15526-dtests-results.png,
> 15526-trunk-4.0.txt, jvm_15526.zip, unit_tests_15526
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.index.sasi.utils.RangeIterator.(RangeIterator.java:46)
> at
> org.apache.cassandra.index.sasi.memory.KeyRangeIterator.(KeyRangeIterator.java:42)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex$ConcurrentTrie.search(TrieMemIndex.java:150)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex.search(TrieMemIndex.java:102)
> at
> org.apache.cassandra.index.sasi.memory.IndexMemtable.search(IndexMemtable.java:70)
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.searchMemtable(ColumnIndex.java:138)
> at org.apache.cassandra.index.sasi.TermIterator.build(TermIterator.java:91)
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndexes(QueryController.java:145)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:434)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68)
> at
> org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:301)
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:455)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getIndexed(SASIIndexTest.java:2576)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getPaged(SASIIndexTest.java:2537)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testConcurrentMemtableReadsAndWrites(SASIIndexTest.java:1108)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15526) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testConcurrentMemtableReadsAndWrites
[
https://issues.apache.org/jira/browse/CASSANDRA-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062872#comment-17062872
]
Ekaterina Dimitrova edited comment on CASSANDRA-15526 at 3/19/20, 7:07 PM:
---
LGTM +1
[~brandon.williams] can you, please commit this one? Thanks!
was (Author: e.dimitrova):
LGTM +1
[~brandon.williams]s can you, please commit this one? Thanks!
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testConcurrentMemtableReadsAndWrites
>
>
> Key: CASSANDRA-15526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15526
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI, Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15526-20200318-trunk-4.0.txt, 15526-dtests-results.png,
> 15526-trunk-4.0.txt, jvm_15526.zip, unit_tests_15526
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.index.sasi.utils.RangeIterator.(RangeIterator.java:46)
> at
> org.apache.cassandra.index.sasi.memory.KeyRangeIterator.(KeyRangeIterator.java:42)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex$ConcurrentTrie.search(TrieMemIndex.java:150)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex.search(TrieMemIndex.java:102)
> at
> org.apache.cassandra.index.sasi.memory.IndexMemtable.search(IndexMemtable.java:70)
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.searchMemtable(ColumnIndex.java:138)
> at org.apache.cassandra.index.sasi.TermIterator.build(TermIterator.java:91)
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndexes(QueryController.java:145)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:434)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68)
> at
> org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:301)
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:455)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getIndexed(SASIIndexTest.java:2576)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getPaged(SASIIndexTest.java:2537)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testConcurrentMemtableReadsAndWrites(SASIIndexTest.java:1108)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15526) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testConcurrentMemtableReadsAndWrites
[
https://issues.apache.org/jira/browse/CASSANDRA-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15526:
Status: Ready to Commit (was: Review In Progress)
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testConcurrentMemtableReadsAndWrites
>
>
> Key: CASSANDRA-15526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15526
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI, Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15526-20200318-trunk-4.0.txt, 15526-dtests-results.png,
> 15526-trunk-4.0.txt, jvm_15526.zip, unit_tests_15526
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.index.sasi.utils.RangeIterator.(RangeIterator.java:46)
> at
> org.apache.cassandra.index.sasi.memory.KeyRangeIterator.(KeyRangeIterator.java:42)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex$ConcurrentTrie.search(TrieMemIndex.java:150)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex.search(TrieMemIndex.java:102)
> at
> org.apache.cassandra.index.sasi.memory.IndexMemtable.search(IndexMemtable.java:70)
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.searchMemtable(ColumnIndex.java:138)
> at org.apache.cassandra.index.sasi.TermIterator.build(TermIterator.java:91)
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndexes(QueryController.java:145)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:434)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68)
> at
> org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:301)
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:455)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getIndexed(SASIIndexTest.java:2576)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getPaged(SASIIndexTest.java:2537)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testConcurrentMemtableReadsAndWrites(SASIIndexTest.java:1108)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15526) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testConcurrentMemtableReadsAndWrites
[
https://issues.apache.org/jira/browse/CASSANDRA-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062872#comment-17062872
]
Ekaterina Dimitrova commented on CASSANDRA-15526:
-
LGTM +1
[~brandon.williams]s can you, please commit this one? Thanks!
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testConcurrentMemtableReadsAndWrites
>
>
> Key: CASSANDRA-15526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15526
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI, Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15526-20200318-trunk-4.0.txt, 15526-dtests-results.png,
> 15526-trunk-4.0.txt, jvm_15526.zip, unit_tests_15526
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.index.sasi.utils.RangeIterator.(RangeIterator.java:46)
> at
> org.apache.cassandra.index.sasi.memory.KeyRangeIterator.(KeyRangeIterator.java:42)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex$ConcurrentTrie.search(TrieMemIndex.java:150)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex.search(TrieMemIndex.java:102)
> at
> org.apache.cassandra.index.sasi.memory.IndexMemtable.search(IndexMemtable.java:70)
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.searchMemtable(ColumnIndex.java:138)
> at org.apache.cassandra.index.sasi.TermIterator.build(TermIterator.java:91)
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndexes(QueryController.java:145)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:434)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68)
> at
> org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:301)
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:455)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getIndexed(SASIIndexTest.java:2576)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getPaged(SASIIndexTest.java:2537)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testConcurrentMemtableReadsAndWrites(SASIIndexTest.java:1108)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15528:
Status: Ready to Commit (was: Review In Progress)
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt, 15528-dtests-results.png
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062871#comment-17062871
]
Ekaterina Dimitrova commented on CASSANDRA-15528:
-
LGTM +1
[~brandon.williams] can you, please commit this one? Thanks!
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt, 15528-dtests-results.png
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15526) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testConcurrentMemtableReadsAndWrites
[
https://issues.apache.org/jira/browse/CASSANDRA-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062866#comment-17062866
]
Gianluca Righetto commented on CASSANDRA-15526:
---
[~e.dimitrova] I just attached the dtests
[results|https://issues.apache.org/jira/secure/attachment/12997150/15526-dtests-results.png]
for the open PR.
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testConcurrentMemtableReadsAndWrites
>
>
> Key: CASSANDRA-15526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15526
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI, Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15526-20200318-trunk-4.0.txt, 15526-dtests-results.png,
> 15526-trunk-4.0.txt, jvm_15526.zip, unit_tests_15526
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.index.sasi.utils.RangeIterator.(RangeIterator.java:46)
> at
> org.apache.cassandra.index.sasi.memory.KeyRangeIterator.(KeyRangeIterator.java:42)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex$ConcurrentTrie.search(TrieMemIndex.java:150)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex.search(TrieMemIndex.java:102)
> at
> org.apache.cassandra.index.sasi.memory.IndexMemtable.search(IndexMemtable.java:70)
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.searchMemtable(ColumnIndex.java:138)
> at org.apache.cassandra.index.sasi.TermIterator.build(TermIterator.java:91)
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndexes(QueryController.java:145)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:434)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68)
> at
> org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:301)
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:455)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getIndexed(SASIIndexTest.java:2576)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getPaged(SASIIndexTest.java:2537)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testConcurrentMemtableReadsAndWrites(SASIIndexTest.java:1108)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15526) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testConcurrentMemtableReadsAndWrites
[
https://issues.apache.org/jira/browse/CASSANDRA-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gianluca Righetto updated CASSANDRA-15526:
--
Attachment: 15526-dtests-results.png
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest
> testConcurrentMemtableReadsAndWrites
>
>
> Key: CASSANDRA-15526
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15526
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SASI, Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Attachments: 15526-20200318-trunk-4.0.txt, 15526-dtests-results.png,
> 15526-trunk-4.0.txt, jvm_15526.zip, unit_tests_15526
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.index.sasi.utils.RangeIterator.(RangeIterator.java:46)
> at
> org.apache.cassandra.index.sasi.memory.KeyRangeIterator.(KeyRangeIterator.java:42)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex$ConcurrentTrie.search(TrieMemIndex.java:150)
> at
> org.apache.cassandra.index.sasi.memory.TrieMemIndex.search(TrieMemIndex.java:102)
> at
> org.apache.cassandra.index.sasi.memory.IndexMemtable.search(IndexMemtable.java:70)
> at
> org.apache.cassandra.index.sasi.conf.ColumnIndex.searchMemtable(ColumnIndex.java:138)
> at org.apache.cassandra.index.sasi.TermIterator.build(TermIterator.java:91)
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndexes(QueryController.java:145)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:434)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.analyze(QueryPlan.java:57)
> at org.apache.cassandra.index.sasi.plan.QueryPlan.execute(QueryPlan.java:68)
> at
> org.apache.cassandra.index.sasi.SASIIndex.lambda$searcherFor$2(SASIIndex.java:301)
> at org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:455)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getIndexed(SASIIndexTest.java:2576)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.getPaged(SASIIndexTest.java:2537)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testConcurrentMemtableReadsAndWrites(SASIIndexTest.java:1108)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062849#comment-17062849
]
Gianluca Righetto commented on CASSANDRA-15528:
---
[~e.dimitrova] dtests results attached
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt, 15528-dtests-results.png
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gianluca Righetto updated CASSANDRA-15528:
--
Attachment: 15528-dtests-results.png
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt, 15528-dtests-results.png
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15539) Extract in-jvm API and tests out of Cassandra and into a separate repository
[ https://issues.apache.org/jira/browse/CASSANDRA-15539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062831#comment-17062831 ] Alex Petrov commented on CASSANDRA-15539: - [~dcapwell] [~djoshi] thank you for the review. I've rebased all branches and addressed all comments. CI is now green. It'd be great to move quickly and commit it shortly to avoid new rebases, since they're very time consuming. Thank you for understanding! > Extract in-jvm API and tests out of Cassandra and into a separate repository > > > Key: CASSANDRA-15539 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15539 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Time Spent: 1h 50m > Remaining Estimate: 0h > > Extract in-jvm DTest _API_ and tests into a separate repository that is > shared between Cassandra branches. Tests themselves should be buildable using > just API, which is not the case now, since cluster creation relies on impl > package, since we do not have factories / constructors in API. > Main goals we’re trying to achieve: > 1. We should be able to fail a build on API incompatibility between versions > 2. Make it as easy as possible to detect break APIs between versions. > 3. Make development of _tests_ based on in-jvm framework simpler > 4. Reduce surface area of impact when making modifications to tests > Potentially, we’d also like to use a plugin to detect API incompatibilities > between in-jvm DTest API and in-branch implementations, and start running > tests using shared in-jvm test repository with each existing implementation > in the branch. This entails both running tests for all branches whenever > there’s a change in tests jar and running tests for a specific branch > whenever the branch has changed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14970) New releases must supply SHA-256 and/or SHA-512 checksums
[ https://issues.apache.org/jira/browse/CASSANDRA-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062786#comment-17062786 ] Michael Semb Wever commented on CASSANDRA-14970: Status on this ticket is that [~mshuler] and myself are going to cut one more release with the expectations that no further manual changes or hacks are required during the post-vote actions. The post-vote actions are a bit tricky and can't be tested thoroughly without actually performing the releases. The next release cut likely is 4.0-alpha4, which is only waiting on CASSANDRA-15358, according to the [dev ML|https://lists.apache.org/thread.html/r2966aa37f42070ed58ed1642eb9f0e24f68a3ecca099d75a840c9ef6%40%3Cdev.cassandra.apache.org%3E]. > New releases must supply SHA-256 and/or SHA-512 checksums > - > > Key: CASSANDRA-14970 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14970 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Michael Shuler >Assignee: Michael Semb Wever >Priority: Urgent > Fix For: 4.0-alpha > > Attachments: > 0001-Update-downloads-for-sha256-sha512-checksum-files.patch, > 0001-Update-release-checksum-algorithms-to-SHA-256-SHA-512.patch, > ant-publish-checksum-fail.jpg, build_cassandra-2.1.png, build_trunk.png, > cassandra-2.1_14970_updated.patch > > > Release policy was updated around 9/2018 to state: > "For new releases, PMCs MUST supply SHA-256 and/or SHA-512; and SHOULD NOT > supply MD5 or SHA-1. Existing releases do not need to be changed." > build.xml needs to be updated from MD5 & SHA-1 to, at least, SHA-256 or both. > cassandra-builds/cassandra-release scripts need to be updated to work with > the new checksum files. > http://www.apache.org/dev/release-distribution#sigs-and-sums -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15626) Need microsecond precision for dropped columns so we can avoid timestamp issues
[ https://issues.apache.org/jira/browse/CASSANDRA-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-15626: --- Fix Version/s: (was: 4.0-beta) > Need microsecond precision for dropped columns so we can avoid timestamp > issues > --- > > Key: CASSANDRA-15626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15626 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Ryan Svihla >Priority: Normal > > In CASSANDRA-15557 the fix for the flaky test is reimplementing the logic > from CASSANDRA-12997 which was removed as part of CASSANDRA-13426 > However, since dropped columns are stored at a millisecond precision instead > of a microsecond precision and ClientState.getTimestamp adds microseconds on > each call we will lose the precision on save and some writes that should be > dropped could reappear. > Note views affected as well > > [https://github.com/apache/cassandra/blob/cb83fbff479bb90e9abeaade9e0f8843634c974d/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L712-L716] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15626) Need microsecond precision for dropped columns so we can avoid timestamp issues
[ https://issues.apache.org/jira/browse/CASSANDRA-15626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ryan Svihla updated CASSANDRA-15626: Fix Version/s: 4.0-beta > Need microsecond precision for dropped columns so we can avoid timestamp > issues > --- > > Key: CASSANDRA-15626 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15626 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Ryan Svihla >Priority: Normal > Fix For: 4.0-beta > > > In CASSANDRA-15557 the fix for the flaky test is reimplementing the logic > from CASSANDRA-12997 which was removed as part of CASSANDRA-13426 > However, since dropped columns are stored at a millisecond precision instead > of a microsecond precision and ClientState.getTimestamp adds microseconds on > each call we will lose the precision on save and some writes that should be > dropped could reappear. > Note views affected as well > > [https://github.com/apache/cassandra/blob/cb83fbff479bb90e9abeaade9e0f8843634c974d/src/java/org/apache/cassandra/schema/SchemaKeyspace.java#L712-L716] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports
[ https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-14904: - Fix Version/s: (was: 3.11.x) (was: 4.0) 4.0-alpha 3.11.7 > SSTableloader doesn't understand listening for CQL connections on multiple > ports > > > Key: CASSANDRA-14904 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14904 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Kurt Greaves >Assignee: Ian Cleasby >Priority: Low > Fix For: 3.11.7, 4.0-alpha > > > sstableloader only searches the yaml for native_transport_port, so if > native_transport_port_ssl is set and encryption is enabled sstableloader will > fail to connect as it will use the non-SSL port for the connection. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports
[ https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-14904: - Since Version: 3.11.6 Source Control Link: https://github.com/apache/cassandra/commit/111f1da0075e1a7d6b1a5dec0f3a90d3ca427083 Resolution: Fixed Status: Resolved (was: Ready to Commit) I rebased this and fixed the test config for 3.11 so it passes correctly. Committed. > SSTableloader doesn't understand listening for CQL connections on multiple > ports > > > Key: CASSANDRA-14904 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14904 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Kurt Greaves >Assignee: Ian Cleasby >Priority: Low > Fix For: 4.0, 3.11.x > > > sstableloader only searches the yaml for native_transport_port, so if > native_transport_port_ssl is set and encryption is enabled sstableloader will > fail to connect as it will use the non-SSL port for the connection. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports
[ https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-14904: - Status: Ready to Commit (was: Review In Progress) > SSTableloader doesn't understand listening for CQL connections on multiple > ports > > > Key: CASSANDRA-14904 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14904 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Kurt Greaves >Assignee: Ian Cleasby >Priority: Low > Fix For: 4.0, 3.11.x > > > sstableloader only searches the yaml for native_transport_port, so if > native_transport_port_ssl is set and encryption is enabled sstableloader will > fail to connect as it will use the non-SSL port for the connection. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports
[ https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062717#comment-17062717 ] Brandon Williams commented on CASSANDRA-14904: -- bq. With this patch, the behavior will be that we will always use the native_transport_port_ssl if both are set unless overridden by command line. Actually if the non-ssl native_transport_port is set, it will always use that. I don't see a problem with that either, if for some reason the port is inaccessible you'll find out quickly and be able to adjust the config. bq. No matter what we choose to do about this behavior, a test case that tests the case of both being set would be good to add. This actually tests both, native with the stock config, and ssl with the supplied one. > SSTableloader doesn't understand listening for CQL connections on multiple > ports > > > Key: CASSANDRA-14904 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14904 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Kurt Greaves >Assignee: Ian Cleasby >Priority: Low > Fix For: 4.0, 3.11.x > > > sstableloader only searches the yaml for native_transport_port, so if > native_transport_port_ssl is set and encryption is enabled sstableloader will > fail to connect as it will use the non-SSL port for the connection. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated: Allow sstableloader to use SSL on the native port.
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.11 by this push:
new c7a8730 Allow sstableloader to use SSL on the native port.
c7a8730 is described below
commit c7a8730447d38698e6b21cf5a3226cd059a543b6
Author: Brandon Williams
AuthorDate: Thu Mar 19 10:50:25 2020 -0500
Allow sstableloader to use SSL on the native port.
---
CHANGES.txt| 1 +
.../org/apache/cassandra/tools/LoaderOptions.java | 16 --
test/conf/unit-test-conf/test-native-port.yaml | 57 ++
.../apache/cassandra/tools/LoaderOptionsTest.java | 47 ++
4 files changed, 117 insertions(+), 4 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index c596be9..bc6ad0e 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.11.7
+ * Allow sstableloader to use SSL on the native port (CASSANDRA-14904)
Merged from 3.0:
* Run evictFromMembership in GossipStage (CASSANDRA-15592)
Merged from 2.2:
diff --git a/src/java/org/apache/cassandra/tools/LoaderOptions.java
b/src/java/org/apache/cassandra/tools/LoaderOptions.java
index 38317b6..b3110f2 100644
--- a/src/java/org/apache/cassandra/tools/LoaderOptions.java
+++ b/src/java/org/apache/cassandra/tools/LoaderOptions.java
@@ -374,10 +374,6 @@ public class LoaderOptions
config.inter_dc_stream_throughput_outbound_megabits_per_sec = 0;
}
-if (cmd.hasOption(NATIVE_PORT_OPTION))
-nativePort =
Integer.parseInt(cmd.getOptionValue(NATIVE_PORT_OPTION));
-else
-nativePort = config.native_transport_port;
if (cmd.hasOption(STORAGE_PORT_OPTION))
storagePort =
Integer.parseInt(cmd.getOptionValue(STORAGE_PORT_OPTION));
else
@@ -406,6 +402,18 @@ public class LoaderOptions
clientEncOptions.enabled = true;
}
+if (cmd.hasOption(NATIVE_PORT_OPTION))
+{
+nativePort =
Integer.parseInt(cmd.getOptionValue(NATIVE_PORT_OPTION));
+}
+else
+{
+if (config.native_transport_port_ssl != null &&
(config.client_encryption_options.enabled || clientEncOptions.enabled))
+nativePort = config.native_transport_port_ssl;
+else
+nativePort = config.native_transport_port;
+}
+
if (cmd.hasOption(SSL_TRUSTSTORE))
{
clientEncOptions.truststore =
cmd.getOptionValue(SSL_TRUSTSTORE);
diff --git a/test/conf/unit-test-conf/test-native-port.yaml
b/test/conf/unit-test-conf/test-native-port.yaml
new file mode 100644
index 000..a90f100
--- /dev/null
+++ b/test/conf/unit-test-conf/test-native-port.yaml
@@ -0,0 +1,57 @@
+#
+# Warning!
+# Consider the effects on 'o.a.c.i.s.LegacySSTableTest' before changing
schemas in this file.
+#
+cluster_name: Test Cluster
+# memtable_allocation_type: heap_buffers
+memtable_allocation_type: offheap_objects
+commitlog_sync: batch
+commitlog_sync_batch_window_in_ms: 1.0
+commitlog_segment_size_in_mb: 5
+commitlog_directory: build/test/cassandra/commitlog
+cdc_raw_directory: build/test/cassandra/cdc_raw
+cdc_enabled: false
+hints_directory: build/test/cassandra/hints
+partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
+listen_address: 127.0.0.1
+storage_port: 7010
+rpc_port: 9170
+start_native_transport: true
+native_transport_port_ssl: 9142
+column_index_size_in_kb: 4
+saved_caches_directory: build/test/cassandra/saved_caches
+data_file_directories:
+- build/test/cassandra/data
+disk_access_mode: mmap
+seed_provider:
+- class_name: org.apache.cassandra.locator.SimpleSeedProvider
+ parameters:
+ - seeds: "127.0.0.1"
+endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
+dynamic_snitch: true
+request_scheduler: org.apache.cassandra.scheduler.RoundRobinScheduler
+request_scheduler_id: keyspace
+server_encryption_options:
+internode_encryption: none
+keystore: conf/.keystore
+keystore_password: cassandra
+truststore: conf/.truststore
+truststore_password: cassandra
+incremental_backups: true
+concurrent_compactors: 4
+compaction_throughput_mb_per_sec: 0
+row_cache_class_name: org.apache.cassandra.cache.OHCProvider
+row_cache_size_in_mb: 16
+enable_user_defined_functions: true
+enable_scripted_user_defined_functions: true
+prepared_statements_cache_size_mb: 1
+client_encryption_options:
+ enabled: true
+ # If enabled and optional is set to true encrypted and unencrypted
connections are handled.
+ optional: false
+ keystore: conf/cassandra_ssl_test.keystore
+ keystore_password: cassandra
+ #
[cassandra] 03/03: Merge branch 'CASSANDRA-14904' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 111f1da0075e1a7d6b1a5dec0f3a90d3ca427083 Merge: 1a25877 1a14c3a Author: Brandon Williams AuthorDate: Thu Mar 19 11:01:07 2020 -0500 Merge branch 'CASSANDRA-14904' into trunk CHANGES.txt| 2 + .../org/apache/cassandra/tools/LoaderOptions.java | 16 -- test/conf/unit-test-conf/test-native-port.yaml | 61 ++ .../apache/cassandra/tools/LoaderOptionsTest.java | 47 + 4 files changed, 122 insertions(+), 4 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/03: Allow sstableloader to use SSL on the native port.
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 1a14c3ac483ce3a118b3f8acfdd3d26d3c5a0241
Author: Brandon Williams
AuthorDate: Thu Mar 19 10:43:45 2020 -0500
Allow sstableloader to use SSL on the native port.
Patch by Ian Cleasby, reviewed by brandonwilliams for CASSANDRA-14904
---
CHANGES.txt| 2 +
.../org/apache/cassandra/tools/LoaderOptions.java | 16 --
test/conf/unit-test-conf/test-native-port.yaml | 61 ++
.../apache/cassandra/tools/LoaderOptionsTest.java | 47 +
4 files changed, 122 insertions(+), 4 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 6a1cf93..508ae19 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -34,6 +34,8 @@
* Make native_transport_max_concurrent_requests_in_bytes updatable
(CASSANDRA-15519)
* Cleanup and improvements to IndexInfo/ColumnIndex (CASSANDRA-15469)
* Potential Overflow in DatabaseDescriptor Functions That Convert Between
KB/MB & Bytes (CASSANDRA-15470)
+Merged from 3.11:
+ * Allow sstableloader to use SSL on the native port (CASSANDRA-14904)
Merged from 3.0:
* Run evictFromMembership in GossipStage (CASSANDRA-15592)
Merged from 2.2:
diff --git a/src/java/org/apache/cassandra/tools/LoaderOptions.java
b/src/java/org/apache/cassandra/tools/LoaderOptions.java
index 9acf8a5..1801941 100644
--- a/src/java/org/apache/cassandra/tools/LoaderOptions.java
+++ b/src/java/org/apache/cassandra/tools/LoaderOptions.java
@@ -386,10 +386,6 @@ public class LoaderOptions
config.inter_dc_stream_throughput_outbound_megabits_per_sec = 0;
}
-if (cmd.hasOption(NATIVE_PORT_OPTION))
-nativePort =
Integer.parseInt(cmd.getOptionValue(NATIVE_PORT_OPTION));
-else
-nativePort = config.native_transport_port;
if (cmd.hasOption(INITIAL_HOST_ADDRESS_OPTION))
{
@@ -462,6 +458,18 @@ public class LoaderOptions
clientEncOptions = clientEncOptions.withEnabled(true);
}
+if (cmd.hasOption(NATIVE_PORT_OPTION))
+{
+nativePort =
Integer.parseInt(cmd.getOptionValue(NATIVE_PORT_OPTION));
+}
+else
+{
+if (config.native_transport_port_ssl != null &&
(config.client_encryption_options.enabled || clientEncOptions.enabled))
+nativePort = config.native_transport_port_ssl;
+else
+nativePort = config.native_transport_port;
+}
+
if (cmd.hasOption(SSL_TRUSTSTORE))
{
clientEncOptions =
clientEncOptions.withTrustStore(cmd.getOptionValue(SSL_TRUSTSTORE));
diff --git a/test/conf/unit-test-conf/test-native-port.yaml
b/test/conf/unit-test-conf/test-native-port.yaml
new file mode 100644
index 000..b46525f
--- /dev/null
+++ b/test/conf/unit-test-conf/test-native-port.yaml
@@ -0,0 +1,61 @@
+#
+# Warning!
+# Consider the effects on 'o.a.c.i.s.LegacySSTableTest' before changing
schemas in this file.
+#
+cluster_name: Test Cluster
+# memtable_allocation_type: heap_buffers
+memtable_allocation_type: offheap_objects
+commitlog_sync: batch
+commitlog_sync_batch_window_in_ms: 1.0
+commitlog_segment_size_in_mb: 5
+commitlog_directory: build/test/cassandra/commitlog
+# commitlog_compression:
+# - class_name: LZ4Compressor
+cdc_raw_directory: build/test/cassandra/cdc_raw
+cdc_enabled: false
+hints_directory: build/test/cassandra/hints
+partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
+listen_address: 127.0.0.1
+storage_port: 7010
+ssl_storage_port: 7011
+start_native_transport: true
+native_transport_port_ssl: 9142
+column_index_size_in_kb: 4
+saved_caches_directory: build/test/cassandra/saved_caches
+data_file_directories:
+- build/test/cassandra/data
+disk_access_mode: mmap
+seed_provider:
+- class_name: org.apache.cassandra.locator.SimpleSeedProvider
+ parameters:
+ - seeds: "127.0.0.1:7010"
+endpoint_snitch: org.apache.cassandra.locator.SimpleSnitch
+dynamic_snitch: true
+server_encryption_options:
+ internode_encryption: none
+ keystore: conf/.keystore
+ keystore_password: cassandra
+ truststore: conf/.truststore
+ truststore_password: cassandra
+incremental_backups: true
+concurrent_compactors: 4
+compaction_throughput_mb_per_sec: 0
+row_cache_class_name: org.apache.cassandra.cache.OHCProvider
+row_cache_size_in_mb: 16
+enable_user_defined_functions: true
+enable_scripted_user_defined_functions: true
+prepared_statements_cache_size_mb: 1
+corrupted_tombstone_strategy: exception
+stream_entire_sstables: true
+stream_throughput_outbound_megabits_per_sec: 2
+
+client_encryption_options:
+ enabled: true
+ #
[cassandra] 02/03: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 1a25877095f9eb416077bca00a0fe990195502db Merge: 107dcc7 c7a8730 Author: Brandon Williams AuthorDate: Thu Mar 19 11:00:46 2020 -0500 Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (107dcc7 -> 111f1da)
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.
from 107dcc7 Add JAVA_HOME and java.rmi.server.randomID to the
system_properties table
new c7a8730 Allow sstableloader to use SSL on the native port.
new 1a25877 Merge branch 'cassandra-3.11' into trunk
new 1a14c3a Allow sstableloader to use SSL on the native port.
new 111f1da Merge branch 'CASSANDRA-14904' into trunk
The 4 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
CHANGES.txt| 2 +
.../org/apache/cassandra/tools/LoaderOptions.java | 16 ++--
.../test-native-port.yaml} | 35 ++--
.../apache/cassandra/tools/LoaderOptionsTest.java | 47 ++
4 files changed, 83 insertions(+), 17 deletions(-)
copy test/conf/{cassandra.yaml => unit-test-conf/test-native-port.yaml} (64%)
create mode 100644 test/unit/org/apache/cassandra/tools/LoaderOptionsTest.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062564#comment-17062564
]
Ekaterina Dimitrova edited comment on CASSANDRA-15528 at 3/19/20, 4:04 PM:
---
That's a good topic. I was looking into the CHANGES.txt and it looks like it's
a practice changes to the unit tests to be added.
Not sure whether someone started that unnecessarily and the others followed
him/her or this is how it was agreed at some point.
was (Author: e.dimitrova):
That's a good topic. I was looking into the CHANGES.txt and it looks like it's
a practice changes to the unit tests to be added.
Not sure whether someone started that unnecessarily and the others followed
him/her or this is how it was agreed.
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14904) SSTableloader doesn't understand listening for CQL connections on multiple ports
[ https://issues.apache.org/jira/browse/CASSANDRA-14904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-14904: - Reviewers: Brandon Williams, Brandon Williams (was: Brandon Williams) Brandon Williams, Brandon Williams Status: Review In Progress (was: Patch Available) > SSTableloader doesn't understand listening for CQL connections on multiple > ports > > > Key: CASSANDRA-14904 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14904 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Kurt Greaves >Assignee: Ian Cleasby >Priority: Low > Fix For: 4.0, 3.11.x > > > sstableloader only searches the yaml for native_transport_port, so if > native_transport_port_ssl is set and encryption is enabled sstableloader will > fail to connect as it will use the non-SSL port for the connection. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062689#comment-17062689
]
Sam Tunnicliffe commented on CASSANDRA-15229:
-
bq. If we address the problem, we should also utilise the BufferPool for native
transport connections like we do for internode messaging, and reduce the number
of pooling solutions we employ.
Just FYI, I'm looking at this aspect of the ticket for CASSANDRA-15299
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15229) BufferPool Regression
[
https://issues.apache.org/jira/browse/CASSANDRA-15229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062682#comment-17062682
]
ZhaoYang commented on CASSANDRA-15229:
--
It looks like current {{normal}} chunk size is already 128kb with 2kb
allocation granularity and 32byte tiny allocation granularity.
[~benedict] do you plan to work on this ticket after CASSANDRA-15358? if not, I
can take a stab at this ticket - seems fun and challenging..
> BufferPool Regression
> -
>
> Key: CASSANDRA-15229
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15229
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Caching
>Reporter: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> The BufferPool was never intended to be used for a {{ChunkCache}}, and we
> need to either change our behaviour to handle uncorrelated lifetimes or use
> something else. This is particularly important with the default chunk size
> for compressed sstables being reduced. If we address the problem, we should
> also utilise the BufferPool for native transport connections like we do for
> internode messaging, and reduce the number of pooling solutions we employ.
> Probably the best thing to do is to improve BufferPool’s behaviour when used
> for things with uncorrelated lifetimes, which essentially boils down to
> tracking those chunks that have not been freed and re-circulating them when
> we run out of completely free blocks. We should probably also permit
> instantiating separate {{BufferPool}}, so that we can insulate internode
> messaging from the {{ChunkCache}}, or at least have separate memory bounds
> for each, and only share fully-freed chunks.
> With these improvements we can also safely increase the {{BufferPool}} chunk
> size to 128KiB or 256KiB, to guarantee we can fit compressed pages and reduce
> the amount of global coordination and per-allocation overhead. We don’t need
> 1KiB granularity for allocations, nor 16 byte granularity for tiny
> allocations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15543) flaky test org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
[
https://issues.apache.org/jira/browse/CASSANDRA-15543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15543:
---
Reviewers: Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Status: Review In Progress (was: Patch Available)
> flaky test
> org.apache.cassandra.distributed.test.SimpleReadWriteTest.readWithSchemaDisagreement
> ---
>
> Key: CASSANDRA-15543
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15543
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Kevin Gallardo
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> This fails infrequently, last seen failure was on java 8
> {code}
> junit.framework.AssertionFailedError
> at
> org.apache.cassandra.distributed.test.DistributedReadWritePathTest.readWithSchemaDisagreement(DistributedReadWritePathTest.java:276)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15586) 4.0 quality testing: Cluster Setup and Maintenance
[ https://issues.apache.org/jira/browse/CASSANDRA-15586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062617#comment-17062617 ] Ekaterina Dimitrova commented on CASSANDRA-15586: - Thank you for your input. I saw also Scott's comment that this one was created with regards to the user experience. So in this case I guess the plan would be to go through the mentioned steps now and verify any potential issues/fix them(just with the tests we have/mentioned). Do one more verification before the final release to confirm there is nothing missed/new. As [~mshuler] described the process. > 4.0 quality testing: Cluster Setup and Maintenance > -- > > Key: CASSANDRA-15586 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15586 > Project: Cassandra > Issue Type: Task > Components: Test/dtest >Reporter: Josh McKenzie >Assignee: Ekaterina Dimitrova >Priority: Normal > Labels: 4.0-QA > Fix For: 4.0-rc > > > We want 4.0 to be easy for users to setup out of the box and just work. This > means having low friction when users download the Cassandra package and start > running it. For example, users should be able to easily configure and start > new 4.0 clusters and have tokens distributed evenly. Another example is > packaging, it should be easy to install Cassandra on all supported platforms > (e.g. packaging) and have Cassandra use standard platform integrations. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062564#comment-17062564
]
Ekaterina Dimitrova commented on CASSANDRA-15528:
-
That's a good topic. I was looking into the CHANGES.txt and it looks like it's
a practice changes to the unit tests to be added.
Not sure whether someone started that unnecessarily and the others followed
him/her or this is how it was agreed.
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062553#comment-17062553
]
ZhaoYang edited comment on CASSANDRA-15528 at 3/19/20, 1:16 PM:
the patch LGTM, one minor nit: I think we don't need to update release note or
CHANGES.txt for test fix, as it's not user visible..
was (Author: jasonstack):
the patch LGTM, one minor nit: I think we don't need to update release note for
test fix, as it's not user visible..
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15528) Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
[
https://issues.apache.org/jira/browse/CASSANDRA-15528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang updated CASSANDRA-15528:
-
Reviewers: Ekaterina Dimitrova, ZhaoYang, ZhaoYang (was: Ekaterina
Dimitrova, ZhaoYang)
Ekaterina Dimitrova, ZhaoYang, ZhaoYang
Status: Review In Progress (was: Patch Available)
the patch LGTM, one minor nit: I think we don't need to update release note for
test fix, as it's not user visible..
> Fix flakey test - org.apache.cassandra.index.sasi.SASIIndexTest testInvalidate
> --
>
> Key: CASSANDRA-15528
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15528
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Gianluca Righetto
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: 15528-4.0-alpha.txt
>
>
> {code}
> junit.framework.AssertionFailedError: [key0, key2918, key2919, key2920,
> key2921, key2922, key2924, key2926, key2927, key2928]
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:874)
> at
> org.apache.cassandra.index.sasi.SASIIndexTest.testInvalidate(SASIIndexTest.java:852)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15630) Fix flakey testSerializeError - org.apache.cassandra.net.ConnectionTest
[
https://issues.apache.org/jira/browse/CASSANDRA-15630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062522#comment-17062522
]
ZhaoYang commented on CASSANDRA-15630:
--
linked both CASSANDRA-15628 and CASSANDRA-15629..
> Fix flakey testSerializeError - org.apache.cassandra.net.ConnectionTest
> ---
>
> Key: CASSANDRA-15630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15630
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> The test fails sometimes with the following error message and trace.
> {code:java}
> processed count values don't match expected:<90> but was:<89>
> junit.framework.AssertionFailedError: processed count values don't match
> expected:<90> but was:<89>
> at
> org.apache.cassandra.net.ConnectionUtils$InboundCountChecker.doCheck(ConnectionUtils.java:217)
> at
> org.apache.cassandra.net.ConnectionUtils$InboundCountChecker.check(ConnectionUtils.java:200)
> at
> org.apache.cassandra.net.ConnectionTest.lambda$testSerializeError$24(ConnectionTest.java:494)
> at
> org.apache.cassandra.net.ConnectionTest.lambda$doTest$8(ConnectionTest.java:240)
> at
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:260)
> at org.apache.cassandra.net.ConnectionTest.doTest(ConnectionTest.java:238)
> at org.apache.cassandra.net.ConnectionTest.test(ConnectionTest.java:227)
> at
> org.apache.cassandra.net.ConnectionTest.testSerializeError(ConnectionTest.java:435){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14754) Add verification of state machine in StreamSession
[
https://issues.apache.org/jira/browse/CASSANDRA-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ZhaoYang reassigned CASSANDRA-14754:
Assignee: ZhaoYang
> Add verification of state machine in StreamSession
> --
>
> Key: CASSANDRA-14754
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14754
> Project: Cassandra
> Issue Type: Task
> Components: Legacy/Streaming and Messaging
>Reporter: Jason Brown
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamSession}} contains an implicit state machine, but we have no
> verification of the safety of the transitions between states. For example, we
> have no checks to ensure we cannot leave the final states (COMPLETED, FAILED).
> I propose we add some program logic in {{StreamSession}}, tests, and
> documentation to ensure the correctness of the state transitions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-14801) calculatePendingRanges no longer safe for multiple adjacent range movements
[
https://issues.apache.org/jira/browse/CASSANDRA-14801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Sorokoumov reassigned CASSANDRA-14801:
Assignee: Aleksandr Sorokoumov
> calculatePendingRanges no longer safe for multiple adjacent range movements
> ---
>
> Key: CASSANDRA-14801
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14801
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination, Legacy/Distributed Metadata
>Reporter: Benedict Elliott Smith
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> Correctness depended upon the narrowing to a {{Set}},
> which we no longer do - we maintain a collection of all {{Replica}}. Our
> {{RangesAtEndpoint}} collection built by {{getPendingRanges}} can as a result
> contain the same endpoint multiple times; and our {{EndpointsForToken}}
> obtained by {{TokenMetadata.pendingEndpointsFor}} may fail to be constructed,
> resulting in cluster-wide failures for writes to the affected token ranges
> for the duration of the range movement.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15599) Improve logging around incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-15599: Status: Ready to Commit (was: Review In Progress) +1 > Improve logging around incremental repair > - > > Key: CASSANDRA-15599 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15599 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > Some repair logging messages omit the parent session id and we don't log > which sstables are created by anticompaction. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15399) Add ability to track state in repair
[
https://issues.apache.org/jira/browse/CASSANDRA-15399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17062305#comment-17062305
]
ZhaoYang commented on CASSANDRA-15399:
--
bq. It was asked to not be 4.0, but CASSANDRA-15566 is much easier with this
I am fine to leave it out of 4.0. "Reliable" message delivery alone in 15566
should make a large improvement..
Some random thoughts about "repair state tracking".. One of the reasons why
repair isn't operator friendly is lack of fault-tolerance. It requires all
participants to be up and responsive through repair task lifetime. It'd be nice
to allow partial repair where some participants repair successfully and it
records failed(crashed/slow/message dropped) participants. A new tool can be
built to repair failed participants only.. By allowing partial repair, it's
more fault tolerant and more likely to make progress on each repair task.
> Add ability to track state in repair
>
>
> Key: CASSANDRA-15399
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15399
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Repair
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> To enhance the visibility in repair, we should expose internal state via
> virtual tables; the state should include coordinator as well as participant
> state (validation, sync, etc.)
> I propose the following tables:
> repairs - high level summary of the global state of repair; this should be
> called on the coordinator.
> {code:sql}
> CREATE TABLE repairs (
> id uuid,
> keyspace_name text,
> table_names frozen>,
> ranges frozen>,
> coordinator text,
> participants frozen>,
> state text,
> progress_percentage float,
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id) )
> )
> {code}
> repair_tasks - represents RepairJob and participants state. This will show
> if validations are running on participants and the progress they are making;
> this should be called on the coordinator.
> {code:sql}
> CREATE TABLE repair_tasks (
> id uuid,
> session_id uuid,
> keyspace_name text,
> table_name text,
> ranges frozen>,
> coordinator text,
> participant text,
> state text,
> state_description text,
> progress_percentage float, -- between 0.0 and 100.0
> last_updated_at_millis bigint,
> duration_micro bigint,
> failure_cause text,
> PRIMARY KEY ( (id), session_id, table_name, participant )
> )
> {code}
> repair_validations - shows the state of the validation task and updated
> periodically while validation is running; this should be called on the
> participants.
> {code:sql}
> CREATE TABLE repair_validations (
> id uuid,
> session_id uuid,
> ranges frozen>,
> keyspace_name text,
> table_name text,
> initiator text,
> state text,
> progress_percentage float,
> queue_duration_ms bigint,
> runtime_duration_ms bigint,
> total_duration_ms bigint,
> estimated_partitions bigint,
> partitions_processed bigint,
> estimated_total_bytes bigint,
> failure_cause text,
> PRIMARY KEY ( (id), session_id, table_name )
> )
> {code}
> The main reason for exposing virtual tables rather than exposing through
> durable tables is to make sure what is exposed is accurate. In cases of
> write failures or node failures, the durable tables could become in-accurate
> and could add edge cases where the repair is not running but the tables say
> it is; by relying on repair's internal in-memory bookkeeping, these problems
> go away.
> This jira does not try to solve the following:
> 1) repair resiliency - there are edge cases where repair hits an error and
> runs forever (at least from nodetool's perspective).
> 2) repair stream tracking - I have not learned the streaming side yet and
> what I see is multiple implementations exist, so seems like high scope. My
> hope is to punt from this jira and tackle separately.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
