[jira] [Commented] (CASSANDRA-16663) Request-Based Native Transport Rate-Limiting

2021-06-08 Thread Caleb Rackliffe (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359652#comment-17359652
 ] 

Caleb Rackliffe commented on CASSANDRA-16663:
-

Just a quick update...making progress on [a 
prototype|https://github.com/apache/cassandra/pull/1045] for the V4 
protocol/transport. What it does, more or less, is takes the Guava 
{{RateLimiter}}, defers waiting/sleeping decisions to the caller, and defers to 
a non-event-loop thread (if necessary) the logic around reenabling auto-read on 
the channel. So far, it follows the configured limit pretty closely in some 
rough experiments based on {{SimpleClientPerfTest}}.

> Request-Based Native Transport Rate-Limiting
> 
>
> Key: CASSANDRA-16663
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16663
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.x
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Together, CASSANDRA-14855, CASSANDRA-15013, and CASSANDRA-15519 added support 
> for a runtime-configurable, per-coordinator limit on the number of bytes 
> allocated for concurrent requests over the native protocol. It supports 
> channel back-pressure by default, and optionally supports throwing 
> OverloadedException if that is requested in the relevant connection’s STARTUP 
> message.
> This can be an effective tool to prevent the coordinator from running out of 
> memory, but it may not correspond to how expensive a queries are or provide a 
> direct conceptual mapping to how users think about request capacity. I 
> propose adding the option of request-based (or perhaps more correctly 
> message-based) back-pressure, coexisting with (and reusing the logic that 
> supports) the current bytes-based back-pressure.
> _We can roll this forward in phases_, where the server’s cost accounting 
> becomes more accurate, we segment limits by operation type/keyspace/etc., and 
> the client/driver reacts more intelligently to (especially non-back-pressure) 
> overload, _but something minimally viable could look like this_:
> 1.) Reuse most of the existing logic in Limits, et al. to support a simple 
> per-coordinator limit only on native transport requests per second. Under 
> this limit will be CQL reads and writes, but also auth requests, prepare 
> requests, and batches. This is obviously simplistic, and it does not account 
> for the variation in cost between individual queries, but even a fixed cost 
> model should be useful in aggregate.
>  * If the client specifies THROW_ON_OVERLOAD in its STARTUP message at 
> connection time, a breach of the per-node limit will result in an 
> OverloadedException being propagated to the client, and the server will 
> discard the request.
>  * If THROW_ON_OVERLOAD is not specified, the server will stop consuming 
> messages from the channel/socket, which should back-pressure the client, 
> while the message continues to be processed.
> 2.) This limit is infinite by default (or simply disabled), and can be 
> enabled via the YAML config or JMX at runtime. (It might be cleaner to have a 
> no-op rate limiter that's used when the feature is disabled entirely.)
> 3.) The current value of the limit is available via JMX, and metrics around 
> coordinator operations/second are already available to compare against it.
> 4.) Any interaction with existing byte-based limits will intersect. (i.e. A 
> breach of any limit, bytes or request-based, will actuate back-pressure or 
> OverloadedExceptions.)
> In this first pass, explicitly out of scope would be any work on the 
> client/driver side.
> In terms of validation/testing, our biggest concern with anything that adds 
> overhead on a very hot path is performance. In particular, we want to fully 
> understand how the client and server perform along two axes constituting 4 
> scenarios. Those are a.) whether or not we are breaching the request limit 
> and b.) whether the server is throwing on overload at the behest of the 
> client. Having said that, query execution should dwarf the cost of limit 
> accounting.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16720) Fix org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test

2021-06-08 Thread Stefan Miklosovic (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359547#comment-17359547
 ] 

Stefan Miklosovic edited comment on CASSANDRA-16720 at 6/8/21, 8:07 PM:


I am not sure I follow, I do exactly same steps as in the test but manually and 
it replays it just fine.

{code}
cql> CREATE KEYSPACE fql_ks WITH replication = {'class': 'SimpleStrategy', 
'replication_factor': 1};
shell> nodetool enablefullquerylog --path /tmp/cassandra/fqllogs
cql> CREATE TABLE fql_ks.fql_table (id int primary key);
cql> INSERT INTO fql_ks.fql_table (id) VALUES (1);
shell> nodetool disablefullquerylog
cql> DROP TABLE fql_ks.fql_table;
shell> nodetool replay --keyspace fql_ks --target 127.0.0.1 -- 
/tmp/cassandra/fqllogs 
cql> SELECT * from fql_ks.fql_table // this fails which we want
shell> nodetool replay --keyspace fql_ks --target 127.0.0.1 
--replay-ddl-statements -- /tmp/cassandra/fqllogs 
cql> SELECT * from fql_ks.fql_table // I get 1 row
{code}


was (Author: stefan.miklosovic):
I am not sure I follow, I do exactly same steps as in the test but manually and 
it replays it just fine.

> Fix  org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test
> -
>
> Key: CASSANDRA-16720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16720
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.0-rc
>
> Attachments: fqllogs.tar.bz2
>
>
> It failed a few times in Jenkins lately. Example:
> [https://jenkins-cm4.apache.org/job/Cassandra-4.0.0/12/testReport/junit/org.apache.cassandra.distributed.test/FqlReplayDDLExclusionTest/test/]
> {code:java}
> Error Message
> Expected: [[1]] Actual: []
> Stacktrace
> junit.framework.AssertionFailedError: Expected: [[1]] Actual: [] at 
> org.apache.cassandra.distributed.shared.AssertUtils.assertRows(AssertUtils.java:58)
>  at 
> org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test(FqlReplayDDLExclusionTest.java:102)
> {code}
> I was also able to reproduce it locally on the latest 4.0.0 so I believe it 
> is worth it to be checked. 
> //CC [~stefan.miklosovic] and [~bereng] as I see you two are the only people 
> who worked on that test up to now so If you have any thoughts to share, as 
> usual, that will be highly appreciated :)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16720) Fix org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test

2021-06-08 Thread Stefan Miklosovic (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359547#comment-17359547
 ] 

Stefan Miklosovic commented on CASSANDRA-16720:
---

I am not sure I follow, I do exactly same steps as in the test but manually and 
it replays it just fine.

> Fix  org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test
> -
>
> Key: CASSANDRA-16720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16720
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.0-rc
>
> Attachments: fqllogs.tar.bz2
>
>
> It failed a few times in Jenkins lately. Example:
> [https://jenkins-cm4.apache.org/job/Cassandra-4.0.0/12/testReport/junit/org.apache.cassandra.distributed.test/FqlReplayDDLExclusionTest/test/]
> {code:java}
> Error Message
> Expected: [[1]] Actual: []
> Stacktrace
> junit.framework.AssertionFailedError: Expected: [[1]] Actual: [] at 
> org.apache.cassandra.distributed.shared.AssertUtils.assertRows(AssertUtils.java:58)
>  at 
> org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test(FqlReplayDDLExclusionTest.java:102)
> {code}
> I was also able to reproduce it locally on the latest 4.0.0 so I believe it 
> is worth it to be checked. 
> //CC [~stefan.miklosovic] and [~bereng] as I see you two are the only people 
> who worked on that test up to now so If you have any thoughts to share, as 
> usual, that will be highly appreciated :)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16725) Implement nodetool getauditlog command

2021-06-08 Thread Stefan Miklosovic (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359542#comment-17359542
 ] 

Stefan Miklosovic commented on CASSANDRA-16725:
---

Yes but I would really love to have this one in if you do not mind :)

> Implement nodetool getauditlog command
> --
>
> Key: CASSANDRA-16725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16725
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tool/auditlogging
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There is getfullquerylog already, there is not any reason why getauditlog 
> should not be there too. A user can not retrieve runtime configuration of 
> Audit log, it might be only enabled and disabled via jmx but its state can 
> not be queried in runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16725) Implement nodetool getauditlog command

2021-06-08 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-16725:
--
Reviewers: Vinay Chella  (was: Vinay Chella)

> Implement nodetool getauditlog command
> --
>
> Key: CASSANDRA-16725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16725
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tool/auditlogging
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There is getfullquerylog already, there is not any reason why getauditlog 
> should not be there too. A user can not retrieve runtime configuration of 
> Audit log, it might be only enabled and disabled via jmx but its state can 
> not be queried in runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16725) Implement nodetool getauditlog command

2021-06-08 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-16725:
--
Reviewers: Vinay Chella

> Implement nodetool getauditlog command
> --
>
> Key: CASSANDRA-16725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16725
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tool/auditlogging
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There is getfullquerylog already, there is not any reason why getauditlog 
> should not be there too. A user can not retrieve runtime configuration of 
> Audit log, it might be only enabled and disabled via jmx but its state can 
> not be queried in runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16725) Implement nodetool getauditlog command

2021-06-08 Thread Paulo Motta (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359540#comment-17359540
 ] 

Paulo Motta commented on CASSANDRA-16725:
-

Did you consider providing this functionality over a virtual table? While it 
may make sense to provide this via nodetool to be consistent with 
getfullquerylog I was wondering if we should not start moving away from JMX and 
towards virtual tables to provide new admin functionality.

> Implement nodetool getauditlog command
> --
>
> Key: CASSANDRA-16725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16725
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tool/auditlogging
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There is getfullquerylog already, there is not any reason why getauditlog 
> should not be there too. A user can not retrieve runtime configuration of 
> Audit log, it might be only enabled and disabled via jmx but its state can 
> not be queried in runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16725) Implement nodetool getauditlog command

2021-06-08 Thread Stefan Miklosovic (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359527#comment-17359527
 ] 

Stefan Miklosovic commented on CASSANDRA-16725:
---

PR  https://github.com/apache/cassandra/pull/1047

> Implement nodetool getauditlog command
> --
>
> Key: CASSANDRA-16725
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16725
> Project: Cassandra
>  Issue Type: New Feature
>  Components: Tool/auditlogging
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There is getfullquerylog already, there is not any reason why getauditlog 
> should not be there too. A user can not retrieve runtime configuration of 
> Audit log, it might be only enabled and disabled via jmx but its state can 
> not be queried in runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16725) Implement nodetool getauditlog command

2021-06-08 Thread Stefan Miklosovic (Jira)
Stefan Miklosovic created CASSANDRA-16725:
-

 Summary: Implement nodetool getauditlog command
 Key: CASSANDRA-16725
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16725
 Project: Cassandra
  Issue Type: New Feature
  Components: Tool/auditlogging
Reporter: Stefan Miklosovic
Assignee: Stefan Miklosovic


There is getfullquerylog already, there is not any reason why getauditlog 
should not be there too. A user can not retrieve runtime configuration of Audit 
log, it might be only enabled and disabled via jmx but its state can not be 
queried in runtime.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16720) Fix org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test

2021-06-08 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359466#comment-17359466
 ] 

Brandon Williams commented on CASSANDRA-16720:
--

I'll note though that I didn't see any of the "background~resource~releaser" 
issues.

> Fix  org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test
> -
>
> Key: CASSANDRA-16720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16720
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.0-rc
>
> Attachments: fqllogs.tar.bz2
>
>
> It failed a few times in Jenkins lately. Example:
> [https://jenkins-cm4.apache.org/job/Cassandra-4.0.0/12/testReport/junit/org.apache.cassandra.distributed.test/FqlReplayDDLExclusionTest/test/]
> {code:java}
> Error Message
> Expected: [[1]] Actual: []
> Stacktrace
> junit.framework.AssertionFailedError: Expected: [[1]] Actual: [] at 
> org.apache.cassandra.distributed.shared.AssertUtils.assertRows(AssertUtils.java:58)
>  at 
> org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test(FqlReplayDDLExclusionTest.java:102)
> {code}
> I was also able to reproduce it locally on the latest 4.0.0 so I believe it 
> is worth it to be checked. 
> //CC [~stefan.miklosovic] and [~bereng] as I see you two are the only people 
> who worked on that test up to now so If you have any thoughts to share, as 
> usual, that will be highly appreciated :)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16690:

Since Version:   (was: 3.0.23)

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 3.0.x, 3.11.x, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16720) Fix org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test

2021-06-08 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359464#comment-17359464
 ] 

Brandon Williams commented on CASSANDRA-16720:
--

This reproduces fairly frequently for me.  Near as I could tell, FQL was simply 
failing to replay any statements aside from DDL.  I modified to test to retain 
the FQL logs, and indeed the statements appear to be missing from a failed run 
- I've attached both here.  This seems to be a legitimate bug in FQL.

> Fix  org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test
> -
>
> Key: CASSANDRA-16720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16720
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.0-rc
>
> Attachments: fqllogs.tar.bz2
>
>
> It failed a few times in Jenkins lately. Example:
> [https://jenkins-cm4.apache.org/job/Cassandra-4.0.0/12/testReport/junit/org.apache.cassandra.distributed.test/FqlReplayDDLExclusionTest/test/]
> {code:java}
> Error Message
> Expected: [[1]] Actual: []
> Stacktrace
> junit.framework.AssertionFailedError: Expected: [[1]] Actual: [] at 
> org.apache.cassandra.distributed.shared.AssertUtils.assertRows(AssertUtils.java:58)
>  at 
> org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test(FqlReplayDDLExclusionTest.java:102)
> {code}
> I was also able to reproduce it locally on the latest 4.0.0 so I believe it 
> is worth it to be checked. 
> //CC [~stefan.miklosovic] and [~bereng] as I see you two are the only people 
> who worked on that test up to now so If you have any thoughts to share, as 
> usual, that will be highly appreciated :)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16720) Fix org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test

2021-06-08 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359464#comment-17359464
 ] 

Brandon Williams edited comment on CASSANDRA-16720 at 6/8/21, 4:47 PM:
---

This reproduces fairly frequently for me.  Near as I could tell, FQL was simply 
failing to replay any statements aside from DDL.  I modified the test to retain 
the FQL logs, and indeed the statements appear to be missing from a failed run 
- I've attached both here.  This seems to be a legitimate bug in FQL.


was (Author: brandon.williams):
This reproduces fairly frequently for me.  Near as I could tell, FQL was simply 
failing to replay any statements aside from DDL.  I modified to test to retain 
the FQL logs, and indeed the statements appear to be missing from a failed run 
- I've attached both here.  This seems to be a legitimate bug in FQL.

> Fix  org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test
> -
>
> Key: CASSANDRA-16720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16720
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.0-rc
>
> Attachments: fqllogs.tar.bz2
>
>
> It failed a few times in Jenkins lately. Example:
> [https://jenkins-cm4.apache.org/job/Cassandra-4.0.0/12/testReport/junit/org.apache.cassandra.distributed.test/FqlReplayDDLExclusionTest/test/]
> {code:java}
> Error Message
> Expected: [[1]] Actual: []
> Stacktrace
> junit.framework.AssertionFailedError: Expected: [[1]] Actual: [] at 
> org.apache.cassandra.distributed.shared.AssertUtils.assertRows(AssertUtils.java:58)
>  at 
> org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test(FqlReplayDDLExclusionTest.java:102)
> {code}
> I was also able to reproduce it locally on the latest 4.0.0 so I believe it 
> is worth it to be checked. 
> //CC [~stefan.miklosovic] and [~bereng] as I see you two are the only people 
> who worked on that test up to now so If you have any thoughts to share, as 
> usual, that will be highly appreciated :)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16690:

Status: Ready to Commit  (was: Review In Progress)

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16690:

  Fix Version/s: 3.11.x
 3.0.x
  Since Version: 3.0.23
Source Control Link: 
https://github.com/apache/cassandra/commit/a9f472c432fbe5c45662837f7d7ee578f59fd862
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 3.0.x, 3.11.x, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16720) Fix org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test

2021-06-08 Thread Brandon Williams (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16720:
-
Attachment: fqllogs.tar.bz2

> Fix  org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test
> -
>
> Key: CASSANDRA-16720
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16720
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.0-rc
>
> Attachments: fqllogs.tar.bz2
>
>
> It failed a few times in Jenkins lately. Example:
> [https://jenkins-cm4.apache.org/job/Cassandra-4.0.0/12/testReport/junit/org.apache.cassandra.distributed.test/FqlReplayDDLExclusionTest/test/]
> {code:java}
> Error Message
> Expected: [[1]] Actual: []
> Stacktrace
> junit.framework.AssertionFailedError: Expected: [[1]] Actual: [] at 
> org.apache.cassandra.distributed.shared.AssertUtils.assertRows(AssertUtils.java:58)
>  at 
> org.apache.cassandra.distributed.test.FqlReplayDDLExclusionTest.test(FqlReplayDDLExclusionTest.java:102)
> {code}
> I was also able to reproduce it locally on the latest 4.0.0 so I believe it 
> is worth it to be checked. 
> //CC [~stefan.miklosovic] and [~bereng] as I see you two are the only people 
> who worked on that test up to now so If you have any thoughts to share, as 
> usual, that will be highly appreciated :)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359463#comment-17359463
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16690:
-

Pushed to all branches after I ran successfully locally the test on all 
branches:

To https://github.com/apache/cassandra.git

   0e1f217079..a9f472c432  cassandra-3.0 -> cassandra-3.0

   dc8ccb7fb9..27f4cb68f2  cassandra-3.11 -> cassandra-3.11

   f83f2a3570..7d1c0131f6  cassandra-4.0 -> cassandra-4.0

   159469834a..82bd4567e2  cassandra-4.0.0 -> cassandra-4.0.0

   33ff36cc42..3b97e4bd7b  trunk -> trunk

 

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch cassandra-3.11 updated (dc8ccb7 -> 27f4cb6)

2021-06-08 Thread edimitrova
This is an automated email from the ASF dual-hosted git repository.

edimitrova pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from dc8ccb7  Merge branch cassandra-3.0 into cassandra-3.11
 add a9f472c  Fix flaky NativeAllocatorTest.testBookKeeping test patch by 
Ekaterina Dimitrova; reviewed by Andres de la Pena and Berenguer Blasi for 
CASSANDRA-16690
 add 27f4cb6  Merge branch 'cassandra-3.0' into cassandra-3.11

No new revisions were added by this update.

Summary of changes:
 test/unit/org/apache/cassandra/utils/memory/NativeAllocatorTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk

2021-06-08 Thread edimitrova
This is an automated email from the ASF dual-hosted git repository.

edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 3b97e4bd7bc2f5f14bdf82e80a1d2e83bb50add5
Merge: 33ff36c 7d1c013
Author: Ekaterina Dimitrova 
AuthorDate: Tue Jun 8 11:14:04 2021 -0400

Merge branch 'cassandra-4.0' into trunk

 test/unit/org/apache/cassandra/utils/memory/NativeAllocatorTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch trunk updated (33ff36c -> 3b97e4b)

2021-06-08 Thread edimitrova
This is an automated email from the ASF dual-hosted git repository.

edimitrova pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 33ff36c  Add a system property to set hostId if not yet initialized
 add a9f472c  Fix flaky NativeAllocatorTest.testBookKeeping test patch by 
Ekaterina Dimitrova; reviewed by Andres de la Pena and Berenguer Blasi for 
CASSANDRA-16690
 add 27f4cb6  Merge branch 'cassandra-3.0' into cassandra-3.11
 add 82bd456  Merge branch 'cassandra-3.11' into cassandra-4.0.0
 add 7d1c013  Merge branch 'cassandra-4.0.0' into cassandra-4.0
 new 3b97e4b  Merge branch 'cassandra-4.0' into trunk

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 test/unit/org/apache/cassandra/utils/memory/NativeAllocatorTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch cassandra-4.0 updated (f83f2a3 -> 7d1c013)

2021-06-08 Thread edimitrova
This is an automated email from the ASF dual-hosted git repository.

edimitrova pushed a change to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from f83f2a3  Merge branch 'cassandra-4.0.0' into cassandra-4.0
 add a9f472c  Fix flaky NativeAllocatorTest.testBookKeeping test patch by 
Ekaterina Dimitrova; reviewed by Andres de la Pena and Berenguer Blasi for 
CASSANDRA-16690
 add 27f4cb6  Merge branch 'cassandra-3.0' into cassandra-3.11
 add 82bd456  Merge branch 'cassandra-3.11' into cassandra-4.0.0
 add 7d1c013  Merge branch 'cassandra-4.0.0' into cassandra-4.0

No new revisions were added by this update.

Summary of changes:
 test/unit/org/apache/cassandra/utils/memory/NativeAllocatorTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch cassandra-4.0.0 updated (1594698 -> 82bd456)

2021-06-08 Thread edimitrova
This is an automated email from the ASF dual-hosted git repository.

edimitrova pushed a change to branch cassandra-4.0.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 1594698  Ignore SlidingTimeRateTest tests patch by Ekaterina 
Dimitrova, reviewed by Brandon Williams for CASSANDRA-16713
 add a9f472c  Fix flaky NativeAllocatorTest.testBookKeeping test patch by 
Ekaterina Dimitrova; reviewed by Andres de la Pena and Berenguer Blasi for 
CASSANDRA-16690
 add 27f4cb6  Merge branch 'cassandra-3.0' into cassandra-3.11
 add 82bd456  Merge branch 'cassandra-3.11' into cassandra-4.0.0

No new revisions were added by this update.

Summary of changes:
 test/unit/org/apache/cassandra/utils/memory/NativeAllocatorTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch cassandra-3.0 updated (0e1f217 -> a9f472c)

2021-06-08 Thread edimitrova
This is an automated email from the ASF dual-hosted git repository.

edimitrova pushed a change to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 0e1f217  Invalidate prepared statements on DROP COMPACT
 add a9f472c  Fix flaky NativeAllocatorTest.testBookKeeping test patch by 
Ekaterina Dimitrova; reviewed by Andres de la Pena and Berenguer Blasi for 
CASSANDRA-16690

No new revisions were added by this update.

Summary of changes:
 test/unit/org/apache/cassandra/utils/memory/NativeAllocatorTest.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16724) Skipping complex column elements in 3.0 can cause read-repair to break row isolation for super columns

2021-06-08 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-16724:
---
Description: CASSANDRA-10655 disabled value skipping of columns but did not 
disabled value skipping of complex columns elements which can also lead to 
read-repair issues for super columns.  (was: CASSANDRA-10655 disabled value 
skipping of columns but did not disabled value skipping of complex columns 
elements which can also lead to read-repair issues.
In 3.0, query filtering complex column element values can only be performed 
throught Thrift.)

> Skipping complex column elements in 3.0 can cause read-repair to break row 
> isolation for super columns
> --
>
> Key: CASSANDRA-16724
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16724
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
>
> CASSANDRA-10655 disabled value skipping of columns but did not disabled value 
> skipping of complex columns elements which can also lead to read-repair 
> issues for super columns.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16724) Skipping complex column elements in 3.0 can cause read-repair to break row isolation for super columns

2021-06-08 Thread Benjamin Lerer (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Benjamin Lerer updated CASSANDRA-16724:
---
Summary: Skipping complex column elements in 3.0 can cause read-repair to 
break row isolation for super columns  (was: Skipping complex column elements 
in 3.0 can cause read-repair to break row isolation)

> Skipping complex column elements in 3.0 can cause read-repair to break row 
> isolation for super columns
> --
>
> Key: CASSANDRA-16724
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16724
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
>
> CASSANDRA-10655 disabled value skipping of columns but did not disabled value 
> skipping of complex columns elements which can also lead to read-repair 
> issues.
> In 3.0, query filtering complex column element values can only be performed 
> throught Thrift.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16573) CQLSH copy defaults appear to be incorrect on website

2021-06-08 Thread Brian Houser (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359414#comment-17359414
 ] 

Brian Houser commented on CASSANDRA-16573:
--

In running this through testing, I discovered a separate issue.  There is a 
license agreement comment found in the layout.html of the theme.  
[https://github.com/apache/cassandra/blob/cassandra-3.11/doc/source/_theme/cassandra_theme/layout.html]

Jekyll assumes that the front matter while be at the top "—".  This causes 
Jekyll to break when serving the page.

> CQLSH copy defaults appear to be incorrect on website
> -
>
> Key: CASSANDRA-16573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16573
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Brian Houser
>Assignee: Brian Houser
>Priority: Normal
>
> The documentation on the website for the defaults of CQLSH appear to be 
> incorrect and contain numerous errors (at least for the latest and greatest)
> For this page:
> [https://cassandra.apache.org/doc/latest/tools/cqlsh.html]
> {{MINBATCHSIZE}} is listed as defaulting to 2.  Code says this is 10.
> https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/copyutil.py#L355
> Chunksize says 1000, actually set to 5000.
> https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/copyutil.py#L352
> NumProcessis is also off...
> "NUMPROCESSES
>  The number of child worker processes to create for COPY tasks. Defaults to a 
> max of 4 for COPY FROM and 16 for COPY TO. However, at most (num_cores - 1) 
> processes will be created."
>  Default is the number of cores -1 or 16 which ever is smaller, and you can 
> set this value to anything.  See the following code
> https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/copyutil.py#L361
> https://github.com/apache/cassandra/blob/trunk/pylib/cqlshlib/copyutil.py#L407
>  
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Created] (CASSANDRA-16724) Skipping complex column elements in 3.0 can cause read-repair to break row isolation

2021-06-08 Thread Benjamin Lerer (Jira)
Benjamin Lerer created CASSANDRA-16724:
--

 Summary: Skipping complex column elements in 3.0 can cause 
read-repair to break row isolation
 Key: CASSANDRA-16724
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16724
 Project: Cassandra
  Issue Type: Bug
Reporter: Benjamin Lerer
Assignee: Benjamin Lerer


CASSANDRA-10655 disabled value skipping of columns but did not disabled value 
skipping of complex columns elements which can also lead to read-repair issues.
In 3.0, query filtering complex column element values can only be performed 
throught Thrift.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-9384) Update jBCrypt dependency to version 0.4

2021-06-08 Thread Brandon Williams (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359381#comment-17359381
 ] 

Brandon Williams commented on CASSANDRA-9384:
-

Looks like this ticket may have been forgotten.

> Update jBCrypt dependency to version 0.4
> 
>
> Key: CASSANDRA-9384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9384
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> https://bugzilla.mindrot.org/show_bug.cgi?id=2097
> Although the bug tracker lists it as NEW/OPEN, the release notes for 0.4 
> indicate that this is now fixed, so we should update.
> Thanks to [~Bereng] for identifying the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-9384) Update jBCrypt dependency to version 0.4

2021-06-08 Thread Brandon Williams (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-9384:

Status: Patch Available  (was: Ready to Commit)

> Update jBCrypt dependency to version 0.4
> 
>
> Key: CASSANDRA-9384
> URL: https://issues.apache.org/jira/browse/CASSANDRA-9384
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Sam Tunnicliffe
>Assignee: Dinesh Joshi
>Priority: Normal
> Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> https://bugzilla.mindrot.org/show_bug.cgi?id=2097
> Although the bug tracker lists it as NEW/OPEN, the release notes for 0.4 
> indicate that this is now fixed, so we should update.
> Thanks to [~Bereng] for identifying the issue.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359373#comment-17359373
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16690:
-

Thank you both, I will apply the patch to 3,3.11 and 4 with the suggested 
raised timeout of 10 seconds.

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16711) Split ViewComplexTest to avoid timeouts after being moved away from 'long' junits

2021-06-08 Thread Ekaterina Dimitrova (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16711:

Status: Ready to Commit  (was: Review In Progress)

> Split ViewComplexTest to avoid timeouts after being moved away from 'long' 
> junits
> -
>
> Key: CASSANDRA-16711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16711
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Under CASSANDRA-16670 we decided to move some tickets from the 'long' junit 
> section to the std one. This required splitting them to avoid timing out but 
> ViewComplexTest needs further splitting.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16711) Split ViewComplexTest to avoid timeouts after being moved away from 'long' junits

2021-06-08 Thread Ekaterina Dimitrova (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359354#comment-17359354
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16711:
-

Sorry, I wasn't clear, +1 to commit and by " if *it* continues to timeout here 
or there for the same reasons, let's leave *it* away for a bit" I meant to 
leave *the problem* post-commit for a bit if we see again timeouts. Thank you

> Split ViewComplexTest to avoid timeouts after being moved away from 'long' 
> junits
> -
>
> Key: CASSANDRA-16711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16711
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Under CASSANDRA-16670 we decided to move some tickets from the 'long' junit 
> section to the std one. This required splitting them to avoid timing out but 
> ViewComplexTest needs further splitting.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies

2021-06-08 Thread Michael Semb Wever (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359317#comment-17359317
 ] 

Michael Semb Wever commented on CASSANDRA-16704:


bq. Is it intentional that the scope for some entries is repeated across the 
two pom definitions, while missing for others?

Not in general. Definitely jsr305 and compile-command-annotations looks like 
they can have the duplicated scope removed.  I think the others are just 
explicit because they are overridden, but yes: duplicated overrides are a bit 
silly. byteman-* for example could be defined as provided, and then overridden 
to test in build-deps-pom. I can't remember how optional interferes with the 
scope (ref hadoop* deps).

> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: test-with-runtime-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars. 
> The tests would be more reflective of the behavior of a runtime environment 
> if the test classpath only contained jars that are bundled with the binary 
> release, together with explicit test dependencies. Ideally we'd use the 
> build/lib/ jars for the classpath since that's what gets packaged, but since 
> these aren't available at test compile time and should be identical to lib/ 
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to 
> "org.apache.commons.lang", which is not available at runtime (should be 
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both 
> test/ and src/ classes, and makes some simple substitutions in the tests such 
> as using AbstractMap.SimpleEntry in place of 
> org.apache.commons.collections.keyvalue.AbstractMapEntry.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16722) ClientMetrics should be initialised in CQLConnectionTest

2021-06-08 Thread Michael Semb Wever (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16722:
---
Status: Ready to Commit  (was: Review In Progress)

+1

> ClientMetrics should be initialised in CQLConnectionTest
> 
>
> Key: CASSANDRA-16722
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16722
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Without a call to {{ClientMetrics.instance.init}}, client protocol errors 
> cause an NPE when the server attempts to mark the relevant meter.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16716) Flaky TestConsistency.test_13880

2021-06-08 Thread Jira


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359280#comment-17359280
 ] 

Andres de la Peña commented on CASSANDRA-16716:
---

Yes, I wanted to try some more runs just in case, and also test other branches 
for completion.

> Flaky TestConsistency.test_13880
> 
>
> Key: CASSANDRA-16716
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16716
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.0.x, 4.x
>
>
> Flaky 
> [TestConsistency.test_13880|https://ci-cassandra.apache.org/job/Cassandra-4.0.0/8/testReport/junit/dtest.consistency_test/TestConsistency/test_13880/]
> {noformat}
> Error Message
> cassandra.Unavailable: Error from server: code=1000 [Unavailable exception] 
> message="Cannot achieve consistency level ALL" info={'consistency': 'ALL', 
> 'required_replicas': 2, 'alive_replicas': 1}
> Stacktrace
> self = 
> @since('3.0')
> @ported_to_in_jvm('4.0')
> def test_13880(self):
> """
> @jira_ticket CASSANDRA-13880
> """
> cluster = self.cluster
> 
> # disable hinted handoff and set batch commit log so this doesn't 
> interfere with the test
> cluster.set_configuration_options(values={'hinted_handoff_enabled': 
> False})
> cluster.set_batch_commitlog(enabled=True)
> 
> cluster.populate(2).start()
> node1, node2 = cluster.nodelist()
> 
> session = self.patient_cql_connection(node1)
> 
> query = "CREATE KEYSPACE IF NOT EXISTS test WITH replication = 
> {'class': 'NetworkTopologyStrategy', 'datacenter1': 2};"
> session.execute(query)
> 
> query = "CREATE TABLE IF NOT EXISTS test.test (id int PRIMARY KEY);"
> session.execute(query)
> 
> stmt = SimpleStatement("INSERT INTO test.test (id) VALUES (0);",
>consistency_level=ConsistencyLevel.ALL)
> session.execute(stmt)
> 
> # with node2 down and hints disabled, delete the partition on node1
> node2.stop(wait_other_notice=True)
> session.execute("DELETE FROM test.test WHERE id = 0;")
> node2.start()
> 
> # with both nodes up, do a CL.ALL query with per partition limit of 1;
> # prior to CASSANDRA-13880 this would cause short read protection to 
> loop forever
> >   assert_none(session, "SELECT DISTINCT id FROM test.test WHERE id = 
> > 0;", cl=ConsistencyLevel.ALL)
> consistency_test.py:1170: 
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> tools/assertions.py:147: in assert_none
> res = session.execute(simple_query)
> ../venv/src/cassandra-driver/cassandra/cluster.py:2618: in execute
> return self.execute_async(query, parameters, trace, custom_payload, 
> timeout, execution_profile, paging_state, host, execute_as).result()
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
> _ 
> self =  info={'consistency': 'ALL', 'required_replicas': 2, 'alive_replicas': 1} 
> coordinator_host=127.0.0.1:9042>
> def result(self):
> """
> Return the final result or raise an Exception if errors were
> encountered.  If the final result or error has not been set
> yet, this method will block until it is set, or the timeout
> set for the request expires.
> 
> Timeout is specified in the Session request execution functions.
> If the timeout is exceeded, an :exc:`cassandra.OperationTimedOut` 
> will be raised.
> This is a client-side timeout. For more information
> about server-side coordinator timeouts, see 
> :class:`.policies.RetryPolicy`.
> 
> Example usage::
> 
> >>> future = session.execute_async("SELECT * FROM mycf")
> >>> # do other stuff...
> 
> >>> try:
> ... rows = future.result()
> ... for row in rows:
> ... ... # process results
> ... except Exception:
> ... log.exception("Operation failed:")
> 
> """
> self._event.wait()
> if self._final_result is not _NOT_SET:
> return ResultSet(self, self._final_result)
> else:
> >   raise self._final_exception
> E   cassandra.Unavailable: Error from server: code=1000 [Unavailable 
> exception] message="Cannot achieve consistency level ALL" 
> info={'consistency': 'ALL', 'required_replicas': 2, 'alive_replicas': 1}
> 

[jira] [Updated] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Jira


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres de la Peña updated CASSANDRA-16690:
--
Reviewers: Andres de la Peña, Berenguer Blasi, Andres de la Peña  (was: 
Andres de la Peña, Berenguer Blasi)
   Andres de la Peña, Berenguer Blasi, Andres de la Peña  (was: 
Andres de la Peña, Berenguer Blasi)
   Status: Review In Progress  (was: Patch Available)

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Jira


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359245#comment-17359245
 ] 

Andres de la Peña commented on CASSANDRA-16690:
---

Agree, raising the timeout and adding the assertion looks good to me. I would 
also raise the timeout, perhaps to even longer than 500ms since we have seen 
longer pauses in CircleCI, maybe something like 5-10 seconds to be on the safe 
side. I think that the patch should also be applied to 3.0 and 3.11, since 
those branches also have the 10ms timeout.

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359176#comment-17359176
 ] 

Berenguer Blasi edited comment on CASSANDRA-16690 at 6/8/21, 9:17 AM:
--

It's easy to repro adding a Thread.sleep() before {{isClean.countDown()}} in 
the cleaner code. Adding the assertion, to check for a successful clean, and 
raising the wait timeout is the way to go +1. But I would raise it to 500ms or 
so, then we won't have to fight this again the day we timeout on 20ms. At the 
end of the day that timeout being high has no semantics.


was (Author: bereng):
It's easy to repro adding a Thread.sleep() before {{isClean.countDown()}} in 
the cleaner code. Adding the assertion to check for a successful clean, and 
raising the wait timeout is the way to go +1. But I would raise it to 500ms or 
so, then we won't have to fight this again the day we timeout on 20ms. At the 
end of the day that timeout being high has no semantics.

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Berenguer Blasi (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16690:

Reviewers: Andres de la Peña, Berenguer Blasi  (was: Andres de la Peña)

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16690) Flaky NativeAllocatorTest.testBookKeeping

2021-06-08 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359176#comment-17359176
 ] 

Berenguer Blasi commented on CASSANDRA-16690:
-

It's easy to repro adding a Thread.sleep() before {{isClean.countDown()}} in 
the cleaner code. Adding the assertion to check for a successful clean, and 
raising the wait timeout is the way to go +1. But I would raise it to 500ms or 
so, then we won't have to fight this again the day we timeout on 20ms. At the 
end of the day that timeout being high has no semantics.

> Flaky NativeAllocatorTest.testBookKeeping
> -
>
> Key: CASSANDRA-16690
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16690
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Flaky 
> [here|https://ci-cassandra.apache.org/job/Cassandra-4.0/52/testReport/junit/org.apache.cassandra.utils.memory/NativeAllocatorTest/testBookKeeping_cdc/]
> {noformat}
> Error Message
> java.lang.AssertionError: expected:<0> but was:<1>
> Stacktrace
> java.util.concurrent.ExecutionException: java.lang.AssertionError: 
> expected:<0> but was:<1>
>   at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.testBookKeeping(NativeAllocatorTest.java:154)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Caused by: java.lang.AssertionError: expected:<0> but was:<1>
>   at 
> org.apache.cassandra.utils.memory.NativeAllocatorTest.lambda$testBookKeeping$2(NativeAllocatorTest.java:131)
>   at 
> java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
>   at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
>   at 
> java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
>   at 
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
>   at java.base/java.lang.Thread.run(Thread.java:834)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14582) Add a system property to set the cassandra hostId if not yet initialized

2021-06-08 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-14582:
--
Test and Documentation Plan: dtest: 
https://github.com/apache/cassandra-dtest/commit/148f57ef8051adfa15d1fefff5edff2709f0c75b
  (was: unit tests)

> Add a system property to set the cassandra hostId if not yet initialized
> 
>
> Key: CASSANDRA-14582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14582
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: vincent royer
>Assignee: Abuli Palagashvili
>Priority: Low
>  Labels: lhf
> Fix For: 4.1
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Add a system property *cassandra.host_id* to set the cassandra hostId if not 
> yet initialized.
> This allow to push the cassandra host ID when provisioning new cassandra 
> nodes rather than to retreive it after the first start.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14582) Add a system property to set the cassandra hostId if not yet initialized

2021-06-08 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-14582:
--
Source Control Link: 
https://github.com/apache/cassandra/commit/33ff36cc42a083311065dee56adc1d72d6866f2d
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> Add a system property to set the cassandra hostId if not yet initialized
> 
>
> Key: CASSANDRA-14582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14582
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: vincent royer
>Assignee: Abuli Palagashvili
>Priority: Low
>  Labels: lhf
> Fix For: 4.1
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Add a system property *cassandra.host_id* to set the cassandra hostId if not 
> yet initialized.
> This allow to push the cassandra host ID when provisioning new cassandra 
> nodes rather than to retreive it after the first start.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-14582) Add a system property to set the cassandra hostId if not yet initialized

2021-06-08 Thread Stefan Miklosovic (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359156#comment-17359156
 ] 

Stefan Miklosovic commented on CASSANDRA-14582:
---

merged based on this build 
[https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/847/]

dtest: 
https://github.com/apache/cassandra-dtest/commit/148f57ef8051adfa15d1fefff5edff2709f0c75b

cassandra: 
https://github.com/apache/cassandra/commit/33ff36cc42a083311065dee56adc1d72d6866f2d

> Add a system property to set the cassandra hostId if not yet initialized
> 
>
> Key: CASSANDRA-14582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14582
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: vincent royer
>Assignee: Abuli Palagashvili
>Priority: Low
>  Labels: lhf
> Fix For: 4.1
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Add a system property *cassandra.host_id* to set the cassandra hostId if not 
> yet initialized.
> This allow to push the cassandra host ID when provisioning new cassandra 
> nodes rather than to retreive it after the first start.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra] branch trunk updated: Add a system property to set hostId if not yet initialized

2021-06-08 Thread smiklosovic
This is an automated email from the ASF dual-hosted git repository.

smiklosovic pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 33ff36c  Add a system property to set hostId if not yet initialized
33ff36c is described below

commit 33ff36cc42a083311065dee56adc1d72d6866f2d
Author: fibersel <0583...@gmail.com>
AuthorDate: Wed May 26 23:04:57 2021 +0300

Add a system property to set hostId if not yet initialized

patch by Abi Palagashvili; reviewed by Stefan Miklosovic and Paulo Motta 
for CASSANDRA-14582
---
 CHANGES.txt  | 1 +
 conf/jvm-server.options  | 3 +++
 src/java/org/apache/cassandra/db/SystemKeyspace.java | 6 --
 3 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index 185ec72..71018a6 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.1
+ * Add a system property to set hostId if not yet initialized (CASSANDRA-14582)
  * GossiperTest.testHasVersion3Nodes didn't take into account trunk version 
changes, fixed to rely on latest version (CASSANDRA-16651)
 Merged from 4.0:
  * Make JmxHistogram#getRecentValues() and JmxTimer#getRecentValues() 
thread-safe (CASSANDRA-16707)
diff --git a/conf/jvm-server.options b/conf/jvm-server.options
index 46967f4..f7845d4 100644
--- a/conf/jvm-server.options
+++ b/conf/jvm-server.options
@@ -22,6 +22,9 @@
 # and perhaps have affinity.
 #-Dcassandra.available_processors=number_of_processors
 
+# Specify host id for the first boot and persist it
+#-Dcassandra.host_id_first_boot=host_id
+
 # The directory location of the cassandra.yaml file.
 #-Dcassandra.config=directory
 
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java 
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index cbb7084..4d2a567 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -1126,13 +1126,15 @@ public final class SystemKeyspace
 return hostId;
 
 // ID not found, generate a new one, persist, and then return it.
-hostId = UUID.randomUUID();
+String hostString = System.getProperty("cassandra.host_id_first_boot", 
UUID.randomUUID().toString());
+hostId = UUID.fromString(hostString);
 logger.warn("No host ID found, created {} (Note: This should happen 
exactly once per node).", hostId);
 return setLocalHostId(hostId);
 }
 
 /**
- * Sets the local host ID explicitly.  Should only be called outside of 
SystemTable when replacing a node.
+ * Sets the local host ID explicitly. Should only be called outside of 
SystemTable when replacing a node.
+ * Used also in CASSANDRA-14582.
  */
 public static synchronized UUID setLocalHostId(UUID hostId)
 {

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Updated] (CASSANDRA-14582) Add a system property to set the cassandra hostId if not yet initialized

2021-06-08 Thread Stefan Miklosovic (Jira)


 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-14582:
--
Status: Ready to Commit  (was: Changes Suggested)

The issues spotted were fixed.

> Add a system property to set the cassandra hostId if not yet initialized
> 
>
> Key: CASSANDRA-14582
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14582
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: vincent royer
>Assignee: Abuli Palagashvili
>Priority: Low
>  Labels: lhf
> Fix For: 4.1
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> Add a system property *cassandra.host_id* to set the cassandra hostId if not 
> yet initialized.
> This allow to push the cassandra host ID when provisioning new cassandra 
> nodes rather than to retreive it after the first start.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[cassandra-dtest] branch trunk updated: dtest for setting hostId upon node startup

2021-06-08 Thread smiklosovic
This is an automated email from the ASF dual-hosted git repository.

smiklosovic pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 148f57e  dtest for setting hostId upon node startup
148f57e is described below

commit 148f57ef8051adfa15d1fefff5edff2709f0c75b
Author: fibersel <0583...@gmail.com>
AuthorDate: Thu May 27 02:49:56 2021 +0300

dtest for setting hostId upon node startup

patch by Abi Palagashvili; reviewed by Stefan Miklosovic and Paulo Motta 
for CASSANDRA-14582
---
 bootstrap_test.py | 81 +++
 1 file changed, 81 insertions(+)

diff --git a/bootstrap_test.py b/bootstrap_test.py
index 9739a63..dc06917 100644
--- a/bootstrap_test.py
+++ b/bootstrap_test.py
@@ -7,6 +7,7 @@ import threading
 import time
 import logging
 import signal
+import uuid
 
 from cassandra import ConsistencyLevel
 from cassandra.concurrent import execute_concurrent_with_args
@@ -1016,3 +1017,83 @@ class TestBootstrap(Tester):
 self.assert_log_had_msg(node3, "Leaving write survey mode and joining 
ring at operator request")
 assert_bootstrap_state(self, node3, 'COMPLETED', user='cassandra', 
password='cassandra')
 node3.wait_for_binary_interface()
+
+@since('4.1')
+def test_invalid_host_id(self):
+"""
+@jira_ticket CASSANDRA-14582
+Test that node fails to bootstrap if host id is invalid
+"""
+cluster = self.cluster
+
cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', 
'True')
+cluster.populate(1)
+cluster.start()
+
+node2 = new_node(cluster)
+
+try:
+
node2.start(jvm_args=["-Dcassandra.host_id_first_boot=invalid-host-id"], 
wait_other_notice=False, wait_for_binary_proto=True)
+pytest.fail('Node should fail to bootstrap because host id set was 
invalid')
+except NodeError:
+pass # node does not start as expected
+
+@since('4.1')
+def test_host_id_override(self):
+"""
+@jira_ticket CASSANDRA-14582
+Test that node persists host id
+"""
+cluster = self.cluster
+
cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED', 
'True')
+cluster.populate(1)
+cluster.start()
+
+host_id = "06fc931f-33b5-4e22-0001-0001"
+
+node1 = cluster.nodes['node1']
+
+node2 = new_node(cluster)
+node2.start(wait_for_binary_proto=True, wait_other_notice=True, 
jvm_args=["-Dcassandra.host_id_first_boot={}".format(host_id)])
+
+address2 = "'{}'".format(node2.address())
+
+# 1. wait for host_id setup
+node2.watch_log_for(host_id)
+
+# 2. check host_id in local table
+session2 = self.patient_exclusive_cql_connection(node2)
+assert_one(session2, "SELECT host_id FROM system.local", 
[uuid.UUID(host_id)])
+
+# 3. check host_id in other node's table
+session1 = self.patient_exclusive_cql_connection(node1)
+assert_one(session1, "SELECT host_id FROM system.peers_v2 WHERE peer = 
{}".format(address2), [uuid.UUID(host_id)])
+
+# restart node and repeat
+node2.stop()
+node2.start(wait_for_binary_proto=True, wait_other_notice=True)
+
+# 1. wait for host_id setup
+node2.watch_log_for(host_id)
+
+# 2. check host_id in local table
+session2 = self.patient_exclusive_cql_connection(node2)
+assert_one(session2, "SELECT host_id FROM system.local", 
[uuid.UUID(host_id)])
+
+# 3. check host_id in other node's table
+session1 = self.patient_exclusive_cql_connection(node1)
+assert_one(session1, "SELECT host_id FROM system.peers_v2 WHERE peer = 
{}".format(address2), [uuid.UUID(host_id)])
+
+# restart node with another host_id and repeat
+node2.stop()
+node2.start(wait_for_binary_proto=True, wait_other_notice=True, 
jvm_args=["-Dcassandra.host_id_first_boot=setting-new-host-id-first-boot"])
+
+# 1. wait for host_id setup
+node2.watch_log_for(host_id)
+
+# 2. check host_id in local table
+session2 = self.patient_exclusive_cql_connection(node2)
+assert_one(session2, "SELECT host_id FROM system.local", 
[uuid.UUID(host_id)])
+
+# 3. check host_id in other node's table
+session1 = self.patient_exclusive_cql_connection(node1)
+assert_one(session1, "SELECT host_id FROM system.peers_v2 WHERE peer = 
{}".format(address2), [uuid.UUID(host_id)])

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16704) Fix imports; run tests with packaged dependencies

2021-06-08 Thread Angelo Polo (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359125#comment-17359125
 ] 

Angelo Polo commented on CASSANDRA-16704:
-

{quote}They are in places, see all those marked as "provided" or "optional" 
[here|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/build.xml#L491-L657]
 and 
[here|https://github.com/apache/cassandra/blob/cassandra-4.0-rc1/build.xml#L749-L833].
{quote}
Is it intentional that the scope for some entries is repeated across the two 
pom definitions, while missing for others? Below is for 'provided' scope, 
similar situation for 'test' scope. Those shown in blue are declared with scope 
in only one place, those in green are given a scope in both places:

parent-pom:
{color:#4c9aff}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#4c9aff}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}


all-pom:
{color:#4c9aff}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#57d9a3}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}
{color:#4c9aff}{color}

 

> Fix imports; run tests with packaged dependencies
> -
>
> Key: CASSANDRA-16704
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16704
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build, Test/burn, Test/unit
>Reporter: Angelo Polo
>Assignee: Angelo Polo
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: test-with-runtime-deps.patch
>
>
> Tests are currently run with a classpath containing _all_ downloaded jars. 
> The tests would be more reflective of the behavior of a runtime environment 
> if the test classpath only contained jars that are bundled with the binary 
> release, together with explicit test dependencies. Ideally we'd use the 
> build/lib/ jars for the classpath since that's what gets packaged, but since 
> these aren't available at test compile time and should be identical to lib/ 
> anyway, I've used the later.
> Doing so exposed a couple of references in src/java to 
> "org.apache.commons.lang", which is not available at runtime (should be 
> "org.apache.commons.lang*3*").
> Attached patch modifies the test classpath, fixes various imports in both 
> test/ and src/ classes, and makes some simple substitutions in the tests such 
> as using AbstractMap.SimpleEntry in place of 
> org.apache.commons.collections.keyvalue.AbstractMapEntry.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Comment Edited] (CASSANDRA-16689) Flaky LeaveAndBootstrapTest

2021-06-08 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359078#comment-17359078
 ] 

Berenguer Blasi edited comment on CASSANDRA-16689 at 6/8/21, 6:54 AM:
--

You're looking at the wrong code. If you check the SHA of the test failure 
you'll see the NPE comes from [pulling endpoint 
states|https://github.com/bereng/cassandra/blob/f7cd1c1f2508a9faf36c9ac2023f65841bb4628d/src/java/org/apache/cassandra/service/StorageService.java#L2418]
 not from config loading iiuc #collaborating


was (Author: bereng):
You're looking at the wrong code. If you check the SHA of the test failure 
you'll see the NPE comes from [pulling endpoint 
states|https://github.com/bereng/cassandra/blob/f7cd1c1f2508a9faf36c9ac2023f65841bb4628d/src/java/org/apache/cassandra/service/StorageService.java]
 not from config loading iiuc #collaborating

> Flaky LeaveAndBootstrapTest
> ---
>
> Key: CASSANDRA-16689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16689
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Failing in a circle run 
> [here|https://app.circleci.com/pipelines/github/bereng/cassandra/309/workflows/a645b956-dcd7-431e-b109-7857af3c523f/jobs/2937]
> {noformat}
> Testcase: 
> testStateJumpToNormal(org.apache.cassandra.service.LeaveAndBootstrapTest):  
> Caused an ERROR
> [junit-timeout] null
> [junit-timeout] java.lang.NullPointerException
> [junit-timeout]   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2418)
> [junit-timeout]   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2756)
> [junit-timeout]   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2299)
> [junit-timeout]   at 
> org.apache.cassandra.Util.createInitialRing(Util.java:236)
> [junit-timeout]   at 
> org.apache.cassandra.service.LeaveAndBootstrapTest.testStateJumpToNormal(LeaveAndBootstrapTest.java:550)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org



[jira] [Commented] (CASSANDRA-16689) Flaky LeaveAndBootstrapTest

2021-06-08 Thread Berenguer Blasi (Jira)


[ 
https://issues.apache.org/jira/browse/CASSANDRA-16689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17359078#comment-17359078
 ] 

Berenguer Blasi commented on CASSANDRA-16689:
-

You're looking at the wrong code. If you check the SHA of the test failure 
you'll see the NPE comes from [pulling endpoint 
states|https://github.com/bereng/cassandra/blob/f7cd1c1f2508a9faf36c9ac2023f65841bb4628d/src/java/org/apache/cassandra/service/StorageService.java]
 not from config loading iiuc #collaborating

> Flaky LeaveAndBootstrapTest
> ---
>
> Key: CASSANDRA-16689
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16689
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc2, 4.0, 4.x
>
>
> Failing in a circle run 
> [here|https://app.circleci.com/pipelines/github/bereng/cassandra/309/workflows/a645b956-dcd7-431e-b109-7857af3c523f/jobs/2937]
> {noformat}
> Testcase: 
> testStateJumpToNormal(org.apache.cassandra.service.LeaveAndBootstrapTest):  
> Caused an ERROR
> [junit-timeout] null
> [junit-timeout] java.lang.NullPointerException
> [junit-timeout]   at 
> org.apache.cassandra.service.StorageService.updatePeerInfo(StorageService.java:2418)
> [junit-timeout]   at 
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:2756)
> [junit-timeout]   at 
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2299)
> [junit-timeout]   at 
> org.apache.cassandra.Util.createInitialRing(Util.java:236)
> [junit-timeout]   at 
> org.apache.cassandra.service.LeaveAndBootstrapTest.testStateJumpToNormal(LeaveAndBootstrapTest.java:550)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org