[jira] [Commented] (CASSANDRA-19393) nodetool: group CMS-related commands into one command
[ https://issues.apache.org/jira/browse/CASSANDRA-19393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827380#comment-17827380 ] n.v.harikrishna commented on CASSANDRA-19393: - Thanks [~smiklosovic] ! And apologies for the slow trun around. > nodetool: group CMS-related commands into one command > - > > Key: CASSANDRA-19393 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19393 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool, Transactional Cluster Metadata >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Fix For: 5.1 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The purpose of this ticket is to group all CMS-related commands under one > "nodetool cms" command where existing command would be subcommands of it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19473) Latency Spike on NTR startup
Cameron Zemek created CASSANDRA-19473:
-
Summary: Latency Spike on NTR startup
Key: CASSANDRA-19473
URL: https://issues.apache.org/jira/browse/CASSANDRA-19473
Project: Cassandra
Issue Type: Improvement
Reporter: Cameron Zemek
Firstly you need the patch from
https://issues.apache.org/jira/browse/CASSANDRA-18845 to solve consistency
query errors on startup. With that patch there is still a further issue we see
on some clusters where the latency spikes too high when initially starting. I
see pending compactions and hints metrics increased during this time.
I tried lowering the hint delivery threshold across the cluster thinking it was
overloading the node starting up, but this didn't resolve the issue. So at this
time I am not sure what the root cause (I still think its combination of the
compactions and hints).
As workaround I have this small code change:
{code:java}
int START_NATIVE_DELAY =
Integer.getInteger("cassandra.start_native_transport_delay_secs", 120);
if (START_NATIVE_DELAY > 0)
{
logger.info("Waiting an extra {} seconds before enabling NTR",
START_NATIVE_DELAY);
Uninterruptibles.sleepUninterruptibly(START_NATIVE_DELAY,
TimeUnit.SECONDS);
}
startNativeTransport();
{code}
Where wait an configurable time before starting native transport. Delaying NTR
startup resolved the issue.
A better solution would be to wait for hints/compactions or whatever is root
cause to complete.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-3057 Allow decoding a UDT that has more fields than expected [cassandra-java-driver]
akhaku commented on code in PR #1635:
URL:
https://github.com/apache/cassandra-java-driver/pull/1635#discussion_r1525623390
##
core/src/main/java/com/datastax/oss/driver/internal/core/type/codec/UdtCodec.java:
##
@@ -105,10 +105,7 @@ public UdtValue decode(@Nullable ByteBuffer bytes,
@NonNull ProtocolVersion prot
int i = 0;
while (input.hasRemaining()) {
if (i == cqlType.getFieldTypes().size()) {
- throw new IllegalArgumentException(
- String.format(
- "Too many fields in encoded UDT value, expected %d",
- cqlType.getFieldTypes().size()));
+ break;
Review Comment:
It's been a while but my memory is a little fuzzy as to the exact scenario
with which we ran into this, but it had to do with the object mapper caching
schemas somewhere too. In any case, the client metadata refresh hint sounds
reasonable for when we tackle what you said earlier about EVENT delivery
failing, since that would be the logical place for the hint to propagate and
affect. For now I'll add a debug log.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19468 Don't swallow exception during metadata refresh [cassandra-java-driver]
akhaku commented on code in PR #1920:
URL:
https://github.com/apache/cassandra-java-driver/pull/1920#discussion_r1525614078
##
core/src/test/java/com/datastax/oss/driver/internal/core/metadata/MetadataManagerTest.java:
##
@@ -286,6 +290,25 @@ public void should_remove_node() {
assertThat(refresh.broadcastRpcAddressToRemove).isEqualTo(broadcastRpcAddress2);
}
+ @Test
+ public void refreshSchema_should_work() {
+// Given
+IllegalStateException expectedException = new IllegalStateException("Error
we're testing");
+when(schemaQueriesFactory.newInstance()).thenThrow(expectedException);
+
when(topologyMonitor.refreshNodeList()).thenReturn(CompletableFuture.completedFuture(ImmutableList.of(mock(NodeInfo.class;
+
when(topologyMonitor.checkSchemaAgreement()).thenReturn(CompletableFuture.completedFuture(Boolean.TRUE));
+when(controlConnection.init(anyBoolean(), anyBoolean(),
anyBoolean())).thenReturn(CompletableFuture.completedFuture(null));
+metadataManager.refreshNodes(); // required internal state setup for this
+waitForPendingAdminTasks(() -> metadataManager.refreshes.size() == 1); //
sanity check
+
+// When
+CompletionStage result =
metadataManager.refreshSchema("foo", true, true);
+
+// Then
+waitForPendingAdminTasks(() -> result.toCompletableFuture().isDone());
Review Comment:
Not strictly required, could remove. Really the `assertThatStage` after this
is what we're testing.
##
core/src/main/java/com/datastax/oss/driver/internal/core/metadata/MetadataManager.java:
##
@@ -437,30 +437,35 @@ private void
startSchemaRequest(CompletableFuture refreshFu
if (agreementError != null) {
refreshFuture.completeExceptionally(agreementError);
} else {
-schemaQueriesFactory
-.newInstance()
-.execute()
-.thenApplyAsync(this::parseAndApplySchemaRows,
adminExecutor)
-.whenComplete(
-(newMetadata, metadataError) -> {
- if (metadataError != null) {
-
refreshFuture.completeExceptionally(metadataError);
- } else {
-refreshFuture.complete(
-new RefreshSchemaResult(newMetadata,
schemaInAgreement));
- }
-
- firstSchemaRefreshFuture.complete(null);
-
- currentSchemaRefresh = null;
- // If another refresh was enqueued during this
one, run it now
- if (queuedSchemaRefresh != null) {
-CompletableFuture tmp =
-this.queuedSchemaRefresh;
-this.queuedSchemaRefresh = null;
-startSchemaRequest(tmp);
- }
-});
+try {
+ schemaQueriesFactory
+ .newInstance()
+ .execute()
+ .thenApplyAsync(this::parseAndApplySchemaRows,
adminExecutor)
+ .whenComplete(
+ (newMetadata, metadataError) -> {
+if (metadataError != null) {
+
refreshFuture.completeExceptionally(metadataError);
+} else {
+ refreshFuture.complete(
+ new
RefreshSchemaResult(newMetadata, schemaInAgreement));
+}
+
+
firstSchemaRefreshFuture.complete(null);
+
+currentSchemaRefresh = null;
+// If another refresh was enqueued
during this one, run it now
+if (queuedSchemaRefresh != null) {
+
CompletableFuture tmp =
+ this.queuedSchemaRefresh;
+ this.queuedSchemaRefresh = null;
+ startSchemaRequest(tmp);
+}
+ });
+} catch (Throwable t) {
Review Comment:
Was considering RuntimeException but ultimately decided Throwable was better
for this use case.
Also, a couple notes here:
- the diff is mostly whitespace since this block is now wrapped with a
try/catch
- the only bits that can throw at this level (vs inside the completion
[jira] [Updated] (CASSANDRA-19393) nodetool: group CMS-related commands into one command
[ https://issues.apache.org/jira/browse/CASSANDRA-19393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19393: -- Fix Version/s: 5.1 (was: 5.x) Source Control Link: https://github.com/apache/cassandra/commit/98ca5f8f1aab6659fdfd084bef38495bf50d35d6 Resolution: Fixed Status: Resolved (was: Ready to Commit) > nodetool: group CMS-related commands into one command > - > > Key: CASSANDRA-19393 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19393 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool, Transactional Cluster Metadata >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Fix For: 5.1 > > Time Spent: 1.5h > Remaining Estimate: 0h > > The purpose of this ticket is to group all CMS-related commands under one > "nodetool cms" command where existing command would be subcommands of it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: Group nodetool cms commands into single command group
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 98ca5f8f1a Group nodetool cms commands into single command group 98ca5f8f1a is described below commit 98ca5f8f1aab6659fdfd084bef38495bf50d35d6 Author: nvharikrishna AuthorDate: Wed Feb 14 15:09:06 2024 +0530 Group nodetool cms commands into single command group patch by N. V. Harikrishna; reviewed by Stefan Miklosovic and Marcus Eriksson for CASSANDRA-19393 --- CHANGES.txt| 1 + .../cassandra/tcm/TransactionalClusterMetadata.md | 4 +- src/java/org/apache/cassandra/tools/NodeTool.java | 10 +- .../apache/cassandra/tools/nodetool/CMSAdmin.java | 163 + .../cassandra/tools/nodetool/DescribeCMS.java | 44 -- .../cassandra/tools/nodetool/InitializeCMS.java| 39 - .../cassandra/tools/nodetool/ReconfigureCMS.java | 128 .../cassandra/distributed/test/TestBaseImpl.java | 2 +- .../distributed/test/log/ReconfigureCMSTest.java | 17 ++- .../test/log/TriggeredReconfigureCMSTest.java | 6 +- .../ClusterMetadataSingleNodeUpgradeTest.java | 2 +- .../upgrade/ClusterMetadataUpgradeHarryTest.java | 4 +- .../upgrade/ClusterMetadataUpgradeHostIdTest.java | 2 +- .../ClusterMetadataUpgradeIgnoreHostTest.java | 6 +- .../ClusterMetadataUpgradeIgnoreHostsTest.java | 6 +- .../upgrade/ClusterMetadataUpgradeTest.java| 4 +- .../upgrade/DropCompactStorageTest.java| 2 +- 17 files changed, 199 insertions(+), 241 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index f7ac26859f..7bd2710ddf 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 5.1 + * Group nodetool cms commands into single command group (CASSANDRA-19393) * Register the measurements of the bootstrap process as Dropwizard metrics (CASSANDRA-19447) * Add LIST SUPERUSERS CQL statement (CASSANDRA-19417) * Modernize CQLSH datetime conversions (CASSANDRA-18879) diff --git a/src/java/org/apache/cassandra/tcm/TransactionalClusterMetadata.md b/src/java/org/apache/cassandra/tcm/TransactionalClusterMetadata.md index 75fada86ae..1b691bba8c 100644 --- a/src/java/org/apache/cassandra/tcm/TransactionalClusterMetadata.md +++ b/src/java/org/apache/cassandra/tcm/TransactionalClusterMetadata.md @@ -71,7 +71,7 @@ Nodes provide each other with missing log entries in the form of a `LogState`, w Following an upgrade, nodes in an existing cluster will enter a minimal modification mode. In this state, the set of allowed cluster metadata modifications is constrained to include only the addition, removal and replacement of nodes, to allow failed hosts to be replaced during the upgrade. In this mode the CMS has no members and each peer maintains its own `ClusterMetadata` instance independently. This metadata is intitialised at startup from system tables and gossip is used to propagate the permitted subset of metadata changes. -When the operator is ready, one node is chosen for promotion to the initial CMS, which is done manually via nodetool `initializecms`. At this point, the candidate node will propose itself as the initial CMS and attempt to gain consensus from the rest of the cluster. If successful, it verifies that all peers have an identical view of cluster metadata and initialises the distributed log with a snapshot of that metadata. +When the operator is ready, one node is chosen for promotion to the initial CMS, which is done manually via nodetool `cms initialize`. At this point, the candidate node will propose itself as the initial CMS and attempt to gain consensus from the rest of the cluster. If successful, it verifies that all peers have an identical view of cluster metadata and initialises the distributed log with a snapshot of that metadata. -Once this process is complete all future cluster metadata updates are performed via the CMS using the global log and reverting to the previous method of metadata management is not supported. Further members can and should be added to the CMS using nodetool's `reconfigurecms` command. +Once this process is complete all future cluster metadata updates are performed via the CMS using the global log and reverting to the previous method of metadata management is not supported. Further members can and should be added to the CMS using nodetool's `cms reconfigure` command. diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java b/src/java/org/apache/cassandra/tools/NodeTool.java index 8e2c6f45cd..e653e7e2b5 100644 --- a/src/java/org/apache/cassandra/tools/NodeTool.java +++ b/src/java/org/apache/cassandra/tools/NodeTool.java @@ -107,7 +107,6 @@ public class NodeTool DataPaths.class, Decommission.class,
(cassandra-dtest) branch trunk updated: align tests with CASSANDRA-19393
This is an automated email from the ASF dual-hosted git repository.
smiklosovic pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new f3ca59c7 align tests with CASSANDRA-19393
f3ca59c7 is described below
commit f3ca59c76dc946e75b3b9dbc90f22dc51bf1c73a
Author: Stefan Miklosovic
AuthorDate: Thu Mar 14 11:10:22 2024 +0100
align tests with CASSANDRA-19393
---
upgrade_tests/storage_engine_upgrade_test.py | 2 +-
upgrade_tests/upgrade_through_versions_test.py | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/upgrade_tests/storage_engine_upgrade_test.py
b/upgrade_tests/storage_engine_upgrade_test.py
index d791f2a7..5992e7e7 100644
--- a/upgrade_tests/storage_engine_upgrade_test.py
+++ b/upgrade_tests/storage_engine_upgrade_test.py
@@ -70,7 +70,7 @@ class TestStorageEngineUpgrade(Tester):
self.install_legacy_parsing(node1)
node1.start(wait_for_binary_proto=True)
if node1.get_cassandra_version() >= '5.1':
-node1.nodetool("initializecms")
+node1.nodetool("cms initialize")
if self.fixture_dtest_setup.bootstrap:
cluster.set_install_dir(install_dir=self.fixture_dtest_setup.default_install_dir)
self.install_nodetool_legacy_parsing()
diff --git a/upgrade_tests/upgrade_through_versions_test.py
b/upgrade_tests/upgrade_through_versions_test.py
index cbd8c720..b27a0a42 100644
--- a/upgrade_tests/upgrade_through_versions_test.py
+++ b/upgrade_tests/upgrade_through_versions_test.py
@@ -500,7 +500,7 @@ class TestUpgrade(Tester):
self._check_select_count()
if self.cluster.version() >= '5.1':
-self.cluster.nodelist()[0].nodetool("initializecms")
+self.cluster.nodelist()[0].nodetool("cms initialize")
# run custom post-upgrade callables
for call in after_upgrade_call:
call()
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19472) CEP-15 (C*) Integrate accord with repair
[ https://issues.apache.org/jira/browse/CASSANDRA-19472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-19472: Test and Documentation Plan: ci Status: Patch Available (was: Open) > CEP-15 (C*) Integrate accord with repair > > > Key: CASSANDRA-19472 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19472 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19472) CEP-15 (C*) Integrate accord with repair
[ https://issues.apache.org/jira/browse/CASSANDRA-19472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-19472: Change Category: Operability Complexity: Normal Component/s: Accord Fix Version/s: 5.x Reviewers: Ariel Weisberg, David Capwell Assignee: Blake Eggleston Status: Open (was: Triage Needed) Wires accord repair into the normal repair path and removes the distinction between an accord repair job and a cassandra repair job, this makes both repair types coexist [https://github.com/apache/cassandra/pull/3174] > CEP-15 (C*) Integrate accord with repair > > > Key: CASSANDRA-19472 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19472 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18335) Push LocalSessions info logs to debug
[ https://issues.apache.org/jira/browse/CASSANDRA-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827284#comment-17827284 ] Arun Ganesh commented on CASSANDRA-18335: - Thanks! I'll close the PRs now. > Push LocalSessions info logs to debug > - > > Key: CASSANDRA-18335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18335 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Brandon Williams >Assignee: Arun Ganesh >Priority: Normal > Fix For: 4.0.13, 4.1.5, 5.0-beta2, 5.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > I don't think these are particularly useful to have at info, but the impetus > for this ticket is specifically [this > message|https://github.com/apache/cassandra/blob/cassandra-4.0/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java#L456] > during automatic cleanup about skipping deleting sessions that seems to > confuse or alarm people. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19016) CEP-15: (C*) per-table transactional configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-19016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-19016: Status: Ready to Commit (was: Review In Progress) Ariel approved on PR > CEP-15: (C*) per-table transactional configuration > -- > > Key: CASSANDRA-19016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19016 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 5.x > > > Accord configuration options should be rolled into as few options as > possible, with as few touch points as possible and, with support for > per-table configuration -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19016) CEP-15: (C*) per-table transactional configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-19016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-19016: Source Control Link: https://github.com/apache/cassandra/commit/85411ebcdd94a5b37b668ab16afc69c8443bc42c Resolution: Fixed Status: Resolved (was: Ready to Commit) > CEP-15: (C*) per-table transactional configuration > -- > > Key: CASSANDRA-19016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19016 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 5.x > > > Accord configuration options should be rolled into as few options as > possible, with as few touch points as possible and, with support for > per-table configuration -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19016) CEP-15: (C*) per-table transactional configuration
[ https://issues.apache.org/jira/browse/CASSANDRA-19016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Blake Eggleston updated CASSANDRA-19016: Reviewers: Ariel Weisberg (was: Ariel Weisberg, David Capwell) Status: Review In Progress (was: Patch Available) > CEP-15: (C*) per-table transactional configuration > -- > > Key: CASSANDRA-19016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19016 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 5.x > > > Accord configuration options should be rolled into as few options as > possible, with as few touch points as possible and, with support for > per-table configuration -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827281#comment-17827281
]
Stefan Miklosovic commented on CASSANDRA-19471:
---
ah great! I have not noticed this was happening on -latest in 19465 ... Anyway,
at least it discovered other issues and we improved both the test and pinned
down testing versions :) cc [~Bereng]
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cep-15-accord updated (698a1ea845 -> 85411ebcdd)
This is an automated email from the ASF dual-hosted git repository. bdeggleston pushed a change to branch cep-15-accord in repository https://gitbox.apache.org/repos/asf/cassandra.git from 698a1ea845 Refactor CommandsForKey for efficiency, and to support transitive dependency elision add 85411ebcdd CEP-15: (C*) per-table transactional configuration No new revisions were added by this update. Summary of changes: .../org/apache/cassandra/config/AccordSpec.java| 20 + src/java/org/apache/cassandra/config/Config.java | 115 --- .../cassandra/config/DatabaseDescriptor.java | 43 +- .../cassandra/cql3/statements/CQL3CasRequest.java | 5 +- .../cql3/statements/TransactionStatement.java | 15 +- .../statements/schema/AlterSchemaStatement.java| 4 + .../statements/schema/AlterTableStatement.java | 41 + .../statements/schema/CreateTableStatement.java| 11 + .../cql3/statements/schema/TableAttributes.java| 14 +- .../org/apache/cassandra/db/SystemKeyspace.java| 4 +- .../db/streaming/CassandraStreamReceiver.java | 2 +- .../apache/cassandra/repair/AccordRepairJob.java | 2 +- .../cassandra/repair/CassandraRepairJob.java | 2 +- .../org/apache/cassandra/repair/RepairResult.java | 2 +- .../org/apache/cassandra/repair/RepairSession.java | 4 +- .../apache/cassandra/schema/DistributedSchema.java | 15 + .../org/apache/cassandra/schema/TableMetadata.java | 15 + .../org/apache/cassandra/schema/TableParams.java | 61 +- .../org/apache/cassandra/service/StorageProxy.java | 84 +- .../apache/cassandra/service/StorageService.java | 25 +- .../cassandra/service/StorageServiceMBean.java | 4 - .../cassandra/service/accord/AccordService.java| 46 +- .../cassandra/service/accord/AccordTopology.java | 78 +- .../cassandra/service/accord/IAccordService.java | 68 +- .../accord/interop/AccordInteropExecution.java | 8 +- .../cassandra/service/accord/txn/TxnQuery.java | 6 - .../service/consensus/TransactionalMode.java | 140 .../migration/ConsensusKeyMigrationState.java | 7 +- .../consensus/migration/ConsensusMigratedAt.java | 70 ++ .../migration/ConsensusMigrationRepairResult.java | 50 ++ .../migration/ConsensusMigrationRepairType.java| 55 ++ .../migration/ConsensusMigrationState.java | 256 ++ .../migration/ConsensusMigrationTarget.java| 59 ++ .../migration/ConsensusRequestRouter.java | 99 ++- .../migration/ConsensusTableMigration.java | 336 .../migration/ConsensusTableMigrationState.java| 907 - .../consensus/migration/TableMigrationState.java | 360 .../migration/TransactionalMigrationFromMode.java | 83 ++ .../cassandra/service/paxos/PaxosPrepare.java | 3 +- .../service/reads/repair/BlockingReadRepair.java | 8 +- .../org/apache/cassandra/streaming/StreamPlan.java | 8 +- .../org/apache/cassandra/tcm/ClusterMetadata.java | 64 +- .../org/apache/cassandra/tcm/MetadataKeys.java | 2 - .../org/apache/cassandra/tcm/Transformation.java | 15 +- .../cassandra/tcm/compatibility/GossipHelper.java | 6 +- .../cassandra/tcm/ownership/AccordTables.java | 109 --- .../tcm/transformations/AddAccordTable.java| 90 -- .../cassandra/tcm/transformations/AlterSchema.java | 90 +- .../BeginConsensusMigrationForTableAndRange.java | 42 +- ...beFinishConsensusMigrationForTableAndRange.java | 67 +- .../SetConsensusMigrationTargetProtocol.java | 131 --- src/java/org/apache/cassandra/tools/NodeTool.java | 1 - .../tools/nodetool/ConsensusMigrationAdmin.java| 23 - .../cassandra/distributed/test/ReadRepairTest.java | 153 ++-- .../distributed/test/ShortReadProtectionTest.java | 60 +- .../test/accord/AccordBootstrapTest.java | 4 +- .../distributed/test/accord/AccordCQLTest.java | 224 +++-- .../test/accord/AccordFeatureFlagTest.java | 67 +- .../test/accord/AccordInteropReadTest.java | 9 +- .../test/accord/AccordInteroperabilityTest.java| 5 +- .../distributed/test/accord/AccordMetricsTest.java | 5 +- .../test/accord/AccordMigrationTest.java | 167 +++- .../test/accord/AccordSimpleFastPathTest.java | 3 +- .../distributed/test/accord/AccordTestBase.java| 9 +- .../distributed/test/accord/NewSchemaTest.java | 2 +- .../test/log/ClusterMetadataTestHelper.java| 4 - .../distributed/test/tcm/AccordAddTableTest.java | 80 -- .../apache/cassandra/audit/AuditLoggerTest.java| 2 +- .../org/apache/cassandra/auth/TxnAuthTest.java | 2 +- .../config/DatabaseDescriptorRefTest.java | 3 + .../cassandra/cql3/NodeLocalConsistencyTest.java | 2 +- .../cassandra/cql3/PreparedStatementsTest.java | 94 ++- .../cql3/statements/DescribeStatementTest.java | 2 + .../apache/cassandra/db/SchemaCQLHelperTest.java | 2 +
[jira] [Created] (CASSANDRA-19472) CEP-15 (C*) Integrate accord with repair
Blake Eggleston created CASSANDRA-19472: --- Summary: CEP-15 (C*) Integrate accord with repair Key: CASSANDRA-19472 URL: https://issues.apache.org/jira/browse/CASSANDRA-19472 Project: Cassandra Issue Type: Improvement Reporter: Blake Eggleston -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827278#comment-17827278
]
Brandon Williams commented on CASSANDRA-19471:
--
After much fretting on CASSANDRA-19465, I now realize that the original report
was actually against a -latest config circle run, so this all makes sense.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19471:
-
Description:
With the commitlog_disk_access_mode set to direct, and the improved
configuration_test.py::TestConfiguration::test_change_durable_writes from
CASSANDRA-19465, this fails with either:
{noformat}
AssertionError: Commitlog was written with durable writes disabled
{noformat}
Or what appears to be the original exception reported in CASSANDRA-19465:
{noformat}
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
StorageService.java:631 - Stopping native transport
node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465 StorageProxy.java:1670
- Failed to apply mutation locally :
java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
at java.base/java.nio.Buffer.position(Buffer.java:316)
at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
at
org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
StorageService.java:636 - Stopping gossiper
{noformat}
was:
With the commitlog_disk_access_mode set to direct, and the improved
configuration_test.py::TestConfiguration::test_change_durable_writes from
CASSANDRA-19465, this fails with either:
{noformat}
AssertionError: Commitlog was written with durable writes disabled
{noformat}
Or what appears to be the original exception reported in CASSANDRA-19465:
{noformat}
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
StorageService.java:631 - Stopping native transport
node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465 StorageProxy.java:1670
- Failed to apply mutation locally :
java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
at java.base/java.nio.Buffer.position(Buffer.java:316)
at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
at
org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
at
[jira] [Updated] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19471:
-
Description:
With the commitlog_disk_access_mode set to direct, and the improved
configuration_test.py::TestConfiguration::test_change_durable_writes from
CASSANDRA-19465, this fails with either:
{noformat}
AssertionError: Commitlog was written with durable writes disabled
{noformat}
Or what appears to be the original exception reported in CASSANDRA-19465:
{noformat}
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
StorageService.java:631 - Stopping native transport
node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465 StorageProxy.java:1670
- Failed to apply mutation locally :
java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
at java.base/java.nio.Buffer.position(Buffer.java:316)
at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
at
org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
StorageService.java:636 - Stopping gossiper
{noformat}
was:
With the commitlog_disk_access_mode set to direct, the improved
configuration_test.py::TestConfiguration::test_change_durable_writes from
CASSANDRA-19465, this fails with either:
{noformat}
AssertionError: Commitlog was written with durable writes disabled
{noformat}
Or what appears to be the original exception reported in CASSANDRA-19465:
{noformat}
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
StorageService.java:631 - Stopping native transport
node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465 StorageProxy.java:1670
- Failed to apply mutation locally :
java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
at java.base/java.nio.Buffer.position(Buffer.java:316)
at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
at
org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827277#comment-17827277
]
Brandon Williams commented on CASSANDRA-19471:
--
/cc [~blambov]
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19471:
-
Bug Category: Parent values: Degradation(12984)Level 1 values: Other
Exception(12998)
Complexity: Normal
Component/s: Local/Commit Log
Discovered By: User Report
Fix Version/s: 5.0-rc
5.x
Severity: Normal
Status: Open (was: Triage Needed)
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19471:
-
Description:
With the commitlog_disk_access_mode set to direct, the improved
configuration_test.py::TestConfiguration::test_change_durable_writes from
CASSANDRA-19465, this fails with either:
{noformat}
AssertionError: Commitlog was written with durable writes disabled
{noformat}
Or what appears to be the original exception reported in CASSANDRA-19465:
{noformat}
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
StorageService.java:631 - Stopping native transport
node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465 StorageProxy.java:1670
- Failed to apply mutation locally :
java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
at java.base/java.nio.Buffer.position(Buffer.java:316)
at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
at
org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
StorageService.java:636 - Stopping gossiper
{noformat}
was:
With the commitlog_disk_access_mode set to direct, the improved
configuration_test.py::TestConfiguration::test_change_durable_writes from
CASSANDRA-19465, this fails with either:
{noforma}
AssertionError: Commitlog was written with durable writes disabled
{noformat}
Or what appears to be the original exception reported in CASSANDRA-19465:
{noformat}
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
StorageService.java:631 - Stopping native transport
node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465 StorageProxy.java:1670
- Failed to apply mutation locally :
java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
at java.base/java.nio.Buffer.position(Buffer.java:316)
at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
at
org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
[jira] [Created] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
Brandon Williams created CASSANDRA-19471:
Summary: Commitlog with direct io fails test_change_durable_writes
Key: CASSANDRA-19471
URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
Project: Cassandra
Issue Type: Bug
Reporter: Brandon Williams
With the commitlog_disk_access_mode set to direct, the improved
configuration_test.py::TestConfiguration::test_change_durable_writes from
CASSANDRA-19465, this fails with either:
{noforma}
AssertionError: Commitlog was written with durable writes disabled
{noformat}
Or what appears to be the original exception reported in CASSANDRA-19465:
{noformat}
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
StorageService.java:631 - Stopping native transport
node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465 StorageProxy.java:1670
- Failed to apply mutation locally :
java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
at java.base/java.nio.Buffer.position(Buffer.java:316)
at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
at
java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
at
org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
at
org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
at
org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
at
org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
at
org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
StorageService.java:636 - Stopping gossiper
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-105) RestoreSliceTask could be stuck due to missing exception handling
[ https://issues.apache.org/jira/browse/CASSANDRASC-105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saranya Krishnakumar updated CASSANDRASC-105: - Authors: Saranya Krishnakumar Test and Documentation Plan: Unit tests Status: Patch Available (was: Open) patch: [https://github.com/apache/cassandra-sidecar/pull/107] > RestoreSliceTask could be stuck due to missing exception handling > - > > Key: CASSANDRASC-105 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-105 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Yifan Cai >Assignee: Saranya Krishnakumar >Priority: Normal > > In RestoreSliceTask, there are a few places could throw exceptions but > missing exception handling in call-sites. As a result, the RetoreSliceTask > never fulfill the promise, i.e. the task is stuck. > For example, downloadObjectIfAbsent could throw instead of returning a > future, in such case, the task will never fail or complete. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-105) RestoreSliceTask could be stuck due to missing exception handling
[ https://issues.apache.org/jira/browse/CASSANDRASC-105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saranya Krishnakumar updated CASSANDRASC-105: - Bug Category: Parent values: Correctness(12982) Complexity: Normal Discovered By: User Report Reviewers: Francisco Guerrero, Yifan Cai Severity: Normal Status: Open (was: Triage Needed) > RestoreSliceTask could be stuck due to missing exception handling > - > > Key: CASSANDRASC-105 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-105 > Project: Sidecar for Apache Cassandra > Issue Type: Bug > Components: Rest API >Reporter: Yifan Cai >Assignee: Saranya Krishnakumar >Priority: Normal > > In RestoreSliceTask, there are a few places could throw exceptions but > missing exception handling in call-sites. As a result, the RetoreSliceTask > never fulfill the promise, i.e. the task is stuck. > For example, downloadObjectIfAbsent could throw instead of returning a > future, in such case, the task will never fail or complete. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Reviewers: Stefan Miklosovic (was: Brandon Williams, Stefan Miklosovic) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.13, 4.1.5, 5.0, 5.1 > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-dtest) branch trunk updated: Pin python packages except for the driver and ccm
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/trunk by this push: new 73106380 Pin python packages except for the driver and ccm 73106380 is described below commit 731063804d203f2c7b38a40c4f3476598a26346d Author: Brandon Williams AuthorDate: Thu Mar 14 10:46:51 2024 -0500 Pin python packages except for the driver and ccm Patch by brandonwilliams; reviewed by smiklosovic for CASSANDRA-19469 --- requirements.txt | 38 +++--- 1 file changed, 23 insertions(+), 15 deletions(-) diff --git a/requirements.txt b/requirements.txt index facb71f6..dbecfbd2 100644 --- a/requirements.txt +++ b/requirements.txt @@ -10,20 +10,28 @@ # In case you want to test a patch with your own CCM branch, further to changing below CCM repo and branch name, you need to add -e flag at the beginning # Example: -e git+https://github.com/userb/ccm.git@cassandra-17182#egg=ccm git+https://github.com/riptano/ccm.git@cassandra-test#egg=ccm -decorator -docopt -enum34 -flaky -mock -pytest>=6.5.0,<=8.0.2 +click==8.1.7 +decorator==5.1.1 +docopt==0.6.2 +enum34==1.1.10 +exceptiongroup==1.2.0 +flaky==3.8.1 +geomet==0.2.1.post1 +iniconfig==2.0.0 +lxml==5.1.0 +mock==5.1.0 +netifaces==0.11.0 +packaging==24.0 +parse==1.20.1 +pluggy==1.4.0 +psutil==5.9.8 +py==1.11.0 +pycodestyle==2.11.1 +pytest==8.0.2 +pytest-repeat==0.9.3 pytest-timeout==1.4.2 -pytest-repeat -py -parse -pycodestyle -psutil -six>=1.12.0 +PyYAML==6.0.1 +six==1.16.0 +soupsieve==2.5 thrift==0.10.0 -netifaces -beautifulsoup4 -lxml +tomli==2.0.1 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Status: Ready to Commit (was: Review In Progress) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Reviewers: Stefan Miklosovic, Brandon Williams Stefan Miklosovic, Brandon Williams (was: Brandon Williams, Stefan Miklosovic) Status: Review In Progress (was: Patch Available) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Fix Version/s: 3.0.30 3.11.17 4.0.13 4.1.5 5.0 5.1 (was: 3.0.x) (was: 3.11.x) (was: 5.x) (was: 4.0.x) (was: 4.1.x) (was: 5.0.x) Since Version: NA Source Control Link: https://github.com/apache/cassandra-dtest/commit/731063804d203f2c7b38a40c4f3476598a26346d Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks, committed. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.13, 4.1.5, 5.0, 5.1 > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19393) nodetool: group CMS-related commands into one command
[ https://issues.apache.org/jira/browse/CASSANDRA-19393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827262#comment-17827262 ] Stefan Miklosovic edited comment on CASSANDRA-19393 at 3/14/24 9:47 PM: These two tests are the only ones dealing with cms: storage_engine_upgrade_test upgrade_through_versions_test repeated dtest for the first one is here https://app.circleci.com/pipelines/github/instaclustr/cassandra/4009/workflows/2b9fd421-82bc-47a8-9d82-7c76c1a22bc8/jobs/204887 I also run the the job for the second one but after 5h it timeouted, probably circle thing / flakiness. I do not have any reason to think that my change introduced the timeouts. It is literally just "initializecms -> cms initialize" dtest branch tested: https://github.com/smiklosovic/cassandra-dtest/commit/505e78ce3d5f1bcc596b29ffdb8a4e10de2b3a33 I am going to ship this. was (Author: smiklosovic): These two tests are the only ones dealing with cms: storage_engine_upgrade_test upgrade_through_versions_test repeated dtest for the first one is here https://app.circleci.com/pipelines/github/instaclustr/cassandra/4009/workflows/2b9fd421-82bc-47a8-9d82-7c76c1a22bc8/jobs/204887 I also run the the job for the second one but after 5h it timeouted, probably circle thing / flakiness. I do not have any reason to think that my changed introduced the timeouts. It is literally just "initializecms -> cms initialize" dtest branch tested: https://github.com/smiklosovic/cassandra-dtest/commit/505e78ce3d5f1bcc596b29ffdb8a4e10de2b3a33 I am going to ship this. > nodetool: group CMS-related commands into one command > - > > Key: CASSANDRA-19393 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19393 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool, Transactional Cluster Metadata >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Fix For: 5.x > > Time Spent: 1.5h > Remaining Estimate: 0h > > The purpose of this ticket is to group all CMS-related commands under one > "nodetool cms" command where existing command would be subcommands of it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19393) nodetool: group CMS-related commands into one command
[ https://issues.apache.org/jira/browse/CASSANDRA-19393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827262#comment-17827262 ] Stefan Miklosovic commented on CASSANDRA-19393: --- These two tests are the only ones dealing with cms: storage_engine_upgrade_test upgrade_through_versions_test repeated dtest for the first one is here https://app.circleci.com/pipelines/github/instaclustr/cassandra/4009/workflows/2b9fd421-82bc-47a8-9d82-7c76c1a22bc8/jobs/204887 I also run the the job for the second one but after 5h it timeouted, probably circle thing / flakiness. I do not have any reason to think that my changed introduced the timeouts. It is literally just "initializecms -> cms initialize" dtest branch tested: https://github.com/smiklosovic/cassandra-dtest/commit/505e78ce3d5f1bcc596b29ffdb8a4e10de2b3a33 I am going to ship this. > nodetool: group CMS-related commands into one command > - > > Key: CASSANDRA-19393 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19393 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool, Transactional Cluster Metadata >Reporter: n.v.harikrishna >Assignee: n.v.harikrishna >Priority: Normal > Fix For: 5.x > > Time Spent: 1.5h > Remaining Estimate: 0h > > The purpose of this ticket is to group all CMS-related commands under one > "nodetool cms" command where existing command would be subcommands of it. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827261#comment-17827261 ] Stefan Miklosovic edited comment on CASSANDRA-19469 at 3/14/24 9:41 PM: I think this is committable, failures seem totally unrelated to the changes proposed and might be treated separately. +1. was (Author: smiklosovic): I think this is committable, failures seems totally unrelated to the changes proposed and might be treated separately. +1. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827261#comment-17827261 ] Stefan Miklosovic commented on CASSANDRA-19469: --- I think this is committable, failures seems totally unrelated to the changes proposed and might be treated separately. +1. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14572) Expose all table metrics in virtual table
[
https://issues.apache.org/jira/browse/CASSANDRA-14572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827259#comment-17827259
]
Chris Lohfink commented on CASSANDRA-14572:
---
+1 great follow through! I like the non-processor approach
> Expose all table metrics in virtual table
> -
>
> Key: CASSANDRA-14572
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14572
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/Observability, Observability/Metrics
>Reporter: Chris Lohfink
>Assignee: Maxim Muzafarov
>Priority: Low
> Labels: virtual-tables
> Fix For: 5.x
>
> Attachments: flight_recording_1270017199_13.jfr, keyspayces_group
> responses times.png, keyspayces_group summary.png, select keyspaces_group by
> string prefix.png, select keyspaces_group compare with wo.png, select
> keyspaces_group without value.png, systemv_views.metrics_dropped_message.png,
> thread_pools benchmark.png
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> While we want a number of virtual tables to display data in a way thats great
> and intuitive like in nodetool. There is also much for being able to expose
> the metrics we have for tooling via CQL instead of JMX. This is more for the
> tooling and adhoc advanced users who know exactly what they are looking for.
> *Schema:*
> Initial idea is to expose data via {{((keyspace, table), metric)}} with a
> column for each metric value. Could also use a Map or UDT instead of the
> column based that can be a bit more specific to each metric type. To that end
> there can be a {{metric_type}} column and then a UDT for each metric type
> filled in, or a single value with more of a Map style. I am
> purposing the column type though as with {{ALLOW FILTERING}} it does allow
> more extensive query capabilities.
> *Implementations:*
> * Use reflection to grab all the metrics from TableMetrics (see:
> CASSANDRA-7622 impl). This is easiest and least abrasive towards new metric
> implementors... but its reflection and a kinda a bad idea.
> * Add a hook in TableMetrics to register with this virtual table when
> registering
> * Pull from the CassandraMetrics registery (either reporter or iterate
> through metrics query on read of virtual table)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827255#comment-17827255 ] Doug Rohrer commented on CASSANDRASC-106: - A few Circle-related issues, but here's the green run: [https://app.circleci.com/pipelines/github/JeetKunDoug/cassandra-sidecar/120/workflows/fc4c3ca6-5560-4865-9941-3e243a392382] > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827228#comment-17827228 ] Yifan Cai commented on CASSANDRASC-106: --- +1, assume green CI Thank you for the patch! > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827219#comment-17827219 ] Doug Rohrer commented on CASSANDRASC-106: - Most recent Circle run is going at [https://app.circleci.com/pipelines/github/JeetKunDoug/cassandra-sidecar/120/workflows/b252143a-46b3-4e45-be14-c7f85bd8c8c5] after incorporating Yifan's suggestions. > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Status: Patch Available (was: Open) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827199#comment-17827199 ] Brandon Williams commented on CASSANDRA-19469: -- The failure on _latest_ with the recently fixed test_change_durable_writes sounds like it may be a problem with that config since it was multiplexed without failure on the stock config. Perhaps direct IO on the commitlog, I'm not sure. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19468) In some situations MetadataManager.SingleThreaded.startSchemaRequest could fail to set CompletableFuture arg
[
https://issues.apache.org/jira/browse/CASSANDRA-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bret McGuire reassigned CASSANDRA-19468:
Assignee: (was: Henry Hughes)
> In some situations MetadataManager.SingleThreaded.startSchemaRequest could
> fail to set CompletableFuture arg
>
>
> Key: CASSANDRA-19468
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19468
> Project: Cassandra
> Issue Type: Bug
> Components: Client/java-driver
>Reporter: Bret McGuire
>Priority: Normal
>
> Relevant code is:
>
>
> {code:java}
> private void startSchemaRequest(CompletableFuture
> refreshFuture) {
> assert adminExecutor.inEventLoop();
> if (closeWasCalled) {
> refreshFuture.complete(new RefreshSchemaResult(metadata));
> return;
> }
> if (currentSchemaRefresh == null) {
> currentSchemaRefresh = refreshFuture;
> LOG.debug("[{}] Starting schema refresh", logPrefix);
> initControlConnectionForSchema()
> .thenCompose(v -> context.getTopologyMonitor().checkSchemaAgreement())
> .whenComplete(
> (schemaInAgreement, agreementError) -> {
> if (agreementError != null) {
> refreshFuture.completeExceptionally(agreementError);
> } else {
> schemaQueriesFactory
> .newInstance()
> .execute()
> .thenApplyAsync(this::parseAndApplySchemaRows,
> adminExecutor)
> .whenComplete(
> (newMetadata, metadataError) -> {
> if (metadataError != null) {
>
> refreshFuture.completeExceptionally(metadataError);
> } else {
> refreshFuture.complete(
> new RefreshSchemaResult(newMetadata,
> schemaInAgreement));
> }
> ...{code}
>
>
> Problem is that the default impl of SchemaQueriesFactory
> (DefaultSchemaQueriesFactory) can chuck exceptions in a few cases, most
> notably if the control connection is in a bad way:
>
>
> {code:java}
> @Override
> public SchemaQueries newInstance() {
> DriverChannel channel = context.getControlConnection().channel();
> if (channel == null || channel.closeFuture().isDone()) {
> throw new IllegalStateException("Control channel not available, aborting
> schema refresh");
> }
> Node node =
> context
> .getMetadataManager()
> .getMetadata()
> .findNode(channel.getEndPoint())
> .orElseThrow(
> () ->
> new IllegalStateException(
> "Could not find control node metadata "
> + channel.getEndPoint()
> + ", aborting schema refresh"));
> return newInstance(node, channel);
> } {code}
>
>
>
> In this case the MetadataManager code above will exit out before it ever sets
> a state on refreshFuture... meaning anything waiting on that future will just
> continue to wait.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19468) In some situations MetadataManager.SingleThreaded.startSchemaRequest could fail to set CompletableFuture arg
[
https://issues.apache.org/jira/browse/CASSANDRA-19468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bret McGuire updated CASSANDRA-19468:
-
Authors: (was: Henry Hughes)
> In some situations MetadataManager.SingleThreaded.startSchemaRequest could
> fail to set CompletableFuture arg
>
>
> Key: CASSANDRA-19468
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19468
> Project: Cassandra
> Issue Type: Bug
> Components: Client/java-driver
>Reporter: Bret McGuire
>Assignee: Henry Hughes
>Priority: Normal
>
> Relevant code is:
>
>
> {code:java}
> private void startSchemaRequest(CompletableFuture
> refreshFuture) {
> assert adminExecutor.inEventLoop();
> if (closeWasCalled) {
> refreshFuture.complete(new RefreshSchemaResult(metadata));
> return;
> }
> if (currentSchemaRefresh == null) {
> currentSchemaRefresh = refreshFuture;
> LOG.debug("[{}] Starting schema refresh", logPrefix);
> initControlConnectionForSchema()
> .thenCompose(v -> context.getTopologyMonitor().checkSchemaAgreement())
> .whenComplete(
> (schemaInAgreement, agreementError) -> {
> if (agreementError != null) {
> refreshFuture.completeExceptionally(agreementError);
> } else {
> schemaQueriesFactory
> .newInstance()
> .execute()
> .thenApplyAsync(this::parseAndApplySchemaRows,
> adminExecutor)
> .whenComplete(
> (newMetadata, metadataError) -> {
> if (metadataError != null) {
>
> refreshFuture.completeExceptionally(metadataError);
> } else {
> refreshFuture.complete(
> new RefreshSchemaResult(newMetadata,
> schemaInAgreement));
> }
> ...{code}
>
>
> Problem is that the default impl of SchemaQueriesFactory
> (DefaultSchemaQueriesFactory) can chuck exceptions in a few cases, most
> notably if the control connection is in a bad way:
>
>
> {code:java}
> @Override
> public SchemaQueries newInstance() {
> DriverChannel channel = context.getControlConnection().channel();
> if (channel == null || channel.closeFuture().isDone()) {
> throw new IllegalStateException("Control channel not available, aborting
> schema refresh");
> }
> Node node =
> context
> .getMetadataManager()
> .getMetadata()
> .findNode(channel.getEndPoint())
> .orElseThrow(
> () ->
> new IllegalStateException(
> "Could not find control node metadata "
> + channel.getEndPoint()
> + ", aborting schema refresh"));
> return newInstance(node, channel);
> } {code}
>
>
>
> In this case the MetadataManager code above will exit out before it ever sets
> a state on refreshFuture... meaning anything waiting on that future will just
> continue to wait.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Correction of exception handling in asynchronous programming [cassandra-java-driver]
absurdfarce commented on PR #1918: URL: https://github.com/apache/cassandra-java-driver/pull/1918#issuecomment-1997947155 Closing this out, will address in CASSANDRA-19468 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] Correction of exception handling in asynchronous programming [cassandra-java-driver]
absurdfarce closed pull request #1918: Correction of exception handling in asynchronous programming URL: https://github.com/apache/cassandra-java-driver/pull/1918 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827176#comment-17827176 ] Brandon Williams commented on CASSANDRA-19469: -- There were a couple of dtest failures, but it looks like those need to be ticketed since they also fail on [stock 5.0|https://app.circleci.com/pipelines/github/driftx/cassandra/1522/workflows/f1cd6321-9a48-4a43-92b3-aed0b610fe5e]. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827169#comment-17827169 ] Doug Rohrer commented on CASSANDRASC-106: - Transient "could not bind to port" race condition in the previous run - rerunning/continuing at [https://app.circleci.com/pipelines/github/JeetKunDoug/cassandra-sidecar/119/workflows/11c1bbba-ffe4-478c-b4ca-84c3c2339c41] > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRASC-106: -- Reviewers: Francisco Guerrero, Yifan Cai Status: Review In Progress (was: Patch Available) > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827159#comment-17827159 ] Brandon Williams edited comment on CASSANDRA-19469 at 3/14/24 4:13 PM: --- I have a branch [here|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-19469] that pins the packages to their existing versions except for the python driver and ccm, and reorders them to same a 'pip freeze' so that they are easy to update later, and CI is [here|https://app.circleci.com/pipelines/github/driftx/cassandra/1521/workflows/0e547d1c-80d7-49d4-975e-9161c7b171d6] for 5.0. I think one branch is probably sufficient for this. was (Author: brandon.williams): I have a branch [here|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-19469] that pins the packages to their existing versions except for the python driver, and reorders them to same a 'pip freeze' so that they are easy to update later, and CI is [here|https://app.circleci.com/pipelines/github/driftx/cassandra/1521/workflows/31de16bc-662f-421d-a46a-003009517be1] for 5.0. I think one branch is probably sufficient for this. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827159#comment-17827159 ] Brandon Williams edited comment on CASSANDRA-19469 at 3/14/24 4:13 PM: --- I have a branch [here|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-19469] that pins the packages to their existing versions except for the python driver and ccm, and reorders them to same as 'pip freeze' so that they are easy to update later, and CI is [here|https://app.circleci.com/pipelines/github/driftx/cassandra/1521/workflows/0e547d1c-80d7-49d4-975e-9161c7b171d6] for 5.0. I think one branch is probably sufficient for this. was (Author: brandon.williams): I have a branch [here|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-19469] that pins the packages to their existing versions except for the python driver and ccm, and reorders them to same a 'pip freeze' so that they are easy to update later, and CI is [here|https://app.circleci.com/pipelines/github/driftx/cassandra/1521/workflows/0e547d1c-80d7-49d4-975e-9161c7b171d6] for 5.0. I think one branch is probably sufficient for this. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-19469: Assignee: Brandon Williams (was: Stefan Miklosovic) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Status: Open (was: Patch Available) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827161#comment-17827161 ] Stefan Miklosovic commented on CASSANDRA-19469: --- I'll close my PR in favor of Brandon's. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827159#comment-17827159 ] Brandon Williams commented on CASSANDRA-19469: -- I have a branch [here|https://github.com/driftx/cassandra-dtest/tree/CASSANDRA-19469] that pins the packages to their existing versions except for the python driver, and reorders them to same a 'pip freeze' so that they are easy to update later, and CI is [here|https://app.circleci.com/pipelines/github/driftx/cassandra/1521/workflows/31de16bc-662f-421d-a46a-003009517be1] for 5.0. I think one branch is probably sufficient for this. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19469: -- Test and Documentation Plan: ci Status: Patch Available (was: In Progress) This is taken from expanding the "Configure virtualenv and Python dependencies" step in any dtest in circle where it shows what it resolves them to. https://github.com/apache/cassandra-dtest/pull/257 > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic reassigned CASSANDRA-19469: - Assignee: Stefan Miklosovic > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.1' into cassandra-5.0
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit bd2803307eb5e8c0bfbd7e9477ce1756a7d77bde
Merge: 2f836fa596 8502da494c
Author: Brandon Williams
AuthorDate: Thu Mar 14 10:26:30 2024 -0500
Merge branch 'cassandra-4.1' into cassandra-5.0
CHANGES.txt| 1 +
.../cassandra/repair/consistent/LocalSessions.java | 36 +++---
2 files changed, 19 insertions(+), 18 deletions(-)
diff --cc CHANGES.txt
index 2e543f1120,a091edb379..4533e86b1c
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,33 -1,6 +1,34 @@@
-4.1.5
+5.0-beta2
+ * Set uuid_sstable_identifiers_enabled to true in cassandra-latest.yaml
(CASSANDRA-19460)
+ * Revert switching to approxTime in Dispatcher (CASSANDRA-19454)
+ * Add an optimized default configuration to tests and make it available for
new users (CASSANDRA-18753)
+ * Fix remote JMX under Java17 (CASSANDRA-19453)
+ * Avoid consistency violations for SAI intersections over unrepaired data at
consistency levels requiring reconciliation (CASSANDRA-19018)
+ * Fix NullPointerException in ANN+WHERE when adding rows in another
partition (CASSANDRA-19404)
+ * Record latencies for SAI post-filtering reads against local storage
(CASSANDRA-18940)
+ * Fix VectorMemoryIndex#update logic to compare vectors. Fix Index view
(CASSANDRA-19168)
+ * Deprecate native_transport_port_ssl (CASSANDRA-19392)
+ * Update packaging shell includes (CASSANDRA-19283)
+ * Fix data corruption in VectorCodec when using heap buffers
(CASSANDRA-19167)
+ * Avoid over-skipping of key iterators from static column indexes during
mixed intersections (CASSANDRA-19278)
+ * Make concurrent_index_builders configurable at runtime (CASSANDRA-19266)
+ * Fix storage_compatibility_mode for streaming (CASSANDRA-19126)
+ * Add support of vector type to cqlsh COPY command (CASSANDRA-19118)
+ * Make CQLSSTableWriter to support building of SAI indexes (CASSANDRA-18714)
+ * Append additional JVM options when using JDK17+ (CASSANDRA-19001)
+ * Upgrade Python driver to 3.29.0 (CASSANDRA-19245)
+ * Creating a SASI index after creating an SAI index does not break secondary
index queries (CASSANDRA-18939)
+ * Optionally fail when a non-partition-restricted query is issued against an
index (CASSANDRA-18796)
+ * Add a startup check to fail startup when using invalid configuration with
certain Kernel and FS type (CASSANDRA-19196)
+ * UCS min_sstable_size should not be lower than target_sstable_size lower
bound (CASSANDRA-19112)
+ * Fix the correspondingMessagingVersion of SSTable format and improve TTL
overflow tests coverage (CASSANDRA-19197)
+ * Fix resource cleanup after SAI query timeouts (CASSANDRA-19177)
+ * Suppress CVE-2023-6481 (CASSANDRA-19184)
+Merged from 4.1:
+ * Memoize Cassandra verion and add a backoff interval for failed schema
pulls (CASSANDRA-18902)
+ * Fix StackOverflowError on ALTER after many previous schema changes
(CASSANDRA-19166)
Merged from 4.0:
+ * Push LocalSessions info logs to debug (CASSANDRA-18335)
* Filter remote DC replicas out when constructing the initial replica plan
for the local read repair (CASSANDRA-19120)
* Remove redundant code in StorageProxy#sendToHintedReplicas
(CASSANDRA-19412)
* Remove bashisms for mx4j tool in cassandra-env.sh (CASSANDRA-19416)
diff --cc src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
index 75c50e5dce,5fcfe986c5..cc6b168461
--- a/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
+++ b/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
@@@ -841,12 -818,10 +841,12 @@@ public class LocalSession
}
LocalSession session = createSessionUnsafe(sessionID, parentSession,
peers);
-putSessionUnsafe(session);
+sendAck(ctx, message);
+if (!putSessionUnsafe(session))
+return;
- logger.info("Beginning local incremental repair session {}", session);
+ logger.debug("Beginning local incremental repair session {}",
session);
-ExecutorService executor = executorFactory().pooled("Repair-" +
sessionID, parentSession.getColumnFamilyStores().size());
+ExecutorService executor = ctx.executorFactory().pooled("Repair-" +
sessionID, parentSession.getColumnFamilyStores().size());
KeyspaceRepairManager repairManager =
parentSession.getKeyspace().getRepairManager();
RangesAtEndpoint tokenRanges =
filterLocalRanges(parentSession.getKeyspace().getName(),
parentSession.getRanges());
@@@ -859,10 -834,13 +859,10 @@@
{
try
{
- logger.info("Prepare phase for incremental repair session
{} completed", sessionID);
+ logger.debug("Prepare phase for incremental repair
session {} completed", sessionID);
(cassandra) branch cassandra-5.0 updated (2f836fa596 -> bd2803307e)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 2f836fa596 Set uuid_sstable_identifiers_enabled to true for cassandra-latest.yaml new a2fbb17867 Push LocalSessions info logs to debug new 8502da494c Merge branch 'cassandra-4.0' into cassandra-4.1 new bd2803307e Merge branch 'cassandra-4.1' into cassandra-5.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/repair/consistent/LocalSessions.java | 36 +++--- 2 files changed, 19 insertions(+), 18 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.0 updated: Push LocalSessions info logs to debug
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new a2fbb17867 Push LocalSessions info logs to debug
a2fbb17867 is described below
commit a2fbb17867469f83b94718d052b6c371f089ec5f
Author: arkn98 <20590666+ark...@users.noreply.github.com>
AuthorDate: Mon Mar 11 13:27:39 2024 -0400
Push LocalSessions info logs to debug
Patch by arkn98; reviewed by bereng and brandonwilliams for CASSANDRA-18335
---
CHANGES.txt| 1 +
.../cassandra/repair/consistent/LocalSessions.java | 34 +++---
2 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index f2712c5088..69f8be60a5 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.13
+ * Push LocalSessions info logs to debug (CASSANDRA-18335)
* Filter remote DC replicas out when constructing the initial replica plan
for the local read repair (CASSANDRA-19120)
* Remove redundant code in StorageProxy#sendToHintedReplicas (CASSANDRA-19412)
* Remove bashisms for mx4j tool in cassandra-env.sh (CASSANDRA-19416)
diff --git a/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
b/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
index cfb90ef04d..e61ad47003 100644
--- a/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
+++ b/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
@@ -322,7 +322,7 @@ public class LocalSessions
*/
public void cancelSession(UUID sessionID, boolean force)
{
-logger.info("Cancelling local repair session {}", sessionID);
+logger.debug("Cancelling local repair session {}", sessionID);
LocalSession session = getSession(sessionID);
Preconditions.checkArgument(session != null, "Session {} does not
exist", sessionID);
Preconditions.checkArgument(force ||
session.coordinator.equals(getBroadcastAddressAndPort()),
@@ -397,7 +397,7 @@ public class LocalSessions
case FINALIZE_PROMISED:
continue;
default:
-logger.info("Found repair session {} with state = {} -
failing the repair", session.sessionID, session.getState());
+logger.debug("Found repair session {} with state = {}
- failing the repair", session.sessionID, session.getState());
failSession(session, true);
}
}
@@ -453,12 +453,12 @@ public class LocalSessions
{
// if we delete a non-superseded session, some ranges
will be mis-reported as
// not having been repaired in repair_admin after a
restart
-logger.info("Skipping delete of FINALIZED LocalSession
{} because it has " +
+logger.debug("Skipping delete of FINALIZED
LocalSession {} because it has " +
"not been superseded by a more recent
session", session.sessionID);
}
else if (!sessionHasData(session))
{
-logger.info("Auto deleting repair session {}",
session);
+logger.debug("Auto deleting repair session {}",
session);
deleteSession(session.sessionID);
}
else
@@ -734,7 +734,7 @@ public class LocalSessions
}
else if (session.getState() != FAILED)
{
-logger.info("Failing local repair session {}",
session.sessionID);
+logger.debug("Failing local repair session {}",
session.sessionID);
setStateAndSave(session, FAILED);
}
}
@@ -747,7 +747,7 @@ public class LocalSessions
public synchronized void deleteSession(UUID sessionID)
{
-logger.info("Deleting local repair session {}", sessionID);
+logger.debug("Deleting local repair session {}", sessionID);
LocalSession session = getSession(sessionID);
Preconditions.checkArgument(session.isCompleted(), "Cannot delete
incomplete sessions");
@@ -817,7 +817,7 @@ public class LocalSessions
LocalSession session = createSessionUnsafe(sessionID, parentSession,
peers);
putSessionUnsafe(session);
-logger.info("Beginning local incremental repair session {}", session);
+logger.debug("Beginning local incremental repair session {}", session);
ExecutorService executor =
Executors.newFixedThreadPool(parentSession.getColumnFamilyStores().size());
@@ -832,9 +832,9 @@ public class LocalSessions
{
try
[jira] [Updated] (CASSANDRA-18335) Push LocalSessions info logs to debug
[ https://issues.apache.org/jira/browse/CASSANDRA-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18335: - Fix Version/s: 4.0.13 4.1.5 5.0-beta2 5.1 (was: 5.x) (was: 4.0.x) (was: 4.1.x) Source Control Link: https://github.com/apache/cassandra/commit/a2fbb17867469f83b94718d052b6c371f089ec5f Resolution: Fixed Status: Resolved (was: Ready to Commit) 5.0 merged cleanly to trunk. Committed. > Push LocalSessions info logs to debug > - > > Key: CASSANDRA-18335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18335 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Brandon Williams >Assignee: Arun Ganesh >Priority: Normal > Fix For: 4.0.13, 4.1.5, 5.0-beta2, 5.1 > > Time Spent: 0.5h > Remaining Estimate: 0h > > I don't think these are particularly useful to have at info, but the impetus > for this ticket is specifically [this > message|https://github.com/apache/cassandra/blob/cassandra-4.0/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java#L456] > during automatic cleanup about skipping deleting sessions that seems to > confuse or alarm people. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.1 updated (e120088d44 -> 8502da494c)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from e120088d44 Merge branch 'cassandra-4.0' into cassandra-4.1 new a2fbb17867 Push LocalSessions info logs to debug new 8502da494c Merge branch 'cassandra-4.0' into cassandra-4.1 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/repair/consistent/LocalSessions.java | 36 +++--- 2 files changed, 19 insertions(+), 18 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-4.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 8502da494cd94882915da4bd003833411d7f
Merge: e120088d44 a2fbb17867
Author: Brandon Williams
AuthorDate: Thu Mar 14 10:25:23 2024 -0500
Merge branch 'cassandra-4.0' into cassandra-4.1
CHANGES.txt| 1 +
.../cassandra/repair/consistent/LocalSessions.java | 36 +++---
2 files changed, 19 insertions(+), 18 deletions(-)
diff --cc CHANGES.txt
index fdb19e0cf0,69f8be60a5..a091edb379
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,5 -1,5 +1,6 @@@
-4.0.13
+4.1.5
+Merged from 4.0:
+ * Push LocalSessions info logs to debug (CASSANDRA-18335)
* Filter remote DC replicas out when constructing the initial replica plan
for the local read repair (CASSANDRA-19120)
* Remove redundant code in StorageProxy#sendToHintedReplicas
(CASSANDRA-19412)
* Remove bashisms for mx4j tool in cassandra-env.sh (CASSANDRA-19416)
diff --cc src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
index ed2bf0bc0b,e61ad47003..5fcfe986c5
--- a/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
+++ b/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java
@@@ -322,9 -320,9 +322,9 @@@ public class LocalSession
* hook for operators to cancel sessions, cancelling from a
non-coordinator is an error, unless
* force is set to true. Messages are sent out to other participants, but
we don't wait for a response
*/
-public void cancelSession(UUID sessionID, boolean force)
+public void cancelSession(TimeUUID sessionID, boolean force)
{
- logger.info("Cancelling local repair session {}", sessionID);
+ logger.debug("Cancelling local repair session {}", sessionID);
LocalSession session = getSession(sessionID);
Preconditions.checkArgument(session != null, "Session {} does not
exist", sessionID);
Preconditions.checkArgument(force ||
session.coordinator.equals(getBroadcastAddressAndPort()),
@@@ -747,9 -745,9 +747,9 @@@
}
}
-public synchronized void deleteSession(UUID sessionID)
+public synchronized void deleteSession(TimeUUID sessionID)
{
- logger.info("Deleting local repair session {}", sessionID);
+ logger.debug("Deleting local repair session {}", sessionID);
LocalSession session = getSession(sessionID);
Preconditions.checkArgument(session.isCompleted(), "Cannot delete
incomplete sessions");
@@@ -819,9 -817,9 +819,9 @@@
LocalSession session = createSessionUnsafe(sessionID, parentSession,
peers);
putSessionUnsafe(session);
- logger.info("Beginning local incremental repair session {}", session);
+ logger.debug("Beginning local incremental repair session {}",
session);
-ExecutorService executor =
Executors.newFixedThreadPool(parentSession.getColumnFamilyStores().size());
+ExecutorService executor = executorFactory().pooled("Repair-" +
sessionID, parentSession.getColumnFamilyStores().size());
KeyspaceRepairManager repairManager =
parentSession.getKeyspace().getRepairManager();
RangesAtEndpoint tokenRanges =
filterLocalRanges(parentSession.getKeyspace().getName(),
parentSession.getRanges());
@@@ -834,9 -832,10 +834,9 @@@
{
try
{
- logger.info("Prepare phase for incremental repair session
{} completed", sessionID);
+ logger.debug("Prepare phase for incremental repair
session {} completed", sessionID);
if (!prepareSessionExceptFailed(session))
- logger.info("Session {} failed before anticompaction
completed", sessionID);
+ logger.debug("Session {} failed before anticompaction
completed", sessionID);
-
Message message =
Message.out(PREPARE_CONSISTENT_RSP,
new PrepareConsistentResponse(sessionID,
getBroadcastAddressAndPort(), session.getState() != FAILED));
@@@ -852,12 -851,7 +852,12 @@@
{
try
{
-logger.error("Prepare phase for incremental repair
session {} failed", sessionID, t);
+if (Throwables.anyCauseMatches(t, (throwable) ->
throwable instanceof CompactionInterruptedException))
- logger.info("Anticompaction interrupted for session
{}: {}", sessionID, t.getMessage());
++logger.debug("Anticompaction interrupted for session
{}: {}", sessionID, t.getMessage());
+else if (Throwables.anyCauseMatches(t, (throwable) ->
throwable instanceof NoSuchRepairSessionException))
+
(cassandra) branch trunk updated (81a2cb782e -> 86dc0c120a)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 81a2cb782e Register the measurements of the bootstrap process as Dropwizard metrics new a2fbb17867 Push LocalSessions info logs to debug new 8502da494c Merge branch 'cassandra-4.0' into cassandra-4.1 new bd2803307e Merge branch 'cassandra-4.1' into cassandra-5.0 new 86dc0c120a Merge branch 'cassandra-5.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../cassandra/repair/consistent/LocalSessions.java | 36 +++--- 2 files changed, 19 insertions(+), 18 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18335) Push LocalSessions info logs to debug
[ https://issues.apache.org/jira/browse/CASSANDRA-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18335: - Reviewers: Berenguer Blasi (was: Berenguer Blasi, Brandon Williams) Status: Review In Progress (was: Needs Committer) > Push LocalSessions info logs to debug > - > > Key: CASSANDRA-18335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18335 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Brandon Williams >Assignee: Arun Ganesh >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > Time Spent: 0.5h > Remaining Estimate: 0h > > I don't think these are particularly useful to have at info, but the impetus > for this ticket is specifically [this > message|https://github.com/apache/cassandra/blob/cassandra-4.0/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java#L456] > during automatic cleanup about skipping deleting sessions that seems to > confuse or alarm people. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 86dc0c120abf369942a38eeb83e1cbaf1fd6ec77 Merge: 81a2cb782e bd2803307e Author: Brandon Williams AuthorDate: Thu Mar 14 10:28:02 2024 -0500 Merge branch 'cassandra-5.0' into trunk CHANGES.txt| 1 + .../cassandra/repair/consistent/LocalSessions.java | 36 +++--- 2 files changed, 19 insertions(+), 18 deletions(-) diff --cc CHANGES.txt index af985568cd,4533e86b1c..f7ac26859f --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -49,9 -25,10 +49,10 @@@ Merged from 5.0 * Fix resource cleanup after SAI query timeouts (CASSANDRA-19177) * Suppress CVE-2023-6481 (CASSANDRA-19184) Merged from 4.1: - * Memoize Cassandra verion and add a backoff interval for failed schema pulls (CASSANDRA-18902) * Fix StackOverflowError on ALTER after many previous schema changes (CASSANDRA-19166) + * Memoize Cassandra verion (CASSANDRA-18902) Merged from 4.0: + * Push LocalSessions info logs to debug (CASSANDRA-18335) * Filter remote DC replicas out when constructing the initial replica plan for the local read repair (CASSANDRA-19120) * Remove redundant code in StorageProxy#sendToHintedReplicas (CASSANDRA-19412) * Remove bashisms for mx4j tool in cassandra-env.sh (CASSANDRA-19416) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18335) Push LocalSessions info logs to debug
[ https://issues.apache.org/jira/browse/CASSANDRA-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18335: - Status: Ready to Commit (was: Review In Progress) > Push LocalSessions info logs to debug > - > > Key: CASSANDRA-18335 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18335 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Repair >Reporter: Brandon Williams >Assignee: Arun Ganesh >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > Time Spent: 0.5h > Remaining Estimate: 0h > > I don't think these are particularly useful to have at info, but the impetus > for this ticket is specifically [this > message|https://github.com/apache/cassandra/blob/cassandra-4.0/src/java/org/apache/cassandra/repair/consistent/LocalSessions.java#L456] > during automatic cleanup about skipping deleting sessions that seems to > confuse or alarm people. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827137#comment-17827137 ] Doug Rohrer commented on CASSANDRASC-106: - After rebase Circle run is at [https://app.circleci.com/pipelines/github/JeetKunDoug/cassandra-sidecar/119/workflows/d24eb4e3-c6c1-44a7-b81f-31874b050bfc] - waiting for green again (pre-rebase was green) > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19470: - Fix Version/s: 4.1.5 5.0 5.1 (was: 5.x) (was: 4.1.x) (was: 5.0.x) Source Control Link: https://github.com/apache/cassandra-dtest/commit/39e238994044d2a71c17e64b3db301ba9f2da51f Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed, thanks for the review. > TestMultiDCWriteFailures doesn't test versions past 4.0.x > - > > Key: CASSANDRA-19470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1.5, 5.0, 5.1 > > > I think this was disabled for future versions in CASSANDRA-17456, but there > is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-dtest) branch trunk updated: remove upper version restriction on TestMultiDCWriteFailures
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 39e23899 remove upper version restriction on TestMultiDCWriteFailures
39e23899 is described below
commit 39e238994044d2a71c17e64b3db301ba9f2da51f
Author: Brandon Williams
AuthorDate: Thu Mar 14 08:30:28 2024 -0500
remove upper version restriction on TestMultiDCWriteFailures
Patch by brandonwilliams; reviewed by smiklosovic for CASSANDRA-19470
---
write_failures_test.py | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/write_failures_test.py b/write_failures_test.py
index 0e936ad6..f47dbe07 100644
--- a/write_failures_test.py
+++ b/write_failures_test.py
@@ -2,7 +2,7 @@ import uuid
import pytest
import logging
-from cassandra import ConsistencyLevel, WriteFailure, WriteTimeout
+from cassandra import ConsistencyLevel, WriteFailure, WriteTimeout,
InvalidRequest
from cassandra.query import SimpleStatement
from dtest import Tester
@@ -234,11 +234,11 @@ class TestWriteFailures(Tester):
def assert_write_failure(session, query, consistency_level):
statement = SimpleStatement(query, consistency_level=consistency_level)
-with pytest.raises(WriteFailure):
+with pytest.raises((WriteFailure, InvalidRequest)):
session.execute(statement)
-@since('3.0', max_version='4.0.x')
+@since('3.0')
class TestMultiDCWriteFailures(Tester):
@pytest.fixture(autouse=True)
def fixture_add_additional_log_patterns(self, fixture_dtest_setup):
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19465:
-
Fix Version/s: 5.1
(was: 5.x)
Since Version: NA
Source Control Link:
https://github.com/apache/cassandra-dtest/commit/791a65a70b6c9576eb8529fb8de1bc50b7a7b3ca
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks, committed!
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.1
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def new_commitlog_cluster_node():
> # writes should block on commitlog fsync
> self.fixture_dtest_setup.cluster.populate(1)
> node = self.fixture_dtest_setup.cluster.nodelist()[0]
>
> self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
> use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
>
> self.fixture_dtest_setup.cluster.start()
> return node
>
> durable_node = new_commitlog_cluster_node()
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> # get a fresh cluster to work on
> durable_session.shutdown()
> self.fixture_dtest_setup.cleanup_and_replace_cluster()
>
> node = new_commitlog_cluster_node()
> init_size = commitlog_size(node)
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then alter it to use them
> session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
> 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = false")
> session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
> int, c int)')
> session.execute('ALTER KEYSPACE ks WITH DURABLE_WRITES=true')
> > write_to_trigger_fsync(session, 'ks', 'tab')
> configuration_test.py:113:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> configuration_test.py:186: in write_to_trigger_fsync
> execute_concurrent_with_args(session,
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:238: in
> execute_concurrent_with_args
> return execute_concurrent(session, zip(cycle((statement,)), parameters),
> *args, **kwargs)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:94: in
> execute_concurrent
> return executor.execute(concurrency,
[jira] [Updated] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19465:
-
Status: Needs Committer (was: Patch Available)
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def new_commitlog_cluster_node():
> # writes should block on commitlog fsync
> self.fixture_dtest_setup.cluster.populate(1)
> node = self.fixture_dtest_setup.cluster.nodelist()[0]
>
> self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
> use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
>
> self.fixture_dtest_setup.cluster.start()
> return node
>
> durable_node = new_commitlog_cluster_node()
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> # get a fresh cluster to work on
> durable_session.shutdown()
> self.fixture_dtest_setup.cleanup_and_replace_cluster()
>
> node = new_commitlog_cluster_node()
> init_size = commitlog_size(node)
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then alter it to use them
> session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
> 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = false")
> session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
> int, c int)')
> session.execute('ALTER KEYSPACE ks WITH DURABLE_WRITES=true')
> > write_to_trigger_fsync(session, 'ks', 'tab')
> configuration_test.py:113:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> configuration_test.py:186: in write_to_trigger_fsync
> execute_concurrent_with_args(session,
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:238: in
> execute_concurrent_with_args
> return execute_concurrent(session, zip(cycle((statement,)), parameters),
> *args, **kwargs)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:94: in
> execute_concurrent
> return executor.execute(concurrency, raise_on_first_error)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:201: in execute
> return super(ConcurrentExecutorListResults, self).execute(concurrency,
> fail_fast)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:120: in execute
> return
[jira] [Updated] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19465:
-
Reviewers: Stefan Miklosovic
Status: Review In Progress (was: Needs Committer)
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def new_commitlog_cluster_node():
> # writes should block on commitlog fsync
> self.fixture_dtest_setup.cluster.populate(1)
> node = self.fixture_dtest_setup.cluster.nodelist()[0]
>
> self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
> use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
>
> self.fixture_dtest_setup.cluster.start()
> return node
>
> durable_node = new_commitlog_cluster_node()
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> # get a fresh cluster to work on
> durable_session.shutdown()
> self.fixture_dtest_setup.cleanup_and_replace_cluster()
>
> node = new_commitlog_cluster_node()
> init_size = commitlog_size(node)
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then alter it to use them
> session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
> 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = false")
> session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
> int, c int)')
> session.execute('ALTER KEYSPACE ks WITH DURABLE_WRITES=true')
> > write_to_trigger_fsync(session, 'ks', 'tab')
> configuration_test.py:113:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> configuration_test.py:186: in write_to_trigger_fsync
> execute_concurrent_with_args(session,
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:238: in
> execute_concurrent_with_args
> return execute_concurrent(session, zip(cycle((statement,)), parameters),
> *args, **kwargs)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:94: in
> execute_concurrent
> return executor.execute(concurrency, raise_on_first_error)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:201: in execute
> return super(ConcurrentExecutorListResults, self).execute(concurrency,
> fail_fast)
>
[jira] [Updated] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19465:
-
Status: Ready to Commit (was: Review In Progress)
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def new_commitlog_cluster_node():
> # writes should block on commitlog fsync
> self.fixture_dtest_setup.cluster.populate(1)
> node = self.fixture_dtest_setup.cluster.nodelist()[0]
>
> self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
> use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
>
> self.fixture_dtest_setup.cluster.start()
> return node
>
> durable_node = new_commitlog_cluster_node()
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> # get a fresh cluster to work on
> durable_session.shutdown()
> self.fixture_dtest_setup.cleanup_and_replace_cluster()
>
> node = new_commitlog_cluster_node()
> init_size = commitlog_size(node)
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then alter it to use them
> session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
> 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = false")
> session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
> int, c int)')
> session.execute('ALTER KEYSPACE ks WITH DURABLE_WRITES=true')
> > write_to_trigger_fsync(session, 'ks', 'tab')
> configuration_test.py:113:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> configuration_test.py:186: in write_to_trigger_fsync
> execute_concurrent_with_args(session,
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:238: in
> execute_concurrent_with_args
> return execute_concurrent(session, zip(cycle((statement,)), parameters),
> *args, **kwargs)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:94: in
> execute_concurrent
> return executor.execute(concurrency, raise_on_first_error)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:201: in execute
> return super(ConcurrentExecutorListResults, self).execute(concurrency,
> fail_fast)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:120: in execute
> return
(cassandra-dtest) branch trunk updated: Remove fixtures and test durable writes disabled in configuration_test
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 791a65a7 Remove fixtures and test durable writes disabled in
configuration_test
791a65a7 is described below
commit 791a65a70b6c9576eb8529fb8de1bc50b7a7b3ca
Author: Brandon Williams
AuthorDate: Tue Mar 12 06:07:27 2024 -0500
Remove fixtures and test durable writes disabled in configuration_test
Patch by brandonwilliams; reviewed by smiklosovic for CASSANDRA-19465
---
configuration_test.py | 47 ---
1 file changed, 24 insertions(+), 23 deletions(-)
diff --git a/configuration_test.py b/configuration_test.py
index 3fd31e27..d307d2a7 100644
--- a/configuration_test.py
+++ b/configuration_test.py
@@ -6,8 +6,6 @@ import tempfile
from cassandra.concurrent import execute_concurrent_with_args
-from tools.misc import ImmutableMapping
-from dtest_setup_overrides import DTestSetupOverrides
from dtest import Tester, create_ks
from tools.jmxutils import (JolokiaAgent, make_mbean)
from distutils.version import LooseVersion
@@ -15,14 +13,6 @@ from distutils.version import LooseVersion
logger = logging.getLogger(__name__)
-@pytest.fixture()
-def fixture_dtest_setup_overrides(request, dtest_config):
-dtest_setup_overrides = DTestSetupOverrides()
-if request.node.name == "test_change_durable_writes":
-dtest_setup_overrides.cluster_options =
ImmutableMapping({'commitlog_segment_size_in_mb': 1})
-return dtest_setup_overrides
-
-
class TestConfiguration(Tester):
def test_compression_chunk_length(self):
@@ -74,16 +64,13 @@ class TestConfiguration(Tester):
- writing a dataset to this keyspace that is known to trigger a
commitlog fsync,
- asserting that the commitlog has grown in size since the data was
written.
"""
-def new_commitlog_cluster_node():
-# writes should block on commitlog fsync
-self.fixture_dtest_setup.cluster.populate(1)
-node = self.fixture_dtest_setup.cluster.nodelist()[0]
-self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
+cluster = self.cluster
+cluster.set_batch_commitlog(enabled=True, use_batch_window =
cluster.version() < '5.0')
+
cluster.set_configuration_options(values={'commitlog_segment_size_in_mb': 1})
-self.fixture_dtest_setup.cluster.start()
-return node
+cluster.populate(1).start()
+durable_node = cluster.nodelist()[0]
-durable_node = new_commitlog_cluster_node()
durable_init_size = commitlog_size(durable_node)
durable_session = self.patient_exclusive_cql_connection(durable_node)
@@ -93,15 +80,17 @@ class TestConfiguration(Tester):
durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
int, b int, c int)')
logger.debug('commitlog size diff = ' +
str(commitlog_size(durable_node) - durable_init_size))
write_to_trigger_fsync(durable_session, 'ks', 'tab')
+logger.debug('commitlog size diff = ' +
str(commitlog_size(durable_node) - durable_init_size))
assert commitlog_size(durable_node) > durable_init_size, \
"This test will not work in this environment;
write_to_trigger_fsync does not trigger fsync."
-# get a fresh cluster to work on
durable_session.shutdown()
-self.fixture_dtest_setup.cleanup_and_replace_cluster()
+cluster.stop()
+cluster.clear()
-node = new_commitlog_cluster_node()
+cluster.start()
+node = cluster.nodelist()[0]
init_size = commitlog_size(node)
session = self.patient_exclusive_cql_connection(node)
@@ -109,9 +98,8 @@ class TestConfiguration(Tester):
session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
'SimpleStrategy', 'replication_factor': 1} "
"AND DURABLE_WRITES = false")
session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
int, c int)')
-session.execute('ALTER KEYSPACE ks WITH DURABLE_WRITES=true')
write_to_trigger_fsync(session, 'ks', 'tab')
-assert commitlog_size(node) > init_size, "ALTER KEYSPACE was not
respected"
+assert commitlog_size(node) == init_size, "Commitlog was written with
durable writes disabled"
def test_relative_paths(self):
"""
@@ -194,3 +182,16 @@ def commitlog_size(node):
commitlog_size_mbean = make_mbean('metrics', type='CommitLog',
name='TotalCommitLogSize')
with JolokiaAgent(node) as jmx:
return jmx.read_attribute(commitlog_size_mbean, 'Value')
+
+# not used but left for debugging
+def commitlog_size_nojmx(node):
+total = 0
[jira] [Commented] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827106#comment-17827106 ] Stefan Miklosovic commented on CASSANDRA-19470: --- +1 > TestMultiDCWriteFailures doesn't test versions past 4.0.x > - > > Key: CASSANDRA-19470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0.x, 5.x > > > I think this was disabled for future versions in CASSANDRA-17456, but there > is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19470: -- Reviewers: Stefan Miklosovic Status: Review In Progress (was: Patch Available) > TestMultiDCWriteFailures doesn't test versions past 4.0.x > - > > Key: CASSANDRA-19470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0.x, 5.x > > > I think this was disabled for future versions in CASSANDRA-17456, but there > is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-19470: -- Status: Ready to Commit (was: Review In Progress) > TestMultiDCWriteFailures doesn't test versions past 4.0.x > - > > Key: CASSANDRA-19470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0.x, 5.x > > > I think this was disabled for future versions in CASSANDRA-17456, but there > is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827105#comment-17827105
]
Stefan Miklosovic commented on CASSANDRA-19465:
---
+1
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def new_commitlog_cluster_node():
> # writes should block on commitlog fsync
> self.fixture_dtest_setup.cluster.populate(1)
> node = self.fixture_dtest_setup.cluster.nodelist()[0]
>
> self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
> use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
>
> self.fixture_dtest_setup.cluster.start()
> return node
>
> durable_node = new_commitlog_cluster_node()
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> # get a fresh cluster to work on
> durable_session.shutdown()
> self.fixture_dtest_setup.cleanup_and_replace_cluster()
>
> node = new_commitlog_cluster_node()
> init_size = commitlog_size(node)
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then alter it to use them
> session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
> 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = false")
> session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
> int, c int)')
> session.execute('ALTER KEYSPACE ks WITH DURABLE_WRITES=true')
> > write_to_trigger_fsync(session, 'ks', 'tab')
> configuration_test.py:113:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> configuration_test.py:186: in write_to_trigger_fsync
> execute_concurrent_with_args(session,
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:238: in
> execute_concurrent_with_args
> return execute_concurrent(session, zip(cycle((statement,)), parameters),
> *args, **kwargs)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:94: in
> execute_concurrent
> return executor.execute(concurrency, raise_on_first_error)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:201: in execute
> return super(ConcurrentExecutorListResults, self).execute(concurrency,
> fail_fast)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:120: in execute
> return self._results()
>
[jira] [Updated] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19470: - Test and Documentation Plan: run CI Status: Patch Available (was: Open) > TestMultiDCWriteFailures doesn't test versions past 4.0.x > - > > Key: CASSANDRA-19470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0.x, 5.x > > > I think this was disabled for future versions in CASSANDRA-17456, but there > is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827097#comment-17827097 ] Brandon Williams commented on CASSANDRA-19469: -- The easy way to do this is run 'pip freeze' in a new environment, then patch that output into requirements.txt > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19465:
-
Test and Documentation Plan: run CI
Status: Patch Available (was: Open)
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def new_commitlog_cluster_node():
> # writes should block on commitlog fsync
> self.fixture_dtest_setup.cluster.populate(1)
> node = self.fixture_dtest_setup.cluster.nodelist()[0]
>
> self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
> use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
>
> self.fixture_dtest_setup.cluster.start()
> return node
>
> durable_node = new_commitlog_cluster_node()
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> # get a fresh cluster to work on
> durable_session.shutdown()
> self.fixture_dtest_setup.cleanup_and_replace_cluster()
>
> node = new_commitlog_cluster_node()
> init_size = commitlog_size(node)
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then alter it to use them
> session.execute("CREATE KEYSPACE ks WITH REPLICATION = {'class':
> 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = false")
> session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a int, b
> int, c int)')
> session.execute('ALTER KEYSPACE ks WITH DURABLE_WRITES=true')
> > write_to_trigger_fsync(session, 'ks', 'tab')
> configuration_test.py:113:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> configuration_test.py:186: in write_to_trigger_fsync
> execute_concurrent_with_args(session,
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:238: in
> execute_concurrent_with_args
> return execute_concurrent(session, zip(cycle((statement,)), parameters),
> *args, **kwargs)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:94: in
> execute_concurrent
> return executor.execute(concurrency, raise_on_first_error)
> ../env3.8/src/cassandra-driver/cassandra/concurrent.py:201: in execute
> return super(ConcurrentExecutorListResults, self).execute(concurrency,
> fail_fast)
>
[jira] [Comment Edited] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827096#comment-17827096 ] Stefan Miklosovic edited comment on CASSANDRA-19469 at 3/14/24 1:44 PM: OK so until CASSANDRA-19464 is resolved to update to 8.1.1 (flaky module does not work with it), then we should change >=6.5.0,<=8.0.2 just to 8.0.2 here https://github.com/apache/cassandra-dtest/blob/trunk/requirements.txt#L18 I am not sure what to do about "six" module, we might look what a CircleCI job resolves and pin it to it too. was (Author: smiklosovic): OK so until CASSANDRA-19464 is resolved to update to 8.1.1 (flaky module does not work with it), then we should change >=6.5.0,=8.0.2 just to 8.0.2 here https://github.com/apache/cassandra-dtest/blob/trunk/requirements.txt#L18 I am not sure what to do about "six" module, we might look what a CircleCI job resolves and pin it to it too. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827096#comment-17827096 ] Stefan Miklosovic commented on CASSANDRA-19469: --- OK so until CASSANDRA-19464 is resolved to update to 8.1.1 (flaky module does not work with it), then we should changed >=6.5.0,=8.0.2 just to 8.0.2 here https://github.com/apache/cassandra-dtest/blob/trunk/requirements.txt#L18 I am not sure what to do about "six" module, we might look what a CircleCI job resolves and pin it to it too. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827096#comment-17827096 ] Stefan Miklosovic edited comment on CASSANDRA-19469 at 3/14/24 1:44 PM: OK so until CASSANDRA-19464 is resolved to update to 8.1.1 (flaky module does not work with it), then we should change >=6.5.0,=8.0.2 just to 8.0.2 here https://github.com/apache/cassandra-dtest/blob/trunk/requirements.txt#L18 I am not sure what to do about "six" module, we might look what a CircleCI job resolves and pin it to it too. was (Author: smiklosovic): OK so until CASSANDRA-19464 is resolved to update to 8.1.1 (flaky module does not work with it), then we should changed >=6.5.0,=8.0.2 just to 8.0.2 here https://github.com/apache/cassandra-dtest/blob/trunk/requirements.txt#L18 I am not sure what to do about "six" module, we might look what a CircleCI job resolves and pin it to it too. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827092#comment-17827092
]
Brandon Williams commented on CASSANDRA-19465:
--
To solve these issues:
* pin python packages (CASSANDRA-19469)
* allow oversized mutation test to run everywhere (CASSANDRA-19470)
* rewrite test to actually test durable writes
For that last point, the test was doing normal writes, tearing down the cluster
and rebuilding without durable writes, enabling durable writes, and testing
normal writes again.
I think it makes more sense to leave durable writes disabled and ensure the
commitlog was not written, which I've done
[here|https://github.com/driftx/cassandra-dtest/commit/2305bd2c819fc80f552c9a6dd3861ed6a961060a],
as well as removing the unneeded complexity of fixtures.
||Branch||CI||
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-19465-4.0]|[repeats|https://app.circleci.com/pipelines/github/driftx/cassandra/1514/workflows/c85ba6e7-1657-4d9e-8e52-480aa9d346a6/jobs/76850]|
|[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-19465-4.1]|[repeats|https://app.circleci.com/pipelines/github/driftx/cassandra/1515/workflows/0c650aed-cc33-47f7-b05f-e7c2976e314f/jobs/76851]|
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-19465-5.0]|[repeats|https://app.circleci.com/pipelines/github/driftx/cassandra/1513/workflows/31639484-078b-4dbb-85a9-b36b60cf13bf/jobs/76852]|
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def new_commitlog_cluster_node():
> # writes should block on commitlog fsync
> self.fixture_dtest_setup.cluster.populate(1)
> node = self.fixture_dtest_setup.cluster.nodelist()[0]
>
> self.fixture_dtest_setup.cluster.set_batch_commitlog(enabled=True,
> use_batch_window = self.fixture_dtest_setup.cluster.version() < '5.0')
>
> self.fixture_dtest_setup.cluster.start()
> return node
>
> durable_node = new_commitlog_cluster_node()
> durable_init_size = commitlog_size(durable_node)
> durable_session = self.patient_exclusive_cql_connection(durable_node)
>
> # test assumption that write_to_trigger_fsync actually triggers a
> commitlog fsync
> durable_session.execute("CREATE KEYSPACE ks WITH REPLICATION =
> {'class': 'SimpleStrategy', 'replication_factor': 1} "
> "AND DURABLE_WRITES = true")
> durable_session.execute('CREATE TABLE ks.tab (key int PRIMARY KEY, a
> int, b int, c int)')
> logger.debug('commitlog size diff = ' +
> str(commitlog_size(durable_node) - durable_init_size))
> write_to_trigger_fsync(durable_session, 'ks', 'tab')
>
> assert commitlog_size(durable_node) > durable_init_size, \
> "This test will not work in this environment;
> write_to_trigger_fsync does not trigger fsync."
>
> # get a fresh cluster to work on
> durable_session.shutdown()
> self.fixture_dtest_setup.cleanup_and_replace_cluster()
>
> node = new_commitlog_cluster_node()
> init_size = commitlog_size(node)
> session = self.patient_exclusive_cql_connection(node)
>
> # set up a keyspace without durable writes, then
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827086#comment-17827086 ] Ekaterina Dimitrova commented on CASSANDRA-19469: - I suggest we just update them when we add new Python versions or if something breaks, as you said - when we update the test images, for example. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827085#comment-17827085 ] Stefan Miklosovic edited comment on CASSANDRA-19469 at 3/14/24 12:58 PM: - Yes, agree ... I think it is a bad idea to use range versions here. By the same logic, why not to have junit of 4.x on ranges too? Can't hurt, right? I think everything should be explicitly versioned otherwise we are on the quicksand here. was (Author: smiklosovic): Yes, agree ... I think this a bad idea to use range versions here. By the same logic, why not to have junit of 4.x on ranges too? Can't hurt, right? I think everything should be explicitly versioned otherwise we are on the quicksand here. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827085#comment-17827085 ] Stefan Miklosovic commented on CASSANDRA-19469: --- Yes, agree ... I think this a bad idea to use range versions here. By the same logic, why not to have junit of 4.x on ranges too? Can't hurt, right? I think everything should be explicitly versioned otherwise we are on the quicksand here. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827084#comment-17827084 ] Brandon Williams commented on CASSANDRA-19469: -- I think the differentiating factor will be in the complexity of the failure modes. When packages update themselves, we get stymied with strange problems like CASSANDRA-19464 or CASSANDRA-19465. This can be further compounded by different environments having different package versions with no obvious indication. When packages are pinned, they should fail mostly by the version no longer being available, which will send us a clear signal allowing us to solve the problem with far less time and effort. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827067#comment-17827067 ] Stefan Miklosovic commented on CASSANDRA-19469: --- I would not update them as long as it works. I mean ... why? This is not a production code that we need to keep updating. It if tests, it is all what matters. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827057#comment-17827057 ] Ekaterina Dimitrova commented on CASSANDRA-19469: - I am not against the idea and I see the value. Just I want to be sure we create a plan how/when we keep on updating them as we have proven record of forgetting things (just too many things to remember and do) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827054#comment-17827054 ] Brandon Williams commented on CASSANDRA-19469: -- bq. How do we mitigate the risk of forgetting to update them? We can update them as necessary. bq. How often do the different packages get new versions? That is completely unpredictable and part of the reason we need to pin them. > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827050#comment-17827050 ] Ekaterina Dimitrova commented on CASSANDRA-19469: - How do we mitigate the risk of forgetting to update them? How often do the different packages get new versions? > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19470: - Change Category: Quality Assurance Complexity: Normal Component/s: Test/dtest/python Fix Version/s: 4.1.x 5.0.x 5.x Status: Open (was: Triage Needed) > TestMultiDCWriteFailures doesn't test versions past 4.0.x > - > > Key: CASSANDRA-19470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/python >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > Fix For: 4.1.x, 5.0.x, 5.x > > > I think this was disabled for future versions in CASSANDRA-17456, but there > is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
[ https://issues.apache.org/jira/browse/CASSANDRA-19470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-19470: Assignee: Brandon Williams > TestMultiDCWriteFailures doesn't test versions past 4.0.x > - > > Key: CASSANDRA-19470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 > Project: Cassandra > Issue Type: Improvement >Reporter: Brandon Williams >Assignee: Brandon Williams >Priority: Normal > > I think this was disabled for future versions in CASSANDRA-17456, but there > is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19470) TestMultiDCWriteFailures doesn't test versions past 4.0.x
Brandon Williams created CASSANDRA-19470: Summary: TestMultiDCWriteFailures doesn't test versions past 4.0.x Key: CASSANDRA-19470 URL: https://issues.apache.org/jira/browse/CASSANDRA-19470 Project: Cassandra Issue Type: Improvement Reporter: Brandon Williams I think this was disabled for future versions in CASSANDRA-17456, but there is no need to do this as I have a patch that will work for all versions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear Impact(13164) Complexity: Normal Component/s: Test/dtest/python Discovered By: User Report Fix Version/s: 3.0.x 3.11.x 4.0.x 4.1.x 5.0.x 5.x Severity: Normal Status: Open (was: Triage Needed) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 how allowing modules to automatically > upgrade themselves causes us problems. We should pin all of these to > specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19469) python dtest packages should pin versions
[ https://issues.apache.org/jira/browse/CASSANDRA-19469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-19469: - Description: We have for example in CASSANDRA-19464 seen how allowing modules to automatically upgrade themselves causes us problems. We should pin all of these to specific versions so that the environment is entirely reproducible. (was: We have for example in CASSANDRA-19464 how allowing modules to automatically upgrade themselves causes us problems. We should pin all of these to specific versions so that the environment is entirely reproducible.) > python dtest packages should pin versions > - > > Key: CASSANDRA-19469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest/python >Reporter: Brandon Williams >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x > > > We have for example in CASSANDRA-19464 seen how allowing modules to > automatically upgrade themselves causes us problems. We should pin all of > these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19469) python dtest packages should pin versions
Brandon Williams created CASSANDRA-19469: Summary: python dtest packages should pin versions Key: CASSANDRA-19469 URL: https://issues.apache.org/jira/browse/CASSANDRA-19469 Project: Cassandra Issue Type: Bug Reporter: Brandon Williams We have for example in CASSANDRA-19464 how allowing modules to automatically upgrade themselves causes us problems. We should pin all of these to specific versions so that the environment is entirely reproducible. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827038#comment-17827038
]
Stefan Miklosovic edited comment on CASSANDRA-19465 at 3/14/24 10:35 AM:
-
So we made some progress after all when it comes to test itself at least. The
problem is that, we do not know exactly when yet, pytest broke this thing (1).
It seems that when the original patch arrives here (2) what we see from the
logs, with help of Brandon's debugging changes, it will have a commit log
segment of size 32MB instead of 1MB. Huh right? Yesterday the test was
privately rewritten in such a way that (1) is removed and we just simply set
1MB manually, it just all passes (3).
I think what happened is that as we had a range version for pytest in
requirements.txt like >= 6.5.0, CircleCI was constantly resolving the latest
pytest available because it was downloading it all the time from scratch, but
if somebody had already installed pytest of version >=6.5.0, then whatever
goes, and that version might still have unbroken integration with the fixture
in (1). Brandon confirmed that had locally pytest of 7.4.4 where this fixture
thing was still doing its job.
I think we should not have ranges in requirements.txt and it should be on some
concrete version, like 8.0.2 for now, to avoid differences like this.
When it comes to the error itself, that mutation was not oversized because it
would throw way before that, as Brandon already mentioned. The exception is
thrown here (4) in
{code}
buffer.duplicate().position(position).limit(position + size));
{code}
as it says that position is > then limit in ByteBuffer (newPosition > limit:
(1048644 > 1048576)). This should never happen in the first place because in
the method in (4), we are trying to allocate some space in a commit log:
{code}
int position = allocate(size);
if (position < 0)
{
opGroup.close();
return null;
}
{code}
and the deal here is that if position is -1, saying it was not able to allocate
it because it would not fit into the current segment, then it would return null
and it keeps trying (5) until it can put it into a new segment.
I personally think that as it can not be oversize (it would throw) there is
some bug in (4), but given the fact that this was in production for years
without any problem, maybe it had something to do with the fact that we were
not actually setting the commit log segment size to 1MB properly for both cases
in the test ... or if the fixture was broken, that it was not clearing
commitlogs for the second node etc ...
(1)
https://github.com/apache/cassandra-dtest/blob/trunk/configuration_test.py#L18-L23
(2)
https://github.com/apache/cassandra-dtest/blob/trunk/configuration_test.py#L101-L114
(3)
https://app.circleci.com/pipelines/github/driftx/cassandra/1512/workflows/eb88ec9d-8ef7-41e0-b1d9-7ea6eafe9a80/jobs/76562
(4)
https://github.com/apache/cassandra/blob/cassandra-5.0/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java#L216
(5)
https://github.com/apache/cassandra/blob/cassandra-5.0/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerStandard.java#L52
was (Author: smiklosovic):
So we made some progress after all when it comes to test itself at least. The
problem is that, we do not know exactly when yet, pytest broke this thing (1).
It seems that when the original patch arrives here (2) what we see from the
logs, with help of Brandon's debugging changes, it will have a commit log
segment of size 32MB instead of 1MB. Huh right? Yesterday the test was
privately rewritten in such a way that (1) is removed and we just simply set
1MB manually, it just all passes (3).
I think what happened is that as we had a range version for pytest in
requirements.txt like >= 6.5.0, CircleCI was constantly resolving the latest
pytest available because it was downloading it all the time from scratch, but
if somebody had already installed pytest of version >=6.5.0, then whatever
goes, and that version might still have unbroken integration with the fixture
in (1). Brandon confirmed that had locally pytest of 7.4.4 where this fixture
thing was still doing its job.
I think we should not have ranges in requirements.txt and it should be on some
concrete version, like 8.0.2 for now, to avoid differences like this.
When it comes to the error itself, that mutation was not oversized because it
would throw way before that, as Brandon already mentioned. The exception is
thrown here (4) in
{code}
buffer.duplicate().position(position).limit(position + size));
{code}
as it says that position is > then limit in ByteBuffer (newPosition > limit:
(1048644 > 1048576)). This should never happen in the first place because in
the method in (4), we are trying to allocate some
[jira] [Commented] (CASSANDRA-19465) Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17827038#comment-17827038
]
Stefan Miklosovic commented on CASSANDRA-19465:
---
So we made some progress after all when it comes to test itself at least. The
problem is that, we do not know exactly when yet, pytest broke this thing (1).
It seems that when the original patch arrives here (2) what we see from the
logs, with help of Brandon's debugging changes, it will have a commit log
segment of size 32MB instead of 1MB. Huh right? Yesterday the test was
privately rewritten in such a way that (1) is removed and we just simply set
1MB manually, it just all passes (3).
I think what happened is that as we had a range version for pytest in
requirements.txt like >= 6.5.0, CircleCI was constantly resolving the latest
pytest available because it was downloading it all the time from scratch, but
if somebody had already installed pytest of version >=6.5.0, then whatever
goes, and that version might still have unbroken integration with the fixture
in (1). Brandon confirmed that had locally pytest of 7.4.4 where this fixture
thing was still doing its job.
I think we should not have ranges in requirements.txt and it should be on some
concrete version, like 8.0.2 for now, to avoid differences like this.
When it comes to the error itself, that mutation was not oversized because it
would throw way before that, as Brandon already mentioned. The exception is
thrown here (4) in
{code}
buffer.duplicate().position(position).limit(position + size));
{code}
as it says that position is > then limit in ByteBuffer (newPosition > limit:
(1048644 > 1048576)). This should never happen in the first place because in
the method in (4), we are trying to allocate some space in a commit log:
{code}
int position = allocate(size);
if (position < 0)
{
opGroup.close();
return null;
}
{code}
and the deal here is that if position is -1, saying it was not able to allocate
it because it would not fit into the current segment, then it would return null
and it keeps trying (5) until it can put it into a new segment.
I personally think that as it can not be oversize (it would throw) there is
some bug in (4), but given the fact that this was in production for years
without any problem, maybe it had something to do with the fact that we were
not actually setting the commit log segment size to 1MB properly for both cases
in the test ... of if the fixture was broken, that it was not clearing
commitlogs for the second node etc ...
(1)
https://github.com/apache/cassandra-dtest/blob/trunk/configuration_test.py#L18-L23
(2)
https://github.com/apache/cassandra-dtest/blob/trunk/configuration_test.py#L101-L114
(3)
https://app.circleci.com/pipelines/github/driftx/cassandra/1512/workflows/eb88ec9d-8ef7-41e0-b1d9-7ea6eafe9a80/jobs/76562
(4)
https://github.com/apache/cassandra/blob/cassandra-5.0/src/java/org/apache/cassandra/db/commitlog/CommitLogSegment.java#L216
(5)
https://github.com/apache/cassandra/blob/cassandra-5.0/src/java/org/apache/cassandra/db/commitlog/CommitLogSegmentManagerStandard.java#L52
> Test Failure: configuration_test.TestConfiguration.test_change_durable_writes
> -
>
> Key: CASSANDRA-19465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19465
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> https://app.circleci.com/pipelines/github/bereng/cassandra/1181/workflows/fe2ac859-f6ba-4f1e-b0b1-e6923b16e874/jobs/39449/tests
> {noformat}
> self =
> @pytest.mark.timeout(60*30)
> def test_change_durable_writes(self):
> """
> @jira_ticket CASSANDRA-9560
>
> Test that changes to the DURABLE_WRITES option on keyspaces is
> respected in subsequent writes.
>
> This test starts by writing a dataset to a cluster and asserting that
> the commitlogs have been written to. The subsequent test depends on
> the assumption that this dataset triggers an fsync.
>
> After checking this assumption, the test destroys the cluster and
> creates a fresh one. Then it tests that DURABLE_WRITES is respected
> by:
>
> - creating a keyspace with DURABLE_WRITES set to false,
> - using ALTER KEYSPACE to set its DURABLE_WRITES option to true,
> - writing a dataset to this keyspace that is known to trigger a
> commitlog fsync,
> - asserting that the commitlog has grown in size since the data was
> written.
> """
> def
