[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[
https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700023#comment-17700023
]
Stefan Miklosovic commented on CASSANDRA-18320:
---
Actually, we do not need to wait for this to appear in 4.1.1 because this is
just test-related.
> Incompatible file system thrown while running Simulator
> ---
>
> Key: CASSANDRA-18320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18320
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> {code}
> java.io.UncheckedIOException
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831)
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047)
> at
> org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816)
> at
> org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370)
> at
> org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345)
> at
> org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
> at
> org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33)
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /cassandra/node1/commitlog
> at
> com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535)
> at
> com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517)
> at
> com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479)
> at
> com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465)
> at
> com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261)
> at java.nio.file.Files.delete(Files.java:1126)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18326) Debian package repository misconfiguration
Tibor Repasi created CASSANDRA-18326:
Summary: Debian package repository misconfiguration
Key: CASSANDRA-18326
URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
Project: Cassandra
Issue Type: Bug
Components: Packaging
Reporter: Tibor Repasi
Debian apt is failing on current jfrog repository access for 40x releases with:
{code}
W: Conflicting distribution: https://debian.cassandra.apache.org 40x InRelease
(expected 40x but got 40)
E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
'Codename' value from '40x' to '40'
N: This must be accepted explicitly before updates for this repository can be
applied. See apt-secure(8) manpage for details.
{code}
This is caused by the typo in
[dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
containing
{code}
Codename: 40
{code}
but it is expected to be
{code}
Codename: 40x
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski commented on CASSANDRA-18143: --- 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 4.0 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18143: -- Reviewers: Jacek Lewandowski, Stefan Miklosovic, Stefan Miklosovic (was: Jacek Lewandowski) Jacek Lewandowski, Stefan Miklosovic, Stefan Miklosovic (was: Jacek Lewandowski, Stefan Miklosovic) Status: Review In Progress (was: Patch Available) > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18143: -- Test and Documentation Plan: ci Status: Patch Available (was: In Progress) > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 10:00 AM: - 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 4.1 5.0 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 4.0 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700067#comment-17700067
]
Stefan Miklosovic commented on CASSANDRA-18326:
---
[~brandon.williams] would you help us with this one, please?
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Priority: Normal
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 10:31 AM: - 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 5.0 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18037) Use snake case for the names CQL native functions
[ https://issues.apache.org/jira/browse/CASSANDRA-18037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18037: -- Change Category: Semantic Complexity: Normal Status: Open (was: Triage Needed) > Use snake case for the names CQL native functions > - > > Key: CASSANDRA-18037 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18037 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Syntax >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > > Most native functions are named all lower case, without underscore nor > hyphen to separate words. That's the case, for example, of "intasblob" or > "blobasint". > We also have some functions using camel case, as in "castAsInt" or > "castAsTimestamp". Note that the came cased names require quoting due to > CQL's case insensitivity. > Differently to CQL native functions, system keyspaces, tables and columns > consistently use snake case. For example, we have "system_schema", > "dropped_columns", "default_time_to_live". > As discussed in [this > thread|https://lists.apache.org/thread/k9ml1k4fg6o7mfby1nr3y0mnq9r90dym], we > should adopt snake_case for CQL native function names. Also we should provide > aliases for the current function names, so we don't break compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-18326:
Assignee: Brandon Williams
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700095#comment-17700095
]
Brandon Williams commented on CASSANDRA-18326:
--
Yep, this is my fault my manually uploading the packages in CASSANDRA-18307. I
thought I had fixed everything from the '40' mistake, but it looks like I
missed this.
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700095#comment-17700095
]
Brandon Williams edited comment on CASSANDRA-18326 at 3/14/23 11:05 AM:
Yep, this is my fault from my manually uploading the packages in
CASSANDRA-18307. I thought I had fixed everything from the '40' mistake, but
it looks like I missed this.
was (Author: brandon.williams):
Yep, this is my fault my manually uploading the packages in CASSANDRA-18307. I
thought I had fixed everything from the '40' mistake, but it looks like I
missed this.
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[
https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700100#comment-17700100
]
Brandon Williams commented on CASSANDRA-18320:
--
This looks good to me and I can unsurprisingly no longer reproduce, +1.
> Incompatible file system thrown while running Simulator
> ---
>
> Key: CASSANDRA-18320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18320
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> {code}
> java.io.UncheckedIOException
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831)
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047)
> at
> org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816)
> at
> org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370)
> at
> org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345)
> at
> org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
> at
> org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33)
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /cassandra/node1/commitlog
> at
> com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535)
> at
> com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517)
> at
> com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479)
> at
> com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465)
> at
> com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261)
> at java.nio.file.Files.delete(Files.java:1126)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700103#comment-17700103
]
Brandon Williams commented on CASSANDRA-18326:
--
I saw that indeed 'Release' was the old file and re-uploaded that which seems
to have solved it; I'm going to go ahead and do the whole debian directory
again since it is idempotent and I don't know if anything else didn't take.
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18327) Fix link to .tar.gz download in documentation
Arnout Engelen created CASSANDRA-18327:
--
Summary: Fix link to .tar.gz download in documentation
Key: CASSANDRA-18327
URL: https://issues.apache.org/jira/browse/CASSANDRA-18327
Project: Cassandra
Issue Type: Improvement
Components: Documentation
Reporter: Arnout Engelen
On
[https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball]
the reader is instructed to:
{{curl -OL
[http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}}
However, that mirror no longer exists.
It would be nicer to use a URL such as
[https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz],
but that requires having the current Cassandra version available as a variable
to asciidoc and inserting in into a code snippet, which I'm not sure is easy to
do.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18327) Fix link to .tar.gz download in documentation
[
https://issues.apache.org/jira/browse/CASSANDRA-18327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18327:
-
Change Category: Operability
Complexity: Normal
Status: Open (was: Triage Needed)
> Fix link to .tar.gz download in documentation
> -
>
> Key: CASSANDRA-18327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18327
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation
>Reporter: Arnout Engelen
>Priority: Normal
>
> On
> [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball]
> the reader is instructed to:
>
> {{curl -OL
> [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}}
> However, that mirror no longer exists.
> It would be nicer to use a URL such as
> [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz],
> but that requires having the current Cassandra version available as a
> variable to asciidoc and inserting in into a code snippet, which I'm not sure
> is easy to do.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18327) Fix link to .tar.gz download in documentation
[
https://issues.apache.org/jira/browse/CASSANDRA-18327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arnout Engelen updated CASSANDRA-18327:
---
Description:
On
[https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball]
the reader is instructed to:
{{curl -OL
[http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}}
However, that mirror no longer exists.
It would be nicer to use a URL such as
[https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz],
but that requires having the current Cassandra version available as a variable
to asciidoc and inserting in into a code snippet, which I'm not sure is easy to
do.
Alternatively we could point people to
[https://cassandra.apache.org/_/download.html] instead of showing a curl
command.
was:
On
[https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball]
the reader is instructed to:
{{curl -OL
[http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}}
However, that mirror no longer exists.
It would be nicer to use a URL such as
[https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz],
but that requires having the current Cassandra version available as a variable
to asciidoc and inserting in into a code snippet, which I'm not sure is easy to
do.
> Fix link to .tar.gz download in documentation
> -
>
> Key: CASSANDRA-18327
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18327
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation
>Reporter: Arnout Engelen
>Priority: Normal
>
> On
> [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball]
> the reader is instructed to:
>
> {{curl -OL
> [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}}
> However, that mirror no longer exists.
> It would be nicer to use a URL such as
> [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz],
> but that requires having the current Cassandra version available as a
> variable to asciidoc and inserting in into a code snippet, which I'm not sure
> is easy to do.
> Alternatively we could point people to
> [https://cassandra.apache.org/_/download.html] instead of showing a curl
> command.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700114#comment-17700114
]
Brandon Williams commented on CASSANDRA-18326:
--
bq. do the whole debian directory
This is done now, [~rtib] please confirm everything works for you.
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18326:
-
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Discovered By: User Report
Fix Version/s: NA
Severity: Normal
Status: Open (was: Triage Needed)
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: NA
>
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
Andres de la Peña created CASSANDRA-18328:
-
Summary: Remove deprecated CQL functions dateof and unixtimestampof
Key: CASSANDRA-18328
URL: https://issues.apache.org/jira/browse/CASSANDRA-18328
Project: Cassandra
Issue Type: Task
Components: CQL/Syntax
Reporter: Andres de la Peña
Assignee: Andres de la Peña
The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on
Cassandra
2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa],
almost eight years ago. They were deprecated in favour of the then new
{{totimestamp}} and {{tounixtimestamp}} functions.
A note about their deprecation was added to
[{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423],
and they were marked as deprecated on
[{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions].
They are also listed as deprecated on [the new
doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions].
We can finally remove those functions in 5.0, since they have been deprecated
for so long.
Discussion thread
[here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[
https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18328:
--
Fix Version/s: 5.x
> Remove deprecated CQL functions dateof and unixtimestampof
> --
>
> Key: CASSANDRA-18328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18328
> Project: Cassandra
> Issue Type: Task
> Components: CQL/Syntax
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
>
> The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on
> Cassandra
> 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa],
> almost eight years ago. They were deprecated in favour of the then new
> {{totimestamp}} and {{tounixtimestamp}} functions.
> A note about their deprecation was added to
> [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423],
> and they were marked as deprecated on
> [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions].
> They are also listed as deprecated on [the new
> doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions].
> We can finally remove those functions in 5.0, since they have been deprecated
> for so long.
> Discussion thread
> [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[
https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18328:
--
Epic Link: CASSANDRA-18306
> Remove deprecated CQL functions dateof and unixtimestampof
> --
>
> Key: CASSANDRA-18328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18328
> Project: Cassandra
> Issue Type: Task
> Components: CQL/Syntax
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
>
> The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on
> Cassandra
> 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa],
> almost eight years ago. They were deprecated in favour of the then new
> {{totimestamp}} and {{tounixtimestamp}} functions.
> A note about their deprecation was added to
> [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423],
> and they were marked as deprecated on
> [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions].
> They are also listed as deprecated on [the new
> doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions].
> We can finally remove those functions in 5.0, since they have been deprecated
> for so long.
> Discussion thread
> [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700139#comment-17700139
]
Tibor Repasi commented on CASSANDRA-18326:
--
Hi [~brandon.williams] , it looks good to me. Apt update took it and recent
version is available now.
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: NA
>
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18326) Debian package repository misconfiguration
[
https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18326:
-
Resolution: Fixed
Status: Resolved (was: Open)
Thanks for confirming!
> Debian package repository misconfiguration
> --
>
> Key: CASSANDRA-18326
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18326
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Tibor Repasi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: NA
>
>
> Debian apt is failing on current jfrog repository access for 40x releases
> with:
> {code}
> W: Conflicting distribution: https://debian.cassandra.apache.org 40x
> InRelease (expected 40x but got 40)
> E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its
> 'Codename' value from '40x' to '40'
> N: This must be accepted explicitly before updates for this repository can be
> applied. See apt-secure(8) manpage for details.
> {code}
> This is caused by the typo in
> [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release]
> containing
> {code}
> Codename: 40
> {code}
> but it is expected to be
> {code}
> Codename: 40x
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 12:46 PM: - 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[
https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700151#comment-17700151
]
Jacek Lewandowski commented on CASSANDRA-18153:
---
[~abonacin] can you provide a full reproduce example?
> Memtable being flushed without hostId in version "me" and newer during
> CommitLogReplay
> --
>
> Key: CASSANDRA-18153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18153
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Adriano Bonacin
>Assignee: Adriano Bonacin
>Priority: Normal
> Time Spent: 50m
> Remaining Estimate: 0h
>
> On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store
> HostID in the new "me" SSTable version.
> But SSTables flushed during CommitLogReplay miss this HostID info.
>
> In the next Cassandra startup, if these SSTables were still present,
> system.log will show:
> {{WARN Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored}}
> {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }}
>
> And debug.log will show a list of SSTables, witch can include "md" and "me"
> version (before upgradesstables):
>
> {{Ignored commitLogIntervals from the following sstables:
> [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}}
>
> https://issues.apache.org/jira/browse/CASSANDRA-16619
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18329) Upgrade jamm
Ekaterina Dimitrova created CASSANDRA-18329: --- Summary: Upgrade jamm Key: CASSANDRA-18329 URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 Project: Cassandra Issue Type: Task Reporter: Ekaterina Dimitrova Jamm is currently under maintenance that will solve JDK11 issues and enable it to work with post JDK11+ versions up to JDK17. This ticket will serve as a placeholder for upgrading Jamm in Cassandra when the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18329) Upgrade jamm
[ https://issues.apache.org/jira/browse/CASSANDRA-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-18329: Change Category: Operability Complexity: Normal Component/s: Jamm Status: Open (was: Triage Needed) > Upgrade jamm > > > Key: CASSANDRA-18329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 > Project: Cassandra > Issue Type: Task > Components: Jamm >Reporter: Ekaterina Dimitrova >Priority: Normal > > Jamm is currently under maintenance that will solve JDK11 issues and enable > it to work with post JDK11+ versions up to JDK17. > This ticket will serve as a placeholder for upgrading Jamm in Cassandra when > the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18329) Upgrade jamm
[ https://issues.apache.org/jira/browse/CASSANDRA-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-18329: Epic Link: CASSANDRA-16895 > Upgrade jamm > > > Key: CASSANDRA-18329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 > Project: Cassandra > Issue Type: Task > Components: Jamm >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > Jamm is currently under maintenance that will solve JDK11 issues and enable > it to work with post JDK11+ versions up to JDK17. > This ticket will serve as a placeholder for upgrading Jamm in Cassandra when > the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18329) Upgrade jamm
[ https://issues.apache.org/jira/browse/CASSANDRA-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-18329: Fix Version/s: 4.0.x 4.1.x 5.x > Upgrade jamm > > > Key: CASSANDRA-18329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 > Project: Cassandra > Issue Type: Task > Components: Jamm >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > Jamm is currently under maintenance that will solve JDK11 issues and enable > it to work with post JDK11+ versions up to JDK17. > This ticket will serve as a placeholder for upgrading Jamm in Cassandra when > the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700164#comment-17700164 ] Stefan Miklosovic commented on CASSANDRA-18143: --- All good. 4.0 should be restarted though. On successful build there I am +1 and we can commit. Thanks. > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 1:35 PM: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a, https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/5ed83a49-a560-46ae-a8db-708a55082a66 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700171#comment-17700171 ] Jacek Lewandowski commented on CASSANDRA-18143: --- 4.0 unit tests retried and passed. All looking good, merging, thanks for the review > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-18143: -- Status: Ready to Commit (was: Review In Progress) > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 1:36 PM: 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a, https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/5ed83a49-a560-46ae-a8db-708a55082a66 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 was (Author: jlewandowski): 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a, https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/5ed83a49-a560-46ae-a8db-708a55082a66 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (8ba3888fcf -> deb8abd2fd)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 8ba3888fcf Merge branch 'cassandra-4.0' into cassandra-4.1 add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 add 5104008db9 Merge branch 'cassandra-3.11' into cassandra-4.0 add deb8abd2fd Merge branch 'cassandra-4.0' into cassandra-4.1 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (02048cffea -> 8df69c8016)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 02048cffea Remove -l / -m / -h designation and have two options: free or paid tier circle config add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.1' into trunk
This is an automated email from the ASF dual-hosted git repository.
jlewandowski pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit d1ba381d96cf1f864f1f9d0c59c5efa4cf660f49
Merge: 7ad746cc83 deb8abd2fd
Author: Jacek Lewandowski
AuthorDate: Tue Mar 14 15:19:22 2023 +0100
Merge branch 'cassandra-4.1' into trunk
* cassandra-4.1:
Fix the ordering of sstables when running sstableupgrade tool
CHANGES.txt | 1 +
src/java/org/apache/cassandra/db/Directories.java | 13 +
src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++--
3 files changed, 16 insertions(+), 2 deletions(-)
diff --cc src/java/org/apache/cassandra/db/Directories.java
index cf07e995ac,9f80ce0619..e7c2535a6e
--- a/src/java/org/apache/cassandra/db/Directories.java
+++ b/src/java/org/apache/cassandra/db/Directories.java
@@@ -59,17 -49,11 +59,18 @@@ import org.apache.cassandra.db.lifecycl
import org.apache.cassandra.io.FSDiskFullWriteError;
import org.apache.cassandra.io.FSError;
import org.apache.cassandra.io.FSNoDiskAvailableForWriteError;
+import org.apache.cassandra.io.FSReadError;
import org.apache.cassandra.io.FSWriteError;
+import org.apache.cassandra.io.sstable.Component;
+import org.apache.cassandra.io.sstable.Descriptor;
+import org.apache.cassandra.io.sstable.SSTable;
+import org.apache.cassandra.io.sstable.SSTableId;
++import org.apache.cassandra.io.sstable.SSTableIdFactory;
+import org.apache.cassandra.io.util.File;
+import org.apache.cassandra.io.util.FileStoreUtils;
import org.apache.cassandra.io.util.FileUtils;
-import org.apache.cassandra.io.sstable.*;
-import org.apache.cassandra.schema.SchemaConstants;
import org.apache.cassandra.io.util.PathUtils;
+import org.apache.cassandra.schema.SchemaConstants;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.service.snapshot.SnapshotManifest;
import org.apache.cassandra.service.snapshot.TableSnapshot;
diff --cc src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
index 657ec05d65,a7c099c7e5..a7a9dcdbb4
--- a/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
+++ b/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java
@@@ -84,11 -76,11 +84,11 @@@ public class StandaloneUpgrade
Collection readers = new ArrayList<>();
- // Upgrade sstables
- for (Map.Entry> entry :
lister.list().entrySet())
+ // Upgrade sstables in id order
+ for (Map.Entry> entry :
lister.sortedList())
{
Set components = entry.getValue();
-if (!components.contains(Component.DATA) ||
!components.contains(Component.PRIMARY_INDEX))
+if
(!components.containsAll(entry.getKey().getFormat().primaryComponents()))
continue;
try
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (7ad746cc83 -> d1ba381d96)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 7ad746cc83 Revert "Switch checkstyle to j11." add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 add 5104008db9 Merge branch 'cassandra-3.11' into cassandra-4.0 add deb8abd2fd Merge branch 'cassandra-4.0' into cassandra-4.1 new d1ba381d96 Merge branch 'cassandra-4.1' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 13 + src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 16 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (4b3cb0a772 -> 58d06d34ab)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 4b3cb0a772 Merge branch 'cassandra-3.0' into cassandra-3.11 add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (3cfb222122 -> 5104008db9)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 3cfb222122 Fix BufferPool.memoryInUse counter add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 add 5104008db9 Merge branch 'cassandra-3.11' into cassandra-4.0 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-18143: -- Fix Version/s: 3.0.29 3.11.15 4.0.9 4.1.1 5.0 Since Version: 3.0.0 Source Control Link: https://github.com/apache/cassandra/commit/8df69c8016fb84e59e558e74508a0767ad3a7cc0 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed, thanks [~claude] and [~smiklosovic] > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > Fix For: 3.0.29, 3.11.15, 4.0.9, 4.1.1, 5.0 > > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[
https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henrik Ingo reassigned CASSANDRA-18260:
---
Assignee: Henrik Ingo
> Add details to Error message: Not enough space for compaction
> --
>
> Key: CASSANDRA-18260
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18260
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.x
>
>
> When compaction fails, the log message should list a) the free space
> available on disk at that point in time and b) perhaps the number and/or size
> of the source sstables being compacted.
> Free space can change from one moment to the next, so when the below
> compaction failed because it needed 161GB, upon checking the server a few
> minutes later, it had 184GB free. Similarly, the error message mentions it
> was writing out one SSTable on this STCS table, but its not clear if it was
> combining X -> 1 tables, or something else.
> [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1
> CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1
> sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in
> 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s,
> Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375.
> Partition merge counts were \{1:21375, }
> [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1
> CassandraDaemon.java:581 - Exception in thread
> Thread[CompactionExecutor:4,1,main]
> java.lang.RuntimeException: Not enough space for compaction, estimated
> sstables = 1, expected write size = 161228934093
> at
> org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386)
> at
> org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
> at
> org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77)
> at
> org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100)
> at
> org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613)
> at
> org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[
https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700237#comment-17700237
]
Jacek Lewandowski commented on CASSANDRA-18153:
---
I don't know exactly how to reproduce this, but assuming this is a possible
scenario, I want to propose a mix of something brought by all of us. In
particular:
- in all the places where we try to get host id, we try first from token
metadata and then from system.local
- if we start a fresh instance (that is, there is no commit log files to
replay), we conditionally create a host id in the system.local table and flush
that table; we do that along with storing other local node metadata, before
touching schema or the commit log
With that, we are guaranteed that host id is stored in sstable and not as a
mutation in the commit log before we actually start to replay anything.
> Memtable being flushed without hostId in version "me" and newer during
> CommitLogReplay
> --
>
> Key: CASSANDRA-18153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18153
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Adriano Bonacin
>Assignee: Adriano Bonacin
>Priority: Normal
> Time Spent: 50m
> Remaining Estimate: 0h
>
> On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store
> HostID in the new "me" SSTable version.
> But SSTables flushed during CommitLogReplay miss this HostID info.
>
> In the next Cassandra startup, if these SSTables were still present,
> system.log will show:
> {{WARN Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored}}
> {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }}
>
> And debug.log will show a list of SSTables, witch can include "md" and "me"
> version (before upgradesstables):
>
> {{Ignored commitLogIntervals from the following sstables:
> [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}}
>
> https://issues.apache.org/jira/browse/CASSANDRA-16619
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[
https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700237#comment-17700237
]
Jacek Lewandowski edited comment on CASSANDRA-18153 at 3/14/23 2:53 PM:
I don't know exactly how to reproduce this, but assuming this is a possible
scenario, I want to propose a mix of something brought by all of us. In
particular:
- in all the places where we try to get host id, we try first from token
metadata and then from system.local
- if we start a fresh instance (that is, there is no commit log files to
replay), we conditionally create a host id in the system.local table and flush
that table; we do that along with storing other local node metadata, before
touching schema or the commit log
https://issues.apache.org/jira/browse/CASSANDRA-18153 (3.11)
With that, we are guaranteed that host id is stored in sstable and not as a
mutation in the commit log before we actually start to replay anything.
was (Author: jlewandowski):
I don't know exactly how to reproduce this, but assuming this is a possible
scenario, I want to propose a mix of something brought by all of us. In
particular:
- in all the places where we try to get host id, we try first from token
metadata and then from system.local
- if we start a fresh instance (that is, there is no commit log files to
replay), we conditionally create a host id in the system.local table and flush
that table; we do that along with storing other local node metadata, before
touching schema or the commit log
With that, we are guaranteed that host id is stored in sstable and not as a
mutation in the commit log before we actually start to replay anything.
> Memtable being flushed without hostId in version "me" and newer during
> CommitLogReplay
> --
>
> Key: CASSANDRA-18153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18153
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Adriano Bonacin
>Assignee: Adriano Bonacin
>Priority: Normal
> Time Spent: 50m
> Remaining Estimate: 0h
>
> On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store
> HostID in the new "me" SSTable version.
> But SSTables flushed during CommitLogReplay miss this HostID info.
>
> In the next Cassandra startup, if these SSTables were still present,
> system.log will show:
> {{WARN Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored}}
> {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }}
>
> And debug.log will show a list of SSTables, witch can include "md" and "me"
> version (before upgradesstables):
>
> {{Ignored commitLogIntervals from the following sstables:
> [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}}
>
> https://issues.apache.org/jira/browse/CASSANDRA-16619
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[
https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700237#comment-17700237
]
Jacek Lewandowski edited comment on CASSANDRA-18153 at 3/14/23 2:53 PM:
I don't know exactly how to reproduce this, but assuming this is a possible
scenario, I want to propose a mix of something brought by all of us. In
particular:
- in all the places where we try to get host id, we try first from token
metadata and then from system.local
- if we start a fresh instance (that is, there is no commit log files to
replay), we conditionally create a host id in the system.local table and flush
that table; we do that along with storing other local node metadata, before
touching schema or the commit log
With that, we are guaranteed that host id is stored in sstable and not as a
mutation in the commit log before we actually start to replay anything.
https://github.com/apache/cassandra/pull/2215 (3.11)
was (Author: jlewandowski):
I don't know exactly how to reproduce this, but assuming this is a possible
scenario, I want to propose a mix of something brought by all of us. In
particular:
- in all the places where we try to get host id, we try first from token
metadata and then from system.local
- if we start a fresh instance (that is, there is no commit log files to
replay), we conditionally create a host id in the system.local table and flush
that table; we do that along with storing other local node metadata, before
touching schema or the commit log
https://issues.apache.org/jira/browse/CASSANDRA-18153 (3.11)
With that, we are guaranteed that host id is stored in sstable and not as a
mutation in the commit log before we actually start to replay anything.
> Memtable being flushed without hostId in version "me" and newer during
> CommitLogReplay
> --
>
> Key: CASSANDRA-18153
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18153
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Adriano Bonacin
>Assignee: Adriano Bonacin
>Priority: Normal
> Time Spent: 50m
> Remaining Estimate: 0h
>
> On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store
> HostID in the new "me" SSTable version.
> But SSTables flushed during CommitLogReplay miss this HostID info.
>
> In the next Cassandra startup, if these SSTables were still present,
> system.log will show:
> {{WARN Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored}}
> {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node;
> commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }}
>
> And debug.log will show a list of SSTables, witch can include "md" and "me"
> version (before upgradesstables):
>
> {{Ignored commitLogIntervals from the following sstables:
> [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db,
>
> /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}}
>
> https://issues.apache.org/jira/browse/CASSANDRA-16619
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: disable jdk switching
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new efdd2223 disable jdk switching
efdd2223 is described below
commit efdd222343e76a9660716af2126ef7b348e18624
Author: Brandon Williams
AuthorDate: Thu Mar 2 11:18:26 2023 -0600
disable jdk switching
Patch by brandonwilliams; reviewed by edimitrova and mck for
CASSANDRA-18106
---
upgrade_tests/upgrade_base.py | 21 -
upgrade_tests/upgrade_through_versions_test.py | 3 ---
2 files changed, 24 deletions(-)
diff --git a/upgrade_tests/upgrade_base.py b/upgrade_tests/upgrade_base.py
index dbd2fc57..30f51b25 100644
--- a/upgrade_tests/upgrade_base.py
+++ b/upgrade_tests/upgrade_base.py
@@ -17,25 +17,6 @@ from dtest import Tester, create_ks
logger = logging.getLogger(__name__)
-def switch_jdks(major_version_int):
-"""
-Changes the jdk version globally, by setting JAVA_HOME = JAVA[N]_HOME.
-This means the environment must have JAVA[N]_HOME set to switch to jdk
version N.
-"""
-new_java_home = 'JAVA{}_HOME'.format(major_version_int)
-
-try:
-os.environ[new_java_home]
-except KeyError:
-raise RuntimeError("You need to set {} to run these
tests!".format(new_java_home))
-
-# don't change if the same version was requested
-current_java_home = os.environ.get('JAVA_HOME')
-if current_java_home != os.environ[new_java_home]:
-logger.debug("Switching jdk to version {} (JAVA_HOME is changing from
{} to {})".format(major_version_int, current_java_home or 'undefined',
os.environ[new_java_home]))
-os.environ['JAVA_HOME'] = os.environ[new_java_home]
-
-
@pytest.mark.upgrade_test
@pytest.mark.skipif(sys.platform == 'win32', reason='Skip upgrade tests on
Windows')
class UpgradeTester(Tester, metaclass=ABCMeta):
@@ -73,7 +54,6 @@ class UpgradeTester(Tester, metaclass=ABCMeta):
previous_java_home = os.environ['JAVA_HOME']
previous_cassandra_version = os.environ['CASSANDRA_VERSION'] if
'CASSANDRA_VERSION' in os.environ else None
-switch_jdks(self.UPGRADE_PATH.starting_meta.java_version)
os.environ['CASSANDRA_VERSION'] = self.UPGRADE_PATH.starting_version
yield
@@ -170,7 +150,6 @@ class UpgradeTester(Tester, metaclass=ABCMeta):
node1.mark_log_for_errors()
logger.debug('upgrading node1 to
{}'.format(self.UPGRADE_PATH.upgrade_version))
-switch_jdks(self.UPGRADE_PATH.upgrade_meta.java_version)
node1.set_install_dir(version=self.UPGRADE_PATH.upgrade_version)
self.install_legacy_parsing(node1)
diff --git a/upgrade_tests/upgrade_through_versions_test.py
b/upgrade_tests/upgrade_through_versions_test.py
index d711a10b..022ccc2e 100644
--- a/upgrade_tests/upgrade_through_versions_test.py
+++ b/upgrade_tests/upgrade_through_versions_test.py
@@ -20,7 +20,6 @@ from cassandra.query import SimpleStatement
from dtest import RUN_STATIC_UPGRADE_MATRIX, Tester
from tools.misc import generate_ssl_stores, new_node
-from .upgrade_base import switch_jdks
from .upgrade_manifest import (build_upgrade_pairs,
current_2_1_x, current_2_2_x, current_3_0_x,
indev_3_11_x,
@@ -328,7 +327,6 @@ class TestUpgrade(Tester):
cluster = self.cluster
cluster.set_install_dir(version=self.test_version_metas[0].version)
self.install_nodetool_legacy_parsing()
-switch_jdks(self.test_version_metas[0].java_version)
self.fixture_dtest_setup.reinitialize_cluster_for_different_version()
logger.debug("Versions to test (%s): %s" % (type(self), str([v.version
for v in self.test_version_metas])))
@@ -508,7 +506,6 @@ class TestUpgrade(Tester):
and upgrade all nodes.
"""
logger.debug('Upgrading {nodes} to {version}'.format(nodes=[n.name for
n in nodes] if nodes is not None else 'all nodes',
version=version_meta.version))
-switch_jdks(version_meta.java_version)
logger.debug("JAVA_HOME: " + os.environ.get('JAVA_HOME'))
if not partial:
nodes = self.cluster.nodelist()
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Remove JAVA8/JAVA11_HOME from circle configs
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 5c51f05dab Remove JAVA8/JAVA11_HOME from circle configs 5c51f05dab is described below commit 5c51f05dab0562c3b95b60224e4698e2a30d2ce7 Author: Brandon Williams AuthorDate: Fri Mar 10 09:47:56 2023 -0600 Remove JAVA8/JAVA11_HOME from circle configs Patch by brandonwilliams; reviewed by edimitrova and mck for CASSANDRA-18106 --- .circleci/config.yml | 190 --- .circleci/config.yml.FREE| 190 --- .circleci/config.yml.PAID| 190 --- .circleci/config_template.yml| 2 - .circleci/config_template.yml.PAID.patch | 68 +-- 5 files changed, 34 insertions(+), 606 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index bb41b759fc..74e5d21c86 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -95,8 +95,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -186,8 +184,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -252,8 +248,6 @@ jobs: path: /tmp/cassandra/pylib environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -363,8 +357,6 @@ jobs: destination: dtest_j8_with_vnodes_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -519,8 +511,6 @@ jobs: destination: dtest_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -606,8 +596,6 @@ jobs: destination: dtest_j8_large_with_vnodes_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -725,8 +713,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -799,8 +785,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -890,8 +874,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -1001,8 +983,6 @@ jobs: destination: dtest_j11_without_vnodes_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -1093,8 +1073,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -1212,8 +1190,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR:
[jira] [Updated] (CASSANDRA-18106) Update CCM for JDK17 and revise current JDK detection strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-18106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18106: - Fix Version/s: 5.0 (was: 5.x) Source Control Link: https://github.com/apache/cassandra/commit/5c51f05dab0562c3b95b60224e4698e2a30d2ce7 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed and tagged CCM, cassandra-dtest, and the circle configs. Thanks! > Update CCM for JDK17 and revise current JDK detection strategy > -- > > Key: CASSANDRA-18106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18106 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0 > > Attachments: Screenshot 2023-03-03 at 09.24.50.png > > > As part of CASSANDRA-16895 initial POC an initial version of CCM patch was > created. This needs to be revisited and reviewed > Recently we closed CASSANDRA-18039 which brought questions, probably we need > to revise how we detect JDK versions in CCM and whether it is correct. To the > best of my knowledge there are certain tests in the repo around that and they > pass so my guess is we need to revise just the strategy and maybe document it > explicitly or consider if we want any changes to be applied. Also, we need to > be careful with breaking changes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18317) Properly synchronize CQLSSTableWriter#build on the Schema.instance
[
https://issues.apache.org/jira/browse/CASSANDRA-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-18317:
Reviewers: Alex Petrov, Yifan Cai (was: Yifan Cai)
Status: Review In Progress (was: Needs Committer)
> Properly synchronize CQLSSTableWriter#build on the Schema.instance
> --
>
> Key: CASSANDRA-18317
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18317
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> The {{CQLSSTableWriter#build}} method should properly synchronize on the
> {{Schema.instance}} class to prevent concurrent Schema operations fail, [when
> the offline tool also updates schema].
> For example, a table creation operation, which modifies the keyspaces tables
> metadata, might end up missing the update when a concurrent call to the
> {{CQLSSTableWriter#build}} method is accessing the {{Schema}} instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18330) Delivery of CEP-21: Transactional Cluster Metadata
Sam Tunnicliffe created CASSANDRA-18330: --- Summary: Delivery of CEP-21: Transactional Cluster Metadata Key: CASSANDRA-18330 URL: https://issues.apache.org/jira/browse/CASSANDRA-18330 Project: Cassandra Issue Type: Epic Reporter: Sam Tunnicliffe -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[
https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700292#comment-17700292
]
Andres de la Peña commented on CASSANDRA-18328:
---
||PR||CI||
|[trunk|https://github.com/apache/cassandra/pull/2216]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/a13238cb-37b3-44af-a5e0-38daac07b285]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/040bc564-7f99-4794-80ea-380c5a4b3123]|
|[dtests|https://github.com/apache/cassandra-dtest/pull/214]||
> Remove deprecated CQL functions dateof and unixtimestampof
> --
>
> Key: CASSANDRA-18328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18328
> Project: Cassandra
> Issue Type: Task
> Components: CQL/Syntax
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
>
> The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on
> Cassandra
> 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa],
> almost eight years ago. They were deprecated in favour of the then new
> {{totimestamp}} and {{tounixtimestamp}} functions.
> A note about their deprecation was added to
> [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423],
> and they were marked as deprecated on
> [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions].
> They are also listed as deprecated on [the new
> doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions].
> We can finally remove those functions in 5.0, since they have been deprecated
> for so long.
> Discussion thread
> [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18330) Delivery of CEP-21: Transactional Cluster Metadata
[ https://issues.apache.org/jira/browse/CASSANDRA-18330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-18330: Change Category: Semantic Complexity: Byzantine Component/s: Cluster/Membership Cluster/Schema Assignee: Sam Tunnicliffe Status: Open (was: Triage Needed) > Delivery of CEP-21: Transactional Cluster Metadata > -- > > Key: CASSANDRA-18330 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18330 > Project: Cassandra > Issue Type: Epic > Components: Cluster/Membership, Cluster/Schema >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[
https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18328:
--
Change Category: Semantic
Complexity: Low Hanging Fruit
Status: Open (was: Triage Needed)
> Remove deprecated CQL functions dateof and unixtimestampof
> --
>
> Key: CASSANDRA-18328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18328
> Project: Cassandra
> Issue Type: Task
> Components: CQL/Syntax
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
>
> The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on
> Cassandra
> 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa],
> almost eight years ago. They were deprecated in favour of the then new
> {{totimestamp}} and {{tounixtimestamp}} functions.
> A note about their deprecation was added to
> [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423],
> and they were marked as deprecated on
> [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions].
> They are also listed as deprecated on [the new
> doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions].
> We can finally remove those functions in 5.0, since they have been deprecated
> for so long.
> Discussion thread
> [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[
https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18328:
--
Test and Documentation Plan:
||PR||CI||
|[trunk|https://github.com/apache/cassandra/pull/2216]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/a13238cb-37b3-44af-a5e0-38daac07b285]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/040bc564-7f99-4794-80ea-380c5a4b3123]|
|[dtests|https://github.com/apache/cassandra-dtest/pull/214]||
Status: Patch Available (was: Open)
> Remove deprecated CQL functions dateof and unixtimestampof
> --
>
> Key: CASSANDRA-18328
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18328
> Project: Cassandra
> Issue Type: Task
> Components: CQL/Syntax
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.x
>
>
> The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on
> Cassandra
> 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa],
> almost eight years ago. They were deprecated in favour of the then new
> {{totimestamp}} and {{tounixtimestamp}} functions.
> A note about their deprecation was added to
> [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423],
> and they were marked as deprecated on
> [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions].
> They are also listed as deprecated on [the new
> doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions].
> We can finally remove those functions in 5.0, since they have been deprecated
> for so long.
> Discussion thread
> [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[
https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700303#comment-17700303
]
Caleb Rackliffe commented on CASSANDRA-18302:
-
Let's look at the particular example from
{{testSetMapElementFromMapElementReference}}, assuming the one-line fix I
mentioned
[above|https://issues.apache.org/jira/browse/CASSANDRA-18302?focusedCommentId=17699878&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17699878]
is in place...
{noformat}
SELECT int_map[?] FROM distributed_test_keyspace.tbl0 WHERE k = 0
{noformat}
...becomes...
{noformat}
SELECT int_map['one'] FROM distributed_test_keyspace.tbl0 WHERE k = 0
{noformat}
The only difference here between the original query and the {{asCQL()}} output
is the value binding for {{'one'}}. Combined w/ a line number, it seems pretty
easy to see what's going on. (In this case, the only reason I think we found
the problem is that we forgot to use {{LazyToString}} in
{{createNamedReads()}}? CC [~dcapwell]) Are there other examples where the
disconnect is more problematic?
Aside from all that, the {{asCQL()}} and {{toCQLString()}} complex has been
around for a while, and the former seems like it needs the fix above
independent of Accord. It would be nice to avoid more new complexity than
necessary, but I guess what we have to hash out is how much new complexity is
worth having error messages w/ the _exact_ CQL strings supplied by the user.
Interested in other opinions here...
> Fix AIOOBE and improve validation messages for transaction statements
> -
>
> Key: CASSANDRA-18302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18302
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently it happens sometimes that ArrayIndexOutOfBoundsException is thrown
> from asCql method when validation transaction statement. In addition, asCql
> does not return precisely the query user entered so the whole error message
> can be misleading.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[
https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700303#comment-17700303
]
Caleb Rackliffe edited comment on CASSANDRA-18302 at 3/14/23 4:43 PM:
--
Let's look at the particular example from
{{testSetMapElementFromMapElementReference}}, assuming the one-line fix I
mentioned
[above|https://issues.apache.org/jira/browse/CASSANDRA-18302?focusedCommentId=17699878&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17699878]
is in place...
{noformat}
SELECT int_map[?] FROM distributed_test_keyspace.tbl0 WHERE k = 0
{noformat}
...becomes...
{noformat}
SELECT int_map['one'] FROM distributed_test_keyspace.tbl0 WHERE k = 0
{noformat}
The only difference here between the original query and the {{asCQL()}} output
is the value binding for {{'one'}}. Combined w/ a line number, it seems pretty
easy to see what's going on. (In this case, the only reason I think we found
the problem is that we forgot to use {{LazyToString}} in
{{createNamedReads()}}? CC [~dcapwell]) Are there other examples where the
disconnect is more problematic?
Aside from all that, the {{asCQL()}} and {{toCQLString()}} complex has been
around for a while, and the former seems like it needs the fix above
independent of Accord. It would be nice to avoid more new complexity than
necessary, but I guess what we have to hash out is how much new complexity (and
up-front, non-lazy processing) is worth having error messages w/ the _exact_
CQL strings supplied by the user.
Interested in other opinions here...
was (Author: maedhroz):
Let's look at the particular example from
{{testSetMapElementFromMapElementReference}}, assuming the one-line fix I
mentioned
[above|https://issues.apache.org/jira/browse/CASSANDRA-18302?focusedCommentId=17699878&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17699878]
is in place...
{noformat}
SELECT int_map[?] FROM distributed_test_keyspace.tbl0 WHERE k = 0
{noformat}
...becomes...
{noformat}
SELECT int_map['one'] FROM distributed_test_keyspace.tbl0 WHERE k = 0
{noformat}
The only difference here between the original query and the {{asCQL()}} output
is the value binding for {{'one'}}. Combined w/ a line number, it seems pretty
easy to see what's going on. (In this case, the only reason I think we found
the problem is that we forgot to use {{LazyToString}} in
{{createNamedReads()}}? CC [~dcapwell]) Are there other examples where the
disconnect is more problematic?
Aside from all that, the {{asCQL()}} and {{toCQLString()}} complex has been
around for a while, and the former seems like it needs the fix above
independent of Accord. It would be nice to avoid more new complexity than
necessary, but I guess what we have to hash out is how much new complexity is
worth having error messages w/ the _exact_ CQL strings supplied by the user.
Interested in other opinions here...
> Fix AIOOBE and improve validation messages for transaction statements
> -
>
> Key: CASSANDRA-18302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18302
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently it happens sometimes that ArrayIndexOutOfBoundsException is thrown
> from asCql method when validation transaction statement. In addition, asCql
> does not return precisely the query user entered so the whole error message
> can be misleading.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 03/25: [CEP-21] Include current epoch in internode header
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 56d289b4fbe104cbf9a3320c0ed7b93991dcaa2d
Author: Sam Tunnicliffe
AuthorDate: Wed Mar 1 10:03:44 2023 +
[CEP-21] Include current epoch in internode header
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../org/apache/cassandra/db/CounterMutation.java | 6 +
src/java/org/apache/cassandra/db/Mutation.java | 5 +
src/java/org/apache/cassandra/net/Message.java | 130 +
.../org/apache/cassandra/net/MessagingService.java | 6 +-
4 files changed, 121 insertions(+), 26 deletions(-)
diff --git a/src/java/org/apache/cassandra/db/CounterMutation.java
b/src/java/org/apache/cassandra/db/CounterMutation.java
index 4f91b83ca2..86086902aa 100644
--- a/src/java/org/apache/cassandra/db/CounterMutation.java
+++ b/src/java/org/apache/cassandra/db/CounterMutation.java
@@ -51,6 +51,7 @@ import static java.util.concurrent.TimeUnit.*;
import static org.apache.cassandra.net.MessagingService.VERSION_30;
import static org.apache.cassandra.net.MessagingService.VERSION_3014;
import static org.apache.cassandra.net.MessagingService.VERSION_40;
+import static org.apache.cassandra.net.MessagingService.VERSION_50;
import static org.apache.cassandra.utils.Clock.Global.nanoTime;
public class CounterMutation implements IMutation
@@ -334,6 +335,7 @@ public class CounterMutation implements IMutation
private int serializedSize30;
private int serializedSize3014;
private int serializedSize40;
+private int serializedSize50;
public int serializedSize(int version)
{
@@ -351,6 +353,10 @@ public class CounterMutation implements IMutation
if (serializedSize40 == 0)
serializedSize40 = (int) serializer.serializedSize(this,
VERSION_40);
return serializedSize40;
+case VERSION_50:
+if (serializedSize50 == 0)
+serializedSize50 = (int) serializer.serializedSize(this,
VERSION_50);
+return serializedSize50;
default:
throw new IllegalStateException("Unknown serialization
version: " + version);
}
diff --git a/src/java/org/apache/cassandra/db/Mutation.java
b/src/java/org/apache/cassandra/db/Mutation.java
index ad43b16d48..9a5cdabb50 100644
--- a/src/java/org/apache/cassandra/db/Mutation.java
+++ b/src/java/org/apache/cassandra/db/Mutation.java
@@ -317,6 +317,7 @@ public class Mutation implements IMutation,
Supplier
private int serializedSize30;
private int serializedSize3014;
private int serializedSize40;
+private int serializedSize50;
public int serializedSize(int version)
{
@@ -334,6 +335,10 @@ public class Mutation implements IMutation,
Supplier
if (serializedSize40 == 0)
serializedSize40 = (int) serializer.serializedSize(this,
VERSION_40);
return serializedSize40;
+case VERSION_50:
+if (serializedSize50 == 0)
+serializedSize50 = (int) serializer.serializedSize(this,
VERSION_50);
+return serializedSize50;
default:
throw new IllegalStateException("Unknown serialization
version: " + version);
}
diff --git a/src/java/org/apache/cassandra/net/Message.java
b/src/java/org/apache/cassandra/net/Message.java
index fa14134933..03562540b4 100644
--- a/src/java/org/apache/cassandra/net/Message.java
+++ b/src/java/org/apache/cassandra/net/Message.java
@@ -25,6 +25,7 @@ import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
+import java.util.function.Supplier;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.primitives.Ints;
@@ -40,11 +41,11 @@ import org.apache.cassandra.io.util.DataInputBuffer;
import org.apache.cassandra.io.util.DataInputPlus;
import org.apache.cassandra.io.util.DataOutputPlus;
import org.apache.cassandra.locator.InetAddressAndPort;
+import org.apache.cassandra.tcm.Epoch;
+import org.apache.cassandra.tcm.ClusterMetadata;
import org.apache.cassandra.tracing.Tracing;
import org.apache.cassandra.tracing.Tracing.TraceType;
-import org.apache.cassandra.utils.MonotonicClockTranslation;
-import org.apache.cassandra.utils.NoSpamLogger;
-import org.apache.cassandra.utils.TimeUUID;
+import org.apache.cassandra.utils.*;
import static java.util.concurrent.TimeUnit.MINUTES;
import static java.util.concurrent.TimeUnit.NANOSECONDS;
@@ -67,6 +68,8 @@ public class Message
private static final Logger logger =
LoggerFactory.getLogger(Message.class);
private static final NoSpamLogger noSpam1m =
NoSpamLogger.getLogger(logger, 1, TimeUnit.MINUTES);
+pr
[cassandra] 05/25: [CEP-21] Add basics of ownership and data placement
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 2434132cc7685729b5bb24446d9257b23bc5c113
Author: Sam Tunnicliffe
AuthorDate: Wed Mar 1 12:52:29 2023 +
[CEP-21] Add basics of ownership and data placement
Introduce new classes for representing placement of data ranges on
replicas, along with the movement of data via transitions from one
placement to the next. Eventually, these placements will be statically
calculated in response to events with alter either the topology of the
cluster (i.e. adding/removing/moving nodes) or the replication profile
of the data itself (i.e. creating/altering keyspaces). These triggering
events will be distributed and enacted consistently using the global log.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../cassandra/dht/OrderPreservingPartitioner.java | 24 +
src/java/org/apache/cassandra/dht/Range.java | 27 ++
src/java/org/apache/cassandra/dht/Token.java | 27 ++
.../apache/cassandra/locator/RangesByEndpoint.java | 69 +++
.../cassandra/tcm/ownership/DataPlacement.java | 211 +
.../cassandra/tcm/ownership/DataPlacements.java| 216 +
.../org/apache/cassandra/tcm/ownership/Delta.java | 125 +
.../tcm/ownership/GlobalPlacementDelta.java| 163 +++
.../cassandra/tcm/ownership/MovementMap.java | 89
.../cassandra/tcm/ownership/PlacementDeltas.java | 244 ++
.../cassandra/tcm/ownership/PlacementForRange.java | 525 +
.../cassandra/tcm/ownership/PlacementProvider.java | 53 +++
.../tcm/ownership/PlacementTransitionPlan.java | 128 +
.../cassandra/tcm/ownership/ReplicationMap.java| 106 +
.../cassandra/tcm/sequences/LockedRanges.java | 430 +
15 files changed, 2437 insertions(+)
diff --git a/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java
b/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java
index 2d4def95d0..fb9d0011a7 100644
--- a/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java
+++ b/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java
@@ -45,6 +45,15 @@ public class OrderPreservingPartitioner implements
IPartitioner
private static final String rndchars =
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
public static final StringToken MINIMUM = new StringToken("");
+public static final StringToken MAXIMUM = new StringToken("") {
+public int compareTo(Token o)
+{
+if (o == MAXIMUM)
+return 0;
+
+return 1;
+}
+};
public static final BigInteger CHAR_MASK = new BigInteger("65535");
@@ -116,6 +125,11 @@ public class OrderPreservingPartitioner implements
IPartitioner
return MINIMUM;
}
+public StringToken getMaximumToken()
+{
+return MAXIMUM;
+}
+
public StringToken getRandomToken()
{
return getRandomToken(ThreadLocalRandom.current());
@@ -208,6 +222,16 @@ public class OrderPreservingPartitioner implements
IPartitioner
{
return ByteSource.of(token, version);
}
+
+@Override
+public int compareTo(Token o)
+{
+// todo (rebase): I have no recollection of why this is needed -
investigate
+if (o == MAXIMUM)
+return -1;
+
+return super.compareTo(o);
+}
}
public StringToken getToken(ByteBuffer key)
diff --git a/src/java/org/apache/cassandra/dht/Range.java
b/src/java/org/apache/cassandra/dht/Range.java
index 2c468990d2..581602cccd 100644
--- a/src/java/org/apache/cassandra/dht/Range.java
+++ b/src/java/org/apache/cassandra/dht/Range.java
@@ -17,6 +17,7 @@
*/
package org.apache.cassandra.dht;
+import java.io.IOException;
import java.io.Serializable;
import java.util.*;
import java.util.function.Predicate;
@@ -25,6 +26,11 @@ import com.google.common.collect.Iterables;
import org.apache.commons.lang3.ObjectUtils;
import org.apache.cassandra.db.PartitionPosition;
+import org.apache.cassandra.io.util.DataInputPlus;
+import org.apache.cassandra.io.util.DataOutputPlus;
+import org.apache.cassandra.net.MessagingService;
+import org.apache.cassandra.tcm.serialization.MetadataSerializer;
+import org.apache.cassandra.tcm.serialization.Version;
import org.apache.cassandra.utils.Pair;
/**
@@ -38,6 +44,7 @@ import org.apache.cassandra.utils.Pair;
*/
public class Range> extends AbstractBounds
implements Comparable>, Serializable
{
+public static final Serializer serializer = new Serializer();
public static final long serialVersionUID = 1L;
public Range(T left, T right)
@@ -626,4 +633,24 @@ public class Range> extends
AbstractBounds imp
[cassandra] 08/25: [CEP-21] Start to remove and deprecate gossip functionality
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 5a09d9441902bcb99051ca23bd19191ee85424f8
Author: Sam Tunnicliffe
AuthorDate: Thu Mar 2 14:00:55 2023 +
[CEP-21] Start to remove and deprecate gossip functionality
WIP commit (i.e. does not compile) beginning the process of removing
gossip as the source of truth regarding membership, ownership, topology
and data placement. This task will be split over mutiple commits.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../statements/schema/AlterKeyspaceStatement.java | 24 +-
.../apache/cassandra/db/filter/ColumnFilter.java | 98 +--
.../org/apache/cassandra/gms/ApplicationState.java | 38 +-
.../org/apache/cassandra/gms/EndpointState.java| 27 +-
.../org/apache/cassandra/gms/FailureDetector.java | 12 +-
.../org/apache/cassandra/gms/GossipDigestAck.java | 2 +-
.../cassandra/gms/GossipDigestAckVerbHandler.java | 11 +-
.../cassandra/gms/GossipDigestSynVerbHandler.java | 5 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 728 +
.../org/apache/cassandra/gms/GossiperEvent.java| 2 +-
src/java/org/apache/cassandra/gms/NewGossiper.java | 143
.../org/apache/cassandra/gms/VersionedValue.java | 96 ++-
.../schema/SystemDistributedKeyspace.java | 45 +-
.../cassandra/tcm/compatibility/GossipHelper.java | 303 +
.../apache/cassandra/tracing/TraceKeyspace.java| 14 +-
15 files changed, 774 insertions(+), 774 deletions(-)
diff --git
a/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java
b/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java
index 9efccd5464..250b18ffed 100644
---
a/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java
+++
b/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java
@@ -48,6 +48,7 @@ import org.apache.cassandra.schema.Keyspaces;
import org.apache.cassandra.schema.Keyspaces.KeyspacesDiff;
import org.apache.cassandra.service.ClientState;
import org.apache.cassandra.tcm.ClusterMetadata;
+import org.apache.cassandra.tcm.membership.NodeId;
import org.apache.cassandra.transport.Event.SchemaChange;
import org.apache.cassandra.transport.Event.SchemaChange.Change;
import org.apache.cassandra.utils.FBUtilities;
@@ -91,7 +92,7 @@ public final class AlterKeyspaceStatement extends
AlterSchemaStatement
newKeyspace.params.validate(keyspaceName, state);
newKeyspace.replicationStrategy.validate();
-
+validateNoRangeMovements();
validateTransientReplication(keyspace.replicationStrategy,
newKeyspace.replicationStrategy);
// Because we used to not properly validate unrecognized options, we
only log a warning if we find one.
@@ -140,14 +141,19 @@ public final class AlterKeyspaceStatement extends
AlterSchemaStatement
if (allow_alter_rf_during_range_movement)
return;
-Stream unreachableNotAdministrativelyInactive =
-Gossiper.instance.getUnreachableMembers().stream().filter(endpoint
-> !FBUtilities.getBroadcastAddressAndPort().equals(endpoint) &&
-
!Gossiper.instance.isAdministrativelyInactiveState(endpoint));
-Stream endpoints =
Stream.concat(Gossiper.instance.getLiveMembers().stream(),
-
unreachableNotAdministrativelyInactive);
-List notNormalEndpoints =
endpoints.filter(endpoint ->
!FBUtilities.getBroadcastAddressAndPort().equals(endpoint) &&
-
!Gossiper.instance.getEndpointStateForEndpoint(endpoint).isNormalState())
-
.collect(Collectors.toList());
+ClusterMetadata metadata = ClusterMetadata.current();
+NodeId nodeId =
metadata.directory.peerId(FBUtilities.getBroadcastAddressAndPort());
+Set notNormalEndpoints =
metadata.directory.states.entrySet().stream().filter(e ->
!e.getKey().equals(nodeId)).filter(e -> {
+switch (e.getValue())
+{
+case BOOTSTRAPPING:
+case LEAVING:
+case MOVING:
+return true;
+default:
+return false;
+}
+}).map(e ->
metadata.directory.endpoint(e.getKey())).collect(Collectors.toSet());
if (!notNormalEndpoints.isEmpty())
{
throw new ConfigurationException("Cannot alter RF while some
endpoints are not in normal state (no range movements): " + notNormalEndpoints);
diff --git a/src/java/org/apache/cassandra/db/filt
[cassandra] 13/25: [CEP-21] Update ColumnFamilyStore (4/7)
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit da5b8ec9cedc29122ec1a3e84f42f8af11f6b650
Author: Sam Tunnicliffe
AuthorDate: Thu Mar 2 19:20:39 2023 +
[CEP-21] Update ColumnFamilyStore (4/7)
Part 4 of 7 modifications to ColumnFamilyStore, mostly related to:
* ShardBoundaries
* DiskBoundaries
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../org/apache/cassandra/db/ColumnFamilyStore.java | 28 +++--
.../org/apache/cassandra/db/DiskBoundaries.java| 23 ++--
.../apache/cassandra/db/DiskBoundaryManager.java | 84 --
.../cassandra/db/memtable/ShardBoundaries.java | 13 ++-
.../org/apache/cassandra/tcm/ClusterMetadata.java | 124 +
5 files changed, 168 insertions(+), 104 deletions(-)
diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
index 0affba5814..99e0bcde68 100644
--- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
+++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java
@@ -90,14 +90,14 @@ import
org.apache.cassandra.db.compaction.CompactionStrategyManager;
import org.apache.cassandra.db.compaction.OperationType;
import org.apache.cassandra.db.filter.ClusteringIndexFilter;
import org.apache.cassandra.db.filter.DataLimits;
+import org.apache.cassandra.db.memtable.Flushing;
+import org.apache.cassandra.db.memtable.Memtable;
+import org.apache.cassandra.db.memtable.ShardBoundaries;
import org.apache.cassandra.db.lifecycle.LifecycleNewTracker;
import org.apache.cassandra.db.lifecycle.LifecycleTransaction;
import org.apache.cassandra.db.lifecycle.SSTableSet;
import org.apache.cassandra.db.lifecycle.Tracker;
import org.apache.cassandra.db.lifecycle.View;
-import org.apache.cassandra.db.memtable.Flushing;
-import org.apache.cassandra.db.memtable.Memtable;
-import org.apache.cassandra.db.memtable.ShardBoundaries;
import org.apache.cassandra.db.partitions.CachedPartition;
import org.apache.cassandra.db.partitions.PartitionUpdate;
import org.apache.cassandra.db.repair.CassandraTableRepairManager;
@@ -162,6 +162,7 @@ import
org.apache.cassandra.service.snapshot.SnapshotManifest;
import org.apache.cassandra.service.snapshot.TableSnapshot;
import org.apache.cassandra.streaming.TableStreamManager;
import org.apache.cassandra.tcm.ClusterMetadata;
+import org.apache.cassandra.tcm.Epoch;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.DefaultValue;
import org.apache.cassandra.utils.ExecutorUtils;
@@ -520,6 +521,7 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean, Memtable.Owner
data.addInitialSSTablesWithoutUpdatingSize(sstables, this);
}
+// compaction strategy should be created after the CFS has been
prepared
compactionStrategyManager = new CompactionStrategyManager(this);
if (maxCompactionThreshold.value() <= 0 ||
minCompactionThreshold.value() <=0)
@@ -1460,17 +1462,16 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean, Memtable.Owner
if (shardBoundaries == null ||
shardBoundaries.shardCount() != shardCount ||
-shardBoundaries.ringVersion != -1 && shardBoundaries.ringVersion
!= StorageService.instance.getTokenMetadata().getRingVersion())
+!shardBoundaries.epoch.equals(Epoch.EMPTY) &&
!shardBoundaries.epoch.equals(metadata.epoch))
{
List weightedRanges;
-long ringVersion;
+Epoch epoch;
if (!SchemaConstants.isLocalSystemKeyspace(keyspace.getName())
-&& getPartitioner() == metadata.current().partitioner)
+&& getPartitioner() ==
metadata.current().tokenMap.partitioner())
{
DiskBoundaryManager.VersionedRangesAtEndpoint
versionedLocalRanges = DiskBoundaryManager.getVersionedLocalRanges(this);
Set> localRanges =
versionedLocalRanges.rangesAtEndpoint.ranges();
-ringVersion = versionedLocalRanges.ringVersion;
-
+epoch = versionedLocalRanges.epoch;
if (!localRanges.isEmpty())
{
@@ -1494,12 +1495,12 @@ public class ColumnFamilyStore implements
ColumnFamilyStoreMBean, Memtable.Owner
// Local tables need to cover the full token range and don't
care about ring changes.
// We also end up here if the table's partitioner is not the
database's, which can happen in tests.
weightedRanges = fullWeightedRange();
-ringVersion = -1;
+epoch = Epoch.EMPTY;
}
List boundaries =
getPartitioner().splitter().get().splitOwnedRanges(shardCount, weightedRanges,
false);
[cassandra] 18/25: [CEP-21] Add missing implementations to concurrent utils
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit bb615495bac125b00caeea93919680dfced6fb6f
Author: Sam Tunnicliffe
AuthorDate: Fri Mar 3 16:37:47 2023 +
[CEP-21] Add missing implementations to concurrent utils
Adds a handful of implementations to subclasses in the
org.apache.cassandra.utils.concurrent package
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../cassandra/utils/concurrent/AbstractFuture.java | 33 ++
.../cassandra/utils/concurrent/AsyncFuture.java| 11
.../apache/cassandra/utils/concurrent/Future.java | 10 +++
.../cassandra/utils/concurrent/LoadingMap.java | 7 +
.../cassandra/utils/concurrent/SyncFuture.java | 11
5 files changed, 72 insertions(+)
diff --git a/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java
b/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java
index 83cd7d3f8b..a6c5aeda8a 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java
@@ -383,6 +383,39 @@ public abstract class AbstractFuture implements
Future
return result;
}
+/**
+ * Support {@link
com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture,
AsyncFunction, Executor)} natively
+ *
+ * See {@link #addListener(GenericFutureListener)} for ordering semantics.
+ */
+@Override
+public Future andThenAsync(Function>
andThen)
+{
+return andThenAsync(andThen, null);
+}
+
+/**
+ * Support {@link
com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture,
AsyncFunction, Executor)} natively
+ *
+ * See {@link #addListener(GenericFutureListener)} for ordering semantics.
+ */
+protected Future andThenAsync(AbstractFuture result, Function> andThen, @Nullable Executor executor)
+{
+addListener(() -> {
+try
+{
+if (isSuccess())
andThen.apply(getNow()).addListener(propagate(result));
+else result.tryFailure(cause());
+}
+catch (Throwable t)
+{
+result.tryFailure(t);
+throw t;
+}
+}, executor);
+return result;
+}
+
/**
* Add a listener to be invoked once this future completes.
* Listeners are submitted to {@link #notifyExecutor} in the order they
are added (or the specified executor
diff --git a/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java
b/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java
index 0ef35d5dde..89cea2dbe3 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java
@@ -144,6 +144,17 @@ public class AsyncFuture extends AbstractFuture
return flatMap(new AsyncFuture<>(), flatMapper, executor);
}
+/**
+ * Support {@link
com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture,
AsyncFunction, Executor)} natively
+ *
+ * See {@link #addListener(GenericFutureListener)} for ordering semantics.
+ */
+@Override
+public Future andThenAsync(Function>
andThen, @Nullable Executor executor)
+{
+return andThenAsync(new AsyncFuture<>(), andThen, executor);
+}
+
/**
* Wait for this future to complete {@link Awaitable#await()}
*/
diff --git a/src/java/org/apache/cassandra/utils/concurrent/Future.java
b/src/java/org/apache/cassandra/utils/concurrent/Future.java
index fae5d43ef1..909d140d17 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/Future.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/Future.java
@@ -177,6 +177,16 @@ public interface Future extends
io.netty.util.concurrent.Future, Listenabl
*/
Future flatMap(Function> flatMapper,
Executor executor);
+/**
+ * Support {@link
com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture,
AsyncFunction, Executor)} natively
+ */
+ Future andThenAsync(Function>
andThen);
+
+/**
+ * Support {@link
com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture,
AsyncFunction, Executor)} natively
+ */
+ Future andThenAsync(Function>
andThen, Executor executor);
+
/**
* Invoke {@code runnable} on completion, using {@code executor}.
*
diff --git a/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java
b/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java
index 399eb0e3b5..1d41a97db8 100644
--- a/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java
+++ b/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java
@@ -18,6
[cassandra] 17/25: [CEP-21] New nodetool commands
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 7bda86e1fe57be880155dd7cb652d415f0650f58
Author: Sam Tunnicliffe
AuthorDate: Fri Mar 3 15:41:59 2023 +
[CEP-21] New nodetool commands
Adds new nodetool commands to:
* list members of the CMS
* initiate a snapshot of ClusterMetadata via submitting a SealPeriod
operation
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
src/java/org/apache/cassandra/tools/NodeProbe.java | 5
src/java/org/apache/cassandra/tools/NodeTool.java | 2 ++
.../cassandra/tools/nodetool/DescribeCMS.java | 34 ++
.../cassandra/tools/nodetool/SealPeriod.java | 33 +
4 files changed, 74 insertions(+)
diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java
b/src/java/org/apache/cassandra/tools/NodeProbe.java
index e983e2fbef..ab051843f1 100644
--- a/src/java/org/apache/cassandra/tools/NodeProbe.java
+++ b/src/java/org/apache/cassandra/tools/NodeProbe.java
@@ -327,6 +327,11 @@ public class NodeProbe implements AutoCloseable
return ssProxy.forceKeyspaceCleanup(jobs, keyspaceName, tables);
}
+public void sealPeriod()
+{
+ssProxy.sealPeriod();
+}
+
public int scrub(boolean disableSnapshot, boolean skipCorrupted, boolean
checkData, boolean reinsertOverflowedTTL, int jobs, String keyspaceName,
String... tables) throws IOException, ExecutionException, InterruptedException
{
return ssProxy.scrub(disableSnapshot, skipCorrupted, checkData,
reinsertOverflowedTTL, jobs, keyspaceName, tables);
diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java
b/src/java/org/apache/cassandra/tools/NodeTool.java
index 5500240477..35a673c406 100644
--- a/src/java/org/apache/cassandra/tools/NodeTool.java
+++ b/src/java/org/apache/cassandra/tools/NodeTool.java
@@ -107,6 +107,7 @@ public class NodeTool
DataPaths.class,
Decommission.class,
DescribeCluster.class,
+DescribeCMS.class,
DescribeRing.class,
DisableAuditLog.class,
DisableAutoCompaction.class,
@@ -190,6 +191,7 @@ public class NodeTool
ResumeHandoff.class,
Ring.class,
Scrub.class,
+SealPeriod.class,
SetAuthCacheConfig.class,
SetBatchlogReplayThrottle.class,
SetCacheCapacity.class,
diff --git a/src/java/org/apache/cassandra/tools/nodetool/DescribeCMS.java
b/src/java/org/apache/cassandra/tools/nodetool/DescribeCMS.java
new file mode 100644
index 00..ce34af4bf8
--- /dev/null
+++ b/src/java/org/apache/cassandra/tools/nodetool/DescribeCMS.java
@@ -0,0 +1,34 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.tools.nodetool;
+
+import io.airlift.airline.Command;
+import org.apache.cassandra.tools.NodeProbe;
+import org.apache.cassandra.tools.NodeTool;
+
+@Command(name = "describecms", description = "Describe the current Cluster
Metadata Service")
+public class DescribeCMS extends NodeTool.NodeToolCmd
+{
+@Override
+protected void execute(NodeProbe probe)
+{
+System.out.println("ClusterMetadataService:");
+System.out.println(probe.getStorageService().describeCMS());
+}
+}
diff --git a/src/java/org/apache/cassandra/tools/nodetool/SealPeriod.java
b/src/java/org/apache/cassandra/tools/nodetool/SealPeriod.java
new file mode 100644
index 00..65bd699ca3
--- /dev/null
+++ b/src/java/org/apache/cassandra/tools/nodetool/SealPeriod.java
@@ -0,0 +1,33 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may
[cassandra] branch cep-21-tcm created (now 2d133ba464)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git at 2d133ba464 TMP - use bundled version of harry This branch includes the following new commits: new 7b6b04a8bf [CEP-21] Add distributed metadata log new 728a9d9a07 [CEP-21] Move schema management to TCM new 56d289b4fb [CEP-21] Include current epoch in internode header new 35910f2425 [CEP-21] Add rudimentary cluster membership to TCM new 2434132cc7 [CEP-21] Add basics of ownership and data placement new 57ad145f19 [CEP-21] Produce placements equivalent to current replication strategies new ee17642571 [CEP-21] Initial transforms for modifying ownership new 5a09d94419 [CEP-21] Start to remove and deprecate gossip functionality new d4d35a7d79 [CEP-21] Replace fake initial implementation of CMS membership new 6fcff5b04e [CEP-21] Major changes to StorageService to remove TMD and Gossip (1/7) new 8425d3177d [CEP-21] Placement and ownership related changes (2/7) new ad7edf5331 [CEP-21] Consistent read/write path (3/7) new da5b8ec9ce [CEP-21] Update ColumnFamilyStore (4/7) new 205331198f [CEP-21] Fix misc compilation errors (5/7) new f528d235c8 [CEP-21] Remove TokenMetadata (6/7) new 9a1546b089 [CEP-21] Restore operation mode reporting (7/7) new 7bda86e1fe [CEP-21] New nodetool commands new bb615495ba [CEP-21] Add missing implementations to concurrent utils new e7d451fd6b [CEP-21] Option to record thread creation stacktrace new 6438a5f17b [CEP-21] Upgrade support new 4892a4e0ea [CEP-21] Update snitches to use Directory new 85748281f4 [CEP-21] Modify CassandraDaemon new e2036cd719 [CEP-21] Test / build config changes new 11e8ded97b [CEP-21] Bulk update of unit and dtests new 2d133ba464 TMP - use bundled version of harry The 25 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 21/25: [CEP-21] Update snitches to use Directory
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 4892a4e0ea39968308771864fc08b5097501243d
Author: Sam Tunnicliffe
AuthorDate: Fri Mar 3 19:02:57 2023 +
[CEP-21] Update snitches to use Directory
Minimal changes to IEndpointSnitch implementations to have them pull
location info from Directory.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../cassandra/locator/AlibabaCloudSnitch.java | 40 ++
.../apache/cassandra/locator/CloudstackSnitch.java | 40 ++
.../org/apache/cassandra/locator/Ec2Snitch.java| 37 ++--
.../cassandra/locator/GoogleCloudSnitch.java | 39 ++---
.../locator/GossipingPropertyFileSnitch.java | 40 +-
5 files changed, 51 insertions(+), 145 deletions(-)
diff --git a/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java
b/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java
index 729e1b3763..45d7fcd857 100644
--- a/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java
+++ b/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java
@@ -25,13 +25,11 @@ import java.net.MalformedURLException;
import java.net.SocketTimeoutException;
import java.net.URL;
import java.nio.charset.StandardCharsets;
-import java.util.Map;
-import org.apache.cassandra.db.SystemKeyspace;
+
import org.apache.cassandra.exceptions.ConfigurationException;
-import org.apache.cassandra.gms.ApplicationState;
-import org.apache.cassandra.gms.EndpointState;
-import org.apache.cassandra.gms.Gossiper;
import org.apache.cassandra.io.util.FileUtils;
+import org.apache.cassandra.tcm.ClusterMetadata;
+import org.apache.cassandra.tcm.membership.NodeId;
import org.apache.cassandra.utils.FBUtilities;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -47,9 +45,6 @@ public class AlibabaCloudSnitch extends
AbstractNetworkTopologySnitch
{
protected static final Logger logger =
LoggerFactory.getLogger(AlibabaCloudSnitch.class);
protected static final String ZONE_NAME_QUERY_URL =
"http://100.100.100.200/latest/meta-data/zone-id";;
-private static final String DEFAULT_DC = "UNKNOWN-DC";
-private static final String DEFAULT_RACK = "UNKNOWN-RACK";
-private Map> savedEndpoints;
protected String ecsZone;
protected String ecsRegion;
@@ -112,17 +107,10 @@ public class AlibabaCloudSnitch extends
AbstractNetworkTopologySnitch
{
if (endpoint.equals(FBUtilities.getBroadcastAddressAndPort()))
return ecsZone;
-EndpointState state =
Gossiper.instance.getEndpointStateForEndpoint(endpoint);
-if (state == null || state.getApplicationState(ApplicationState.RACK)
== null)
-{
-if (savedEndpoints == null)
-savedEndpoints = SystemKeyspace.loadDcRackInfo();
-if (savedEndpoints.containsKey(endpoint))
-return savedEndpoints.get(endpoint).get("rack");
-return DEFAULT_RACK;
-}
-return state.getApplicationState(ApplicationState.RACK).value;
-
+
+ClusterMetadata metadata = ClusterMetadata.current();
+NodeId nodeId = metadata.directory.peerId(endpoint);
+return metadata.directory.location(nodeId).rack;
}
@Override
@@ -130,17 +118,9 @@ public class AlibabaCloudSnitch extends
AbstractNetworkTopologySnitch
{
if (endpoint.equals(FBUtilities.getBroadcastAddressAndPort()))
return ecsRegion;
-EndpointState state =
Gossiper.instance.getEndpointStateForEndpoint(endpoint);
-if (state == null || state.getApplicationState(ApplicationState.DC) ==
null)
-{
-if (savedEndpoints == null)
-savedEndpoints = SystemKeyspace.loadDcRackInfo();
-if (savedEndpoints.containsKey(endpoint))
-return savedEndpoints.get(endpoint).get("data_center");
-return DEFAULT_DC;
-}
-return state.getApplicationState(ApplicationState.DC).value;
-
+ClusterMetadata metadata = ClusterMetadata.current();
+NodeId nodeId = metadata.directory.peerId(endpoint);
+return metadata.directory.location(nodeId).datacenter;
}
}
diff --git a/src/java/org/apache/cassandra/locator/CloudstackSnitch.java
b/src/java/org/apache/cassandra/locator/CloudstackSnitch.java
index d8579534fb..a4535d6374 100644
--- a/src/java/org/apache/cassandra/locator/CloudstackSnitch.java
+++ b/src/java/org/apache/cassandra/locator/CloudstackSnitch.java
@@ -25,7 +25,6 @@ import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URI;
import java.nio.charset.StandardCharsets;
-import java.util.Map;
import java.util.regex.Pattern;
import java.util.regex.Matcher;
@@ -33,12 +
[cassandra] 09/25: [CEP-21] Replace fake initial implementation of CMS membership
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit d4d35a7d790df44fc151c52f405fc10484f84930
Author: Sam Tunnicliffe
AuthorDate: Thu Mar 2 14:42:04 2023 +
[CEP-21] Replace fake initial implementation of CMS membership
WIP commit (i.e. does not compile) replacing initial toy implementation
of CMS membership with proper implementation. Membership of the CMS is
determined by ownership of keyspaces with the META replication strategy
(more precisely, by being a member of the _read_ placements for meta
strategy keyspaces, a node is considered a member of the CMS).
Also implements more of the "real" [pre]initialization of the CMS, in
preparation for supporting upgrading a running cluster from a gossip
based system.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../org/apache/cassandra/auth/AuthKeyspace.java| 55 +++---
src/java/org/apache/cassandra/config/Config.java | 2 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 6 +-
.../org/apache/cassandra/locator/MetaStrategy.java | 4 +-
.../schema/SystemDistributedKeyspace.java | 114 +--
.../org/apache/cassandra/tcm/ClusterMetadata.java | 63 ++
src/java/org/apache/cassandra/tcm/Discovery.java | 2 +-
.../apache/cassandra/tcm/PaxosBackedProcessor.java | 7 +-
.../org/apache/cassandra/tcm/RemoteProcessor.java | 2 +-
.../org/apache/cassandra/tcm/Transformation.java | 7 +
.../cassandra/tcm/compatibility/GossipHelper.java | 3 -
.../org/apache/cassandra/tcm/log/LocalLog.java | 4 +-
.../apache/cassandra/tcm/sequences/AddToCMS.java | 217 +
.../tcm/sequences/InProgressSequences.java | 1 +
...lize.java => BaseMembershipTransformation.java} | 57 ++
.../tcm/transformations/cms/FinishAddMember.java | 80
.../tcm/transformations/cms/Initialize.java| 58 +-
.../tcm/transformations/cms/PreInitialize.java | 29 ++-
.../tcm/transformations/cms/RemoveMember.java | 59 ++
.../tcm/transformations/cms/StartAddMember.java| 84
.../apache/cassandra/tracing/TraceKeyspace.java| 51 +++--
21 files changed, 690 insertions(+), 215 deletions(-)
diff --git a/src/java/org/apache/cassandra/auth/AuthKeyspace.java
b/src/java/org/apache/cassandra/auth/AuthKeyspace.java
index 67fc9c1a06..12760706fc 100644
--- a/src/java/org/apache/cassandra/auth/AuthKeyspace.java
+++ b/src/java/org/apache/cassandra/auth/AuthKeyspace.java
@@ -37,7 +37,7 @@ public final class AuthKeyspace
{
}
-private static final int DEFAULT_RF =
CassandraRelevantProperties.SYSTEM_AUTH_DEFAULT_RF.getInt();
+public static final int DEFAULT_RF =
CassandraRelevantProperties.SYSTEM_AUTH_DEFAULT_RF.getInt();
/**
* Generation is used as a timestamp for automatic table creation on
startup.
@@ -57,49 +57,54 @@ public final class AuthKeyspace
public static final long SUPERUSER_SETUP_DELAY =
Long.getLong("cassandra.superuser_setup_delay_ms", 1);
+public static String ROLES_CQL = "CREATE TABLE IF NOT EXISTS %s ("
+ + "role text,"
+ + "is_superuser boolean,"
+ + "can_login boolean,"
+ + "salted_hash text,"
+ + "member_of set,"
+ + "PRIMARY KEY(role))";
private static final TableMetadata Roles =
parse(ROLES,
"role definitions",
- "CREATE TABLE %s ("
- + "role text,"
- + "is_superuser boolean,"
- + "can_login boolean,"
- + "salted_hash text,"
- + "member_of set,"
- + "PRIMARY KEY(role))");
+ ROLES_CQL);
+public static String ROLE_MEMBERS_CQL = "CREATE TABLE IF NOT EXISTS %s ("
++ "role text,"
++ "member text,"
++ "PRIMARY KEY(role, member))";
private static final TableMetadata RoleMembers =
parse(ROLE_MEMBERS,
"role memberships lookup table",
- "CREATE TABLE %s ("
- + "role text,"
- + "member text,"
- + "PRIMARY KEY(role, member))");
+ ROLE_MEMBERS_CQL);
+public static String ROLE_PERMISSIONS_CQL = "CREATE TABLE IF NOT EXISTS %s
("
++ "role text,"
++ "resource text,"
++ "permissions set,"
++ "PRIMARY KEY(role,
res
[cassandra] 20/25: [CEP-21] Upgrade support
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 6438a5f17b9d19aa2a0954521f3049a282fb93ff
Author: Sam Tunnicliffe
AuthorDate: Fri Mar 3 17:40:14 2023 +
[CEP-21] Upgrade support
Following an upgrade, nodes in an existing cluster will enter a minimal
modification mode. In this state, the set of allowed cluster metadata
modifications is constrained to include only the addition, removal and
replacement of nodes, to allow failed hosts to be replaced during the
upgrade.
In this mode the CMS has no members and each peer maintains its
own ClusterMetadata independently. This metadata is intitialised at
startup from system tables and gossip is used to propagate the permitted
metadata changes.
When the operator is ready, one node is chosen for promotion to the initial
CMS, which is done manually via nodetool. At this point, the candidate node
will propose itself as the initial CMS and attempt to gain consensus from
the rest of the cluster. If successful, it verifies that all peers have an
identical view of cluster metadata and initialises the distributed log with
a snapshot of that metadata.
Once this process is complete all future cluster metadata updates are
performed
via the CMS using the global log and reverting to the previous method of
metadata management is not supported. Further members can and should be
added
to the CMS via the nodetool command.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../db/commitlog/CommitLogDescriptor.java | 5 +-
.../apache/cassandra/db/virtual/PeersTable.java| 187 ++
.../cassandra/db/virtual/SystemViewsKeyspace.java | 1 +
.../apache/cassandra/hints/HintsDescriptor.java| 5 +-
.../cassandra/locator/InetAddressAndPort.java | 14 +-
src/java/org/apache/cassandra/net/InboundSink.java | 28 ++-
.../apache/cassandra/net/ResponseVerbHandler.java | 2 +
src/java/org/apache/cassandra/net/Verb.java| 11 +-
.../cassandra/tcm/ClusterMetadataService.java | 121 +++-
src/java/org/apache/cassandra/tcm/Commit.java | 14 +-
src/java/org/apache/cassandra/tcm/Discovery.java | 1 +
src/java/org/apache/cassandra/tcm/Startup.java | 116 ++-
.../tcm/listeners/LegacyStateListener.java | 91 +
.../org/apache/cassandra/tcm/log/LocalLog.java | 28 ++-
.../tcm/migration/ClusterMetadataHolder.java | 66 +++
.../apache/cassandra/tcm/migration/Election.java | 212 -
.../cassandra/tcm/migration/GossipCMSListener.java | 7 +-
.../cassandra/tcm/migration/GossipProcessor.java | 41
src/java/org/apache/cassandra/tools/NodeTool.java | 1 +
.../apache/cassandra/tools/nodetool/AddToCMS.java | 39
20 files changed, 953 insertions(+), 37 deletions(-)
diff --git
a/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java
b/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java
index ed2af1bd0d..41444fecde 100644
--- a/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java
+++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java
@@ -62,13 +62,14 @@ public class CommitLogDescriptor
// We don't support anything pre-3.0
public static final int VERSION_30 = 6;
public static final int VERSION_40 = 7;
+public static final int VERSION_50 = 8;
/**
* Increment this number if there is a changes in the commit log disc
layout or MessagingVersion changes.
* Note: make sure to handle {@link #getMessagingVersion()}
*/
@VisibleForTesting
-public static final int current_version = VERSION_40;
+public static final int current_version = VERSION_50;
final int version;
public final long id;
@@ -222,6 +223,8 @@ public class CommitLogDescriptor
return MessagingService.VERSION_30;
case VERSION_40:
return MessagingService.VERSION_40;
+case VERSION_50:
+return MessagingService.VERSION_50;
default:
throw new IllegalStateException("Unknown commitlog version " +
version);
}
diff --git a/src/java/org/apache/cassandra/db/virtual/PeersTable.java
b/src/java/org/apache/cassandra/db/virtual/PeersTable.java
new file mode 100644
index 00..157bed2f60
--- /dev/null
+++ b/src/java/org/apache/cassandra/db/virtual/PeersTable.java
@@ -0,0 +1,187 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0
[cassandra] 14/25: [CEP-21] Fix misc compilation errors (5/7)
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 205331198fd1c45e5257a93137e01b22a1af4c4c
Author: Sam Tunnicliffe
AuthorDate: Thu Mar 2 19:33:16 2023 +
[CEP-21] Fix misc compilation errors (5/7)
Part 5 of 7 only compilation errors in non-test code are directly
related to TokenMetadata
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../org/apache/cassandra/locator/PropertyFileSnitch.java | 12 +---
.../org/apache/cassandra/tcm/ClusterMetadataService.java | 14 --
2 files changed, 17 insertions(+), 9 deletions(-)
diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
index 3a9b161356..4ceef2e84e 100644
--- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
+++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java
@@ -31,6 +31,7 @@ import org.slf4j.LoggerFactory;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.service.StorageService;
+import org.apache.cassandra.tcm.ClusterMetadataService;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.ResourceWatcher;
import org.apache.cassandra.utils.WrappedRunnable;
@@ -213,14 +214,9 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
endpointMap = reloadedMap;
if (StorageService.instance != null) // null check tolerates circular
dependency; see CASSANDRA-4145
{
-if (isUpdate)
-StorageService.instance.updateTopology();
-else
-
StorageService.instance.getTokenMetadata().invalidateCachedRings();
+//if (isUpdate)
+//StorageService.instance.updateTopology();
}
-
-if (gossipStarted)
-StorageService.instance.gossipSnitchInfo();
}
/**
@@ -232,6 +228,8 @@ public class PropertyFileSnitch extends
AbstractNetworkTopologySnitch
*/
private static boolean livenessCheck(HashMap
reloadedMap, String[] reloadedDefaultDCRack)
{
+if (ClusterMetadataService.instance() == null)
+return false;
// If the default has changed we must check all live hosts but
hopefully we will find a live
// host quickly and interrupt the loop. Otherwise we only check the
live hosts that were either
// in the old set or in the new set
diff --git a/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
b/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
index 37381354aa..4cf0db438a 100644
--- a/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
+++ b/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
@@ -35,7 +35,6 @@ import org.apache.cassandra.net.IVerbHandler;
import org.apache.cassandra.net.Message;
import org.apache.cassandra.net.NoPayload;
import org.apache.cassandra.schema.DistributedSchema;
-import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.tcm.log.Entry;
import org.apache.cassandra.tcm.log.LocalLog;
import org.apache.cassandra.tcm.log.Replication;
@@ -390,6 +389,17 @@ public class ClusterMetadataService
});
}
+public void initRecentlySealedPeriodsIndex()
+{
+Sealed.initIndexFromSystemTables();
+}
+
+public boolean isMigrating()
+{
+return false;
+//return Election.instance.isMigrating();
+}
+
/**
* Switchable implementations that allow us to go between local and remote
implementation whenever we need it.
* When the node becomes a member of CMS, it switches back to being a
regular member of a cluster, and all
@@ -469,6 +479,6 @@ public class ClusterMetadataService
public enum State
{
-LOCAL, REMOTE
+LOCAL, REMOTE, GOSSIP
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 22/25: [CEP-21] Modify CassandraDaemon
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 85748281f4a7216581a94f1ca73183921534dca6
Author: Sam Tunnicliffe
AuthorDate: Fri Mar 3 19:43:03 2023 +
[CEP-21] Modify CassandraDaemon
Alter CassandraDaemon intialization to accomodate TCM and replay of the
cluster metadata log. This is something of a WIP and there is clearly
scope to further clean up this part of the code.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../apache/cassandra/service/CassandraDaemon.java | 79 ++
1 file changed, 36 insertions(+), 43 deletions(-)
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index 4aea5d47aa..38a94a7d7b 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -71,7 +71,8 @@ import org.apache.cassandra.gms.Gossiper;
import org.apache.cassandra.io.sstable.SSTableHeaderFix;
import org.apache.cassandra.io.util.File;
import org.apache.cassandra.io.util.FileUtils;
-import org.apache.cassandra.locator.InetAddressAndPort;
+import org.apache.cassandra.tcm.ClusterMetadataService;
+import org.apache.cassandra.tcm.Startup;
import org.apache.cassandra.metrics.CassandraMetricsRegistry;
import org.apache.cassandra.metrics.DefaultNameFactory;
import org.apache.cassandra.net.StartupClusterConnectivityChecker;
@@ -79,11 +80,10 @@ import org.apache.cassandra.schema.Schema;
import org.apache.cassandra.schema.SchemaConstants;
import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.security.ThreadAwareSecurityManager;
-import org.apache.cassandra.streaming.StreamManager;
-import org.apache.cassandra.service.paxos.PaxosState;
import org.apache.cassandra.tcm.ClusterMetadata;
import org.apache.cassandra.tcm.InProgressSequence;
-import org.apache.cassandra.tcm.Startup;
+import org.apache.cassandra.streaming.StreamManager;
+import org.apache.cassandra.service.paxos.PaxosState;
import org.apache.cassandra.utils.FBUtilities;
import org.apache.cassandra.utils.JMXServerUtils;
import org.apache.cassandra.utils.JVMStabilityInspector;
@@ -96,7 +96,6 @@ import
org.apache.cassandra.utils.logging.LoggingSupportFactory;
import org.apache.cassandra.utils.logging.VirtualTableAppender;
import static java.util.concurrent.TimeUnit.NANOSECONDS;
-import static
org.apache.cassandra.config.CassandraRelevantProperties.CASSANDRA_FOREGROUND;
import static
org.apache.cassandra.config.CassandraRelevantProperties.CASSANDRA_JMX_REMOTE_PORT;
import static
org.apache.cassandra.config.CassandraRelevantProperties.CASSANDRA_PID_FILE;
import static
org.apache.cassandra.config.CassandraRelevantProperties.COM_SUN_MANAGEMENT_JMXREMOTE_PORT;
@@ -113,6 +112,7 @@ import static
org.apache.cassandra.config.CassandraRelevantProperties.JAVA_VM_NA
public class CassandraDaemon
{
public static final String MBEAN_NAME =
"org.apache.cassandra.db:type=NativeAccess";
+public static boolean SKIP_GC_INSPECTOR =
Boolean.getBoolean("cassandra.startup.skip_gc_inspector");
private static final Logger logger;
@@ -259,9 +259,7 @@ public class CassandraDaemon
NativeLibrary.tryMlockall();
DatabaseDescriptor.createAllDirectories();
-
Keyspace.setInitialized();
-
CommitLog.instance.start();
try
@@ -272,7 +270,6 @@ public class CassandraDaemon
{
throw new AssertionError("Can't initialize cluster metadata
service");
}
-
QueryProcessor.registerStatementInvalidatingListener();
//TODO disabled b/c this involves checking schema but log replay
hasn't run yet so it hasn't been constructed
@@ -295,20 +292,9 @@ public class CassandraDaemon
SystemKeyspaceMigrator41.migrate();
+// TODO (TM/alexp)
// Populate token metadata before flushing, for token-aware sstable
partitioning (#6696)
-//StorageService.instance.populateTokenMetadata();
-
-//try
-//{
-//// load schema from disk
-//Schema.instance.loadFromDisk();
-//}
-//catch (Exception e)
-//{
-//logger.error("Error while loading schema: ", e);
-//throw e;
-//}
-
+// StorageService.instance.populateTokenMetadata();
setupVirtualKeyspaces();
SSTableHeaderFix.fixNonFrozenUDTIfUpgradeFrom30();
@@ -322,13 +308,12 @@ public class CassandraDaemon
exitOrFail(e.returnCode, e.getMessage(), e.getCause());
}
-Keyspace.setInitialized();
-
// initialize keyspaces
for (String keyspaceName : Schema.instance.getKeyspaces())
{
if (logger.isDebugEnabled())
[cassandra] 23/25: [CEP-21] Test / build config changes
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit e2036cd719b051c0cb9b58709f29028f0a51b0be
Author: Sam Tunnicliffe
AuthorDate: Mon Mar 6 10:09:36 2023 +
[CEP-21] Test / build config changes
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.build/cassandra-build-deps-template.xml | 4
.build/parent-pom-template.xml | 5 +
test/conf/logback-dtest.xml | 5 +++--
test/conf/logback-simulator.xml | 1 -
4 files changed, 12 insertions(+), 3 deletions(-)
diff --git a/.build/cassandra-build-deps-template.xml
b/.build/cassandra-build-deps-template.xml
index 93387996ef..a317f861a5 100644
--- a/.build/cassandra-build-deps-template.xml
+++ b/.build/cassandra-build-deps-template.xml
@@ -62,6 +62,10 @@
com.google.jimfs
jimfs
+
+ com.google.guava
+ guava-testlib
+
com.puppycrawl.tools
checkstyle
diff --git a/.build/parent-pom-template.xml b/.build/parent-pom-template.xml
index 4edd92e65c..f016c98679 100644
--- a/.build/parent-pom-template.xml
+++ b/.build/parent-pom-template.xml
@@ -297,6 +297,11 @@
+
+com.google.guava
+guava-testlib
+27.0-jre
+
com.google.jimfs
jimfs
diff --git a/test/conf/logback-dtest.xml b/test/conf/logback-dtest.xml
index 52eaf335de..48d9859b67 100644
--- a/test/conf/logback-dtest.xml
+++ b/test/conf/logback-dtest.xml
@@ -27,7 +27,7 @@
./build/test/logs/${cassandra.testtag}/${suitename}/${cluster_id}/${instance_id}/system.log
- %-5level [%thread] ${instance_id} %date{ISO8601}
%msg%n
+ %-5level [%thread] ${instance_id} %date{ISO8601} %F:%L -
%msg%n
true
@@ -50,7 +50,8 @@
-
+
+
diff --git a/test/conf/logback-simulator.xml b/test/conf/logback-simulator.xml
index 25c9de8a6a..3a9fabc380 100644
--- a/test/conf/logback-simulator.xml
+++ b/test/conf/logback-simulator.xml
@@ -40,7 +40,6 @@
-
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 25/25: TMP - use bundled version of harry
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 2d133ba46499d9e4c4a5c0d2ab85df69b5f424a1
Author: Sam Tunnicliffe
AuthorDate: Thu Mar 9 19:09:48 2023 +
TMP - use bundled version of harry
---
.build/build-resolver.xml | 3 +++
.build/parent-pom-template.xml| 5 +++--
.gitignore| 1 +
build.xml | 2 +-
lib/harry-0.0.2-internal-20221121.14211-2.jar | Bin 0 -> 435204 bytes
5 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/.build/build-resolver.xml b/.build/build-resolver.xml
index 60122b854f..25c1c86519 100644
--- a/.build/build-resolver.xml
+++ b/.build/build-resolver.xml
@@ -259,5 +259,8 @@
+
+
+
diff --git a/.build/parent-pom-template.xml b/.build/parent-pom-template.xml
index f016c98679..7e0bb51dcf 100644
--- a/.build/parent-pom-template.xml
+++ b/.build/parent-pom-template.xml
@@ -487,8 +487,9 @@
org.apache.cassandra
harry-core
-0.0.1
-test
+0.0.2-internal-20221121.14211-2
+system
+
${test.lib}/harry-0.0.2-internal-20221121.14211-2.jar
org.reflections
diff --git a/.gitignore b/.gitignore
index 916ab0de99..7e32b06b4e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -16,6 +16,7 @@ pylib/src/
**/cqlshlib.xml
!lib/cassandra-driver-internal-only-*.zip
!lib/puresasl-*.zip
+!lib/harry-0.0.2-internal-20221121.14211-2.jar
# C* debs
build-stamp
diff --git a/build.xml b/build.xml
index 197f720472..ee1f5892cd 100644
--- a/build.xml
+++ b/build.xml
@@ -389,7 +389,7 @@
-
+
diff --git a/lib/harry-0.0.2-internal-20221121.14211-2.jar
b/lib/harry-0.0.2-internal-20221121.14211-2.jar
new file mode 100644
index 00..44bf1be4a3
Binary files /dev/null and b/lib/harry-0.0.2-internal-20221121.14211-2.jar
differ
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 19/25: [CEP-21] Option to record thread creation stacktrace
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit e7d451fd6b0516a0566dcb466ceba677d08759be
Author: Sam Tunnicliffe
AuthorDate: Fri Mar 3 17:52:36 2023 +
[CEP-21] Option to record thread creation stacktrace
Adds a property for use in tests and debugging which preserves
the stacktrace of when a thread is created by NamedThreadFactory.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../cassandra/concurrent/NamedThreadFactory.java | 38 +-
1 file changed, 37 insertions(+), 1 deletion(-)
diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
index 9816649424..1df561e609 100644
--- a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
+++ b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java
@@ -17,6 +17,7 @@
*/
package org.apache.cassandra.concurrent;
+import java.util.Arrays;
import java.util.concurrent.ThreadFactory;
import java.util.concurrent.atomic.AtomicInteger;
@@ -33,6 +34,9 @@ import org.apache.cassandra.utils.JVMStabilityInspector;
public class NamedThreadFactory implements ThreadFactory
{
+public static final String
PRESERVE_THREAD_CREATION_STACKTRACE_PROPERTY_NAME =
"cassandra.test.preserve_thread_creation_stacktrace";
+public static final Boolean PRESERVE_THREAD_CREATION_STACKTRACE =
Boolean.getBoolean(PRESERVE_THREAD_CREATION_STACKTRACE_PROPERTY_NAME);
+
private static final AtomicInteger anonymousCounter = new AtomicInteger();
private static volatile String globalPrefix;
@@ -159,11 +163,43 @@ public class NamedThreadFactory implements ThreadFactory
public static Thread createThread(ThreadGroup threadGroup, Runnable
runnable, String name, boolean daemon)
{
String prefix = globalPrefix;
-Thread thread = new FastThreadLocalThread(threadGroup, runnable,
prefix != null ? prefix + name : name);
+Thread thread;
+String threadName = prefix != null ? prefix + name : name;
+if (PRESERVE_THREAD_CREATION_STACKTRACE)
+thread = new InspectableFastThreadLocalThread(threadGroup,
runnable, threadName);
+else
+thread = new FastThreadLocalThread(threadGroup, runnable,
threadName);
thread.setDaemon(daemon);
return thread;
}
+public static class InspectableFastThreadLocalThread extends
FastThreadLocalThread
+{
+public StackTraceElement[] creationTrace;
+
+private void setStack()
+{
+creationTrace = Thread.currentThread().getStackTrace();
+creationTrace = Arrays.copyOfRange(creationTrace, 2,
creationTrace.length);
+}
+
+public InspectableFastThreadLocalThread() { super(); setStack(); }
+
+public InspectableFastThreadLocalThread(Runnable target) {
super(target); setStack(); }
+
+public InspectableFastThreadLocalThread(ThreadGroup group, Runnable
target) { super(group, target); setStack(); }
+
+public InspectableFastThreadLocalThread(String name) { super(name);
setStack(); }
+
+public InspectableFastThreadLocalThread(ThreadGroup group, String
name) { super(group, name); setStack(); }
+
+public InspectableFastThreadLocalThread(Runnable target, String name)
{ super(target, name); setStack(); }
+
+public InspectableFastThreadLocalThread(ThreadGroup group, Runnable
target, String name) { super(group, target, name); setStack(); }
+
+public InspectableFastThreadLocalThread(ThreadGroup group, Runnable
target, String name, long stackSize) { super(group, target, name, stackSize);
setStack(); }
+
+}
public static T setupThread(T thread, int priority,
ClassLoader contextClassLoader, Thread.UncaughtExceptionHandler
uncaughtExceptionHandler)
{
thread.setPriority(priority);
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 04/25: [CEP-21] Add rudimentary cluster membership to TCM
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 35910f2425e61d275eed511691a96c208e686fbf
Author: Sam Tunnicliffe
AuthorDate: Wed Mar 1 10:53:51 2023 +
[CEP-21] Add rudimentary cluster membership to TCM
Adds a new Directory component to ClusterMetadata to manage member
identity, state location and addressing. This duplicates some of the
functions of TokenMetadata, Topology et al but with updates performed
consistently via the global log. Although it isn't actually used for
anything yet it is a prerequisite for managing data ownership through
TCM, which will eventually replace TokenMetadata completely.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../apache/cassandra/service/StorageService.java | 3 +
.../org/apache/cassandra/tcm/ClusterMetadata.java | 68 ++-
.../org/apache/cassandra/tcm/MetadataKeys.java | 1 +
.../org/apache/cassandra/tcm/Transformation.java | 2 +
.../cassandra/tcm/compatibility/GossipHelper.java | 2 +
.../apache/cassandra/tcm/membership/Directory.java | 578 +
.../apache/cassandra/tcm/membership/Location.java | 83 +++
.../cassandra/tcm/membership/NodeAddresses.java| 125 +
.../apache/cassandra/tcm/membership/NodeId.java| 89
.../NodeState.java}| 35 +-
.../cassandra/tcm/membership/NodeVersion.java | 118 +
.../cassandra/tcm/transformations/Register.java| 208
.../apache/cassandra/utils/CassandraVersion.java | 1 +
.../apache/cassandra/utils/btree/BTreeBiMap.java | 102
.../cassandra/utils/btree/BTreeMultimap.java | 214
.../org/apache/cassandra/utils/btree/BTreeSet.java | 45 +-
16 files changed, 1639 insertions(+), 35 deletions(-)
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index f9362a018d..48c6168742 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -105,6 +105,8 @@ import
org.apache.cassandra.service.disk.usage.DiskUsageBroadcaster;
import org.apache.cassandra.tcm.ClusterMetadata;
import org.apache.cassandra.tcm.ClusterMetadataService;
import org.apache.cassandra.tcm.Startup;
+import org.apache.cassandra.tcm.membership.NodeId;
+import org.apache.cassandra.tcm.transformations.Register;
import org.apache.cassandra.utils.concurrent.Future;
import org.apache.cassandra.schema.TableId;
import org.apache.cassandra.utils.concurrent.FutureCombiner;
@@ -1015,6 +1017,7 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
throw new RuntimeException(e);
}
+NodeId self = Register.maybeRegister();
completeInitialization();
}
diff --git a/src/java/org/apache/cassandra/tcm/ClusterMetadata.java
b/src/java/org/apache/cassandra/tcm/ClusterMetadata.java
index 4d9f5a28cf..90d5343d2c 100644
--- a/src/java/org/apache/cassandra/tcm/ClusterMetadata.java
+++ b/src/java/org/apache/cassandra/tcm/ClusterMetadata.java
@@ -38,6 +38,12 @@ import org.apache.cassandra.locator.InetAddressAndPort;
import org.apache.cassandra.schema.DistributedSchema;
import org.apache.cassandra.tcm.extensions.ExtensionKey;
import org.apache.cassandra.tcm.extensions.ExtensionValue;
+import org.apache.cassandra.tcm.membership.Directory;
+import org.apache.cassandra.tcm.membership.Location;
+import org.apache.cassandra.tcm.membership.NodeAddresses;
+import org.apache.cassandra.tcm.membership.NodeId;
+import org.apache.cassandra.tcm.membership.NodeState;
+import org.apache.cassandra.tcm.membership.NodeVersion;
import org.apache.cassandra.tcm.serialization.MetadataSerializer;
import org.apache.cassandra.tcm.serialization.Version;
import org.apache.cassandra.tcm.transformations.cms.EntireRange;
@@ -58,16 +64,28 @@ public class ClusterMetadata
public final ImmutableMap, ExtensionValue> extensions;
public final DistributedSchema schema;
+public final Directory directory;
public final EndpointsForRange cmsReplicas;
public final ImmutableSet cmsMembers;
public ClusterMetadata(IPartitioner partitioner)
+{
+this(partitioner, Directory.EMPTY);
+}
+
+private ClusterMetadata(IPartitioner partitioner, Directory directory)
+{
+this(partitioner, directory, DistributedSchema.first());
+}
+
+private ClusterMetadata(IPartitioner partitioner, Directory directory,
DistributedSchema schema)
{
this(Epoch.EMPTY,
Period.EMPTY,
true,
partitioner,
- DistributedSchema.first(),
+ schema,
+ directory,
ImmutableSet.of(),
Immutable
[cassandra] 06/25: [CEP-21] Produce placements equivalent to current replication strategies
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 57ad145f1946961a4da506a9b4c35fa95725916d
Author: Sam Tunnicliffe
AuthorDate: Wed Mar 1 12:53:19 2023 +
[CEP-21] Produce placements equivalent to current replication strategies
Minimal modifications to AbstractReplicationStrategy implementations to
support the production of DataPlacements using ClusterMetadata while
retaining calculateNaturalReplicas. Also adds tests to compare the
output of both methods and assert their equivalence. Eventually, the
original implementations based on TokenMetadata will be retired and
will be retained in the test source to guard against regressions.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../locator/AbstractReplicaCollection.java | 40 ++-
.../locator/AbstractReplicationStrategy.java | 10 +-
.../apache/cassandra/locator/LocalStrategy.java| 50 +--
.../org/apache/cassandra/locator/MetaStrategy.java | 23 +-
.../cassandra/locator/NetworkTopologyStrategy.java | 70 ++--
.../apache/cassandra/locator/SimpleStrategy.java | 41 ++-
.../apache/cassandra/locator/TokenMetadata.java| 159 +
.../apache/cassandra/service/StorageService.java | 11 +-
.../service/reads/range/ReplicaPlanIterator.java | 3 +-
.../org/apache/cassandra/tcm/ClusterMetadata.java | 54 ++-
.../org/apache/cassandra/tcm/MetadataKeys.java | 5 +-
.../AsEndpoints.java} | 21 +-
.../AsLocations.java} | 27 +-
.../AsTokenMap.java} | 24 +-
.../cassandra/tcm/compatibility/GossipHelper.java | 2 +
.../tcm/compatibility/TokenRingUtils.java | 150
.../apache/cassandra/tcm/membership/Directory.java | 4 +-
.../tcm/ownership/PrimaryRangeComparator.java | 51 +++
.../apache/cassandra/tcm/ownership/TokenMap.java | 302
.../tcm/ownership/UniformRangePlacement.java | 324 +
.../tcm/transformations/cms/EntireRange.java | 8 +-
.../org/apache/cassandra/utils/BiMultiValMap.java | 16 +
.../locator/NetworkTopologyStrategyTest.java | 3 +-
.../cassandra/locator/TokenMetadataTest.java | 3 +-
.../cassandra/service/LeaveAndBootstrapTest.java | 3 +-
.../org/apache/cassandra/service/MoveTest.java | 3 +-
.../cassandra/tcm/membership/MembershipUtils.java | 10 +
.../cassandra/tcm/ownership/OwnershipUtils.java| 139
.../tcm/ownership/PlacementDeltasTest.java | 201 +++
.../UniformRangePlacementEquivalenceTest.java | 388 +
.../tcm/ownership/UniformRangePlacementTest.java | 290 +++
31 files changed, 2233 insertions(+), 202 deletions(-)
diff --git
a/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java
b/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java
index cc210622d1..fa8d17facd 100644
--- a/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java
+++ b/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java
@@ -22,6 +22,7 @@ import com.carrotsearch.hppc.ObjectIntHashMap;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
import com.google.common.collect.Iterables;
+import com.google.common.collect.Iterators;
import java.util.AbstractList;
import java.util.AbstractMap;
@@ -100,6 +101,18 @@ public abstract class AbstractReplicaCollection map)
+{
+Replica[] contents = new Replica[size];
+for (int i = 0; i < contents.length; i++)
+{
+if (this.contents[i] != null)
+contents[i] = map.apply(this.contents[i]);
+}
+
+return new ReplicaList(contents, begin, contents.length);
+}
+
public void add(Replica replica)
{
// can only add to full array - if we have sliced it, we must be a
snapshot
@@ -591,33 +604,30 @@ public abstract class AbstractReplicaCollection
- * It's not clear whether {@link AbstractReplicaCollection} should
implement the order sensitive {@link Object#equals(Object) equals}
- * of {@link java.util.List} or the order oblivious {@link
Object#equals(Object) equals} of {@link java.util.Set}. We never rely on
equality
- * in the database so rather then leave in a potentially surprising
implementation we have it throw {@link UnsupportedOperationException}.
- *
- *
- * Don't implement this and pick one behavior over the other. If you want
equality you can static import {@link
com.google.common.collect.Iterables#elementsEqual(Iterable, Iterable)}
- * and use that to get order sensitive equals.
+ * Implements order sensitive
[cassandra] 12/25: [CEP-21] Consistent read/write path (3/7)
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit ad7edf53314a89cbec94adf13cdc679cc4006c85 Author: Sam Tunnicliffe AuthorDate: Thu Mar 2 19:07:12 2023 + [CEP-21] Consistent read/write path (3/7) Part 3 of 7 adds the ability to detect version mismatches between peers on the read/write path and to handle such divergence. Lagging peers will attempt to catch up from the CMS if the coordinator in a r/w operation has seen newer metadata. Coordinators may fail writes if the cluster metadata changes while the write is in flight, if the consistency level can no longer be satisfied by the original replica plan. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- src/java/org/apache/cassandra/db/ReadCommand.java | 9 + .../cassandra/db/ReadCommandVerbHandler.java | 80 +++-- src/java/org/apache/cassandra/db/ReadResponse.java | 7 + .../cassandra/db/partitions/PartitionUpdate.java | 18 ++ .../org/apache/cassandra/locator/ReplicaPlan.java | 271 ++-- .../org/apache/cassandra/locator/ReplicaPlans.java | 350 - .../apache/cassandra/net/ResponseVerbHandler.java | 14 + .../service/AbstractWriteResponseHandler.java | 4 + .../cassandra/service/WriteResponseHandler.java| 12 + .../org/apache/cassandra/service/paxos/Paxos.java | 34 +- .../cassandra/service/reads/ReadCallback.java | 35 +-- .../service/reads/ReplicaFilteringProtection.java | 3 +- .../cassandra/service/reads/ResponseResolver.java | 1 - .../service/reads/range/ReplicaPlanIterator.java | 7 +- .../reads/repair/BlockingPartitionRepair.java | 54 ++-- .../service/reads/repair/BlockingReadRepair.java | 9 +- .../service/reads/repair/NoopReadRepair.java | 2 +- .../service/reads/repair/ReadOnlyReadRepair.java | 2 +- .../cassandra/service/reads/repair/ReadRepair.java | 2 +- .../reads/repair/RowIteratorMergeListener.java | 20 +- 20 files changed, 744 insertions(+), 190 deletions(-) diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java b/src/java/org/apache/cassandra/db/ReadCommand.java index 0978b8ef05..de218d9b67 100644 --- a/src/java/org/apache/cassandra/db/ReadCommand.java +++ b/src/java/org/apache/cassandra/db/ReadCommand.java @@ -38,6 +38,7 @@ import org.apache.cassandra.config.*; import org.apache.cassandra.db.filter.*; import org.apache.cassandra.exceptions.QueryCancelledException; import org.apache.cassandra.net.MessageFlag; +import org.apache.cassandra.net.MessagingService; import org.apache.cassandra.net.ParamType; import org.apache.cassandra.net.Verb; import org.apache.cassandra.db.partitions.*; @@ -66,6 +67,7 @@ import org.apache.cassandra.schema.TableMetadata; import org.apache.cassandra.schema.SchemaProvider; import org.apache.cassandra.service.ActiveRepairService; import org.apache.cassandra.service.ClientWarn; +import org.apache.cassandra.tcm.Epoch; import org.apache.cassandra.tracing.Tracing; import org.apache.cassandra.utils.FBUtilities; import org.apache.cassandra.utils.ObjectSizes; @@ -1070,6 +1072,8 @@ public abstract class ReadCommand extends AbstractReadQuery if (command.isDigestQuery()) out.writeUnsignedVInt32(command.digestVersion()); command.metadata().id.serialize(out); +if (version >= MessagingService.VERSION_50) +Epoch.serializer.serialize(command.metadata().epoch, out); out.writeInt(command.nowInSec()); ColumnFilter.serializer.serialize(command.columnFilter(), out, version); RowFilter.serializer.serialize(command.rowFilter(), out, version); @@ -1097,6 +1101,10 @@ public abstract class ReadCommand extends AbstractReadQuery boolean hasIndex = hasIndex(flags); int digestVersion = isDigest ? in.readUnsignedVInt32() : 0; TableMetadata metadata = schema.getExistingTableMetadata(TableId.deserialize(in)); +Epoch schemaVersion = null; +if (version >= MessagingService.VERSION_50) +schemaVersion = Epoch.serializer.deserialize(in); +assert schemaVersion == null || metadata.epoch.equals(schemaVersion) : metadata.epoch + " " + schemaVersion; // TODO: handle etc int nowInSec = in.readInt(); ColumnFilter columnFilter = ColumnFilter.serializer.deserialize(in, version, metadata); RowFilter rowFilter = RowFilter.serializer.deserialize(in, version, metadata); @@ -1128,6 +1136,7 @@ public abstract class ReadCommand extends AbstractReadQuery return 2 // kind + flags + (command.isDigestQuery() ? TypeSizes.sizeofUnsignedVInt(command.digestVersion()) : 0) + command.metadata().id.serializedSize() +
[cassandra] 16/25: [CEP-21] Restore operation mode reporting (7/7)
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 9a1546b089b21d724242ec88b6362607c57338db
Author: Sam Tunnicliffe
AuthorDate: Fri Mar 3 10:03:41 2023 +
[CEP-21] Restore operation mode reporting (7/7)
Part 7 of 7 brings StorageService.operationMode back into sync with
previous behaviour. Many external coordination tools depend on accessing
this state via JMX, so this is an important external interface.
This commit also adds a virtual version of the system.local table, as we
can fully construct the data for this from ClusterMetadata, meaning we no
longer the on-disk system table, though this is retained for now. In
future, more system tables can be virtualised (system.peers,
system_schema, etc).
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../org/apache/cassandra/db/SystemKeyspace.java| 32 -
.../apache/cassandra/db/virtual/LocalTable.java| 145 +
.../cassandra/db/virtual/SystemViewsKeyspace.java | 1 +
3 files changed, 173 insertions(+), 5 deletions(-)
diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java
b/src/java/org/apache/cassandra/db/SystemKeyspace.java
index 4879260406..947aafbd07 100644
--- a/src/java/org/apache/cassandra/db/SystemKeyspace.java
+++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java
@@ -97,7 +97,6 @@ import org.apache.cassandra.schema.TableMetadata;
import org.apache.cassandra.schema.Tables;
import org.apache.cassandra.schema.Types;
import org.apache.cassandra.schema.Views;
-import org.apache.cassandra.service.StorageService;
import org.apache.cassandra.service.paxos.Ballot;
import org.apache.cassandra.service.paxos.Commit;
import org.apache.cassandra.service.paxos.Commit.Accepted;
@@ -108,8 +107,10 @@ import org.apache.cassandra.service.paxos.PaxosState;
import org.apache.cassandra.service.paxos.uncommitted.PaxosRows;
import org.apache.cassandra.service.paxos.uncommitted.PaxosUncommittedIndex;
import org.apache.cassandra.streaming.StreamOperation;
+import org.apache.cassandra.tcm.ClusterMetadata;
import org.apache.cassandra.tcm.Epoch;
import org.apache.cassandra.tcm.Sealed;
+import org.apache.cassandra.tcm.membership.NodeState;
import org.apache.cassandra.transport.ProtocolVersion;
import org.apache.cassandra.utils.ByteBufferUtil;
import org.apache.cassandra.utils.CassandraVersion;
@@ -598,7 +599,27 @@ public final class SystemKeyspace
NEEDS_BOOTSTRAP,
COMPLETED,
IN_PROGRESS,
-DECOMMISSIONED
+DECOMMISSIONED;
+
+public static BootstrapState fromNodeState(NodeState nodeState)
+{
+if (nodeState == null) // todo, handle this properly
+return DECOMMISSIONED;
+switch (nodeState)
+{
+case REGISTERED:
+return NEEDS_BOOTSTRAP;
+case BOOTSTRAPPING:
+return IN_PROGRESS;
+case JOINED:
+case LEAVING:
+case MOVING:
+return COMPLETED;
+case LEFT:
+default:
+return DECOMMISSIONED;
+}
+}
}
public static void persistLocalMetadata()
@@ -912,11 +933,11 @@ public final class SystemKeyspace
executeInternal(format(req, LOCAL, LOCAL), version);
}
-private static Set tokensAsSet(Collection tokens)
+public static Set tokensAsSet(Collection tokens)
{
if (tokens.isEmpty())
return Collections.emptySet();
-Token.TokenFactory factory = StorageService.instance.getTokenFactory();
+Token.TokenFactory factory =
ClusterMetadata.current().partitioner.getTokenFactory();
Set s = new HashSet<>(tokens.size());
for (Token tk : tokens)
s.add(factory.toString(tk));
@@ -925,7 +946,7 @@ public final class SystemKeyspace
private static Collection deserializeTokens(Collection
tokensStrings)
{
-Token.TokenFactory factory = StorageService.instance.getTokenFactory();
+Token.TokenFactory factory =
ClusterMetadata.current().partitioner.getTokenFactory();
List tokens = new ArrayList<>(tokensStrings.size());
for (String tk : tokensStrings)
tokens.add(factory.fromString(tk));
@@ -1261,6 +1282,7 @@ public final class SystemKeyspace
* Read the host ID from the system keyspace, creating (and storing) one if
* none exists.
*/
+// TODO: this method should not exist. Only CMS can give out host ids.
public static synchronized UUID getOrInitializeLocalHostId()
{
UUID hostId = getLocalHostId();
diff --git a/src/java/org/apache/cassandra/db/virtual/LocalTable.java
b/src/java/org/apache/cassandra/db/v
[cassandra] 15/25: [CEP-21] Remove TokenMetadata (6/7)
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cep-21-tcm
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit f528d235c89ab2a5e19b227b01d70fe11b2fbfcc
Author: Sam Tunnicliffe
AuthorDate: Thu Mar 2 19:47:53 2023 +
[CEP-21] Remove TokenMetadata (6/7)
Part 6 of 7 Completely remove TokenMetadata, the intention is to bring
it back in a stripped down form, available to tests only, so we can
continue to verify equivalence between old and new code.
Test code is still extremely broken at this point, but non-test code is
buildable again, though almost certainly not actually runnable.
Co-authored-by: Marcus Eriksson
Co-authored-by: Alex Petrov
Co-authored-by: Sam Tunnicliffe
---
.../apache/cassandra/locator/TokenMetadata.java| 1626
.../locator/TokenMetadataDiagnostics.java | 46 -
.../cassandra/locator/TokenMetadataEvent.java | 62 -
.../apache/cassandra/service/StorageService.java |1 -
.../apache/cassandra/stress/CompactionStress.java | 18 +-
5 files changed, 11 insertions(+), 1742 deletions(-)
diff --git a/src/java/org/apache/cassandra/locator/TokenMetadata.java
b/src/java/org/apache/cassandra/locator/TokenMetadata.java
deleted file mode 100644
index abbc76d482..00
--- a/src/java/org/apache/cassandra/locator/TokenMetadata.java
+++ /dev/null
@@ -1,1626 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.locator;
-
-import java.nio.ByteBuffer;
-import java.util.*;
-import java.util.concurrent.ConcurrentHashMap;
-import java.util.concurrent.ConcurrentMap;
-import java.util.concurrent.atomic.AtomicReference;
-import java.util.concurrent.locks.ReadWriteLock;
-import java.util.concurrent.locks.ReentrantReadWriteLock;
-import java.util.function.Supplier;
-import java.util.stream.Collectors;
-
-import javax.annotation.concurrent.GuardedBy;
-
-import com.google.common.annotations.VisibleForTesting;
-import com.google.common.collect.*;
-
-import org.apache.commons.lang3.StringUtils;
-import org.slf4j.Logger;
-import org.slf4j.LoggerFactory;
-
-import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.db.DecoratedKey;
-import org.apache.cassandra.db.Keyspace;
-import org.apache.cassandra.dht.IPartitioner;
-import org.apache.cassandra.dht.Range;
-import org.apache.cassandra.dht.Token;
-import org.apache.cassandra.gms.FailureDetector;
-import org.apache.cassandra.locator.ReplicaCollection.Builder.Conflict;
-import org.apache.cassandra.service.StorageService;
-import org.apache.cassandra.tcm.compatibility.AsEndpoints;
-import org.apache.cassandra.tcm.compatibility.AsLocations;
-import org.apache.cassandra.tcm.compatibility.AsTokenMap;
-import org.apache.cassandra.tcm.compatibility.TokenRingUtils;
-import org.apache.cassandra.tcm.membership.Location;
-import org.apache.cassandra.tcm.membership.NodeId;
-import org.apache.cassandra.utils.BiMultiValMap;
-import org.apache.cassandra.utils.Pair;
-import org.apache.cassandra.utils.SortedBiMultiValMap;
-
-import static
org.apache.cassandra.config.CassandraRelevantProperties.LINE_SEPARATOR;
-import static org.apache.cassandra.utils.Clock.Global.currentTimeMillis;
-
-public class TokenMetadata implements AsEndpoints, AsLocations, AsTokenMap
-{
-private static final Logger logger =
LoggerFactory.getLogger(TokenMetadata.class);
-
-/**
- * Maintains token to endpoint map of every node in the cluster.
- * Each Token is associated with exactly one Address, but each Address may
have
- * multiple tokens. Hence, the BiMultiValMap collection.
- */
-private final BiMultiValMap tokenToEndpointMap;
-
-/** Maintains endpoint to host ID map of every node in the cluster */
-private final BiMap endpointToHostIdMap;
-
-// Prior to CASSANDRA-603, we just had Map
pendingRanges,
-// which was added to when a node began bootstrap and removed from when it
finished.
-//
-// This is inadequate when multiple changes are allowed simultaneously.
For example,
-// suppose that there is a ring of nodes A, C and E, with replication
factor 3.
[jira] [Updated] (CASSANDRA-18330) Delivery of CEP-21: Transactional Cluster Metadata
[ https://issues.apache.org/jira/browse/CASSANDRA-18330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-18330: Complexity: Challenging (was: Byzantine) > Delivery of CEP-21: Transactional Cluster Metadata > -- > > Key: CASSANDRA-18330 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18330 > Project: Cassandra > Issue Type: Epic > Components: Cluster/Membership, Cluster/Schema >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[
https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700311#comment-17700311
]
David Capwell commented on CASSANDRA-18302:
---
bq. Therefore, using asCQL may confuse the users
I guess my main question is how much of a concern is this? Right now when
users violate a data pattern (too large read, tombstone, etc.) we call
org.apache.cassandra.db.AbstractReadQuery#toCQLString which is yet another way
to go from internal type to CQL,
org.apache.cassandra.cql3.statements.SelectStatement#asCQL was only used before
for CoordinatorReadSizeWarnThresholdBytes and
CoordinatorReadSizeAbortThresholdBytes, and in those cases we log the CQL only
and don't send back to the user (by design, we only show the partition token);
when I did this it was more to inform the operators of what is going on so they
are more informed when talking with users, so the "exact" string was never
important.
So, that then gets to the current error messages for txn only, so if we look at
a few examples
{code}
cqlsh> BEGIN TRANSACTION SELECT * FROM system.peers; COMMIT TRANSACTION;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Normal
SELECT must specify either all primary key elements or all partition key
elements and LIMIT 1. In both cases partition key elements must be always
specified with equality operators; CQL SELECT * FROM system.peers"
cqlsh> BEGIN TRANSACTION SELECT tokens FROM system.peers; COMMIT TRANSACTION;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Normal
SELECT must specify either all primary key elements or all partition key
elements and LIMIT 1. In both cases partition key elements must be always
specified with equality operators; CQL SELECT tokens FROM system.peers"
cqlsh> BEGIN TRANSACTION SELECT * FROM system.peers WHERE tokens contains ('not
exists'); COMMIT TRANSACTION;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
execute this query as it might involve data filtering and thus may have
unpredictable performance. If you want to execute this query despite the
performance unpredictability, use ALLOW FILTERING"
-- this shows a bug with quoting
cqlsh> BEGIN TRANSACTION SELECT * FROM system.peers WHERE tokens contains ('not
exists') ALLOW FILTERING; COMMIT TRANSACTION;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Normal
SELECT must specify either all primary key elements or all partition key
elements and LIMIT 1. In both cases partition key elements must be always
specified with equality operators; CQL SELECT * FROM system.peers WHERE tokens
CONTAINS not exists"
{code}
in these examples there are some that are not 100% correct but give the main
thing users need. I do feel adding line/column to the error would help with
transactions, but the "correct" string is less of a concern I have.
This gets us to trade offs. We limit the txn string to 4k and in this case we
fail to log the CQL to users for large enough transactions, so for larger
transactions we are failing to show the CQL.We are also forced to hold this
string in the case it is prepared, but we already have the non-truncated string
(as you point out), so this causes us to double this memory which is also
defined by users, so not something we really can control and limit well (if you
have 1k prepared statements that are all truncated you have ~200mb (1k txn *
4096 bytes * 48 bytes for string object) of memory just for this case which we
hope to be rare.
> Fix AIOOBE and improve validation messages for transaction statements
> -
>
> Key: CASSANDRA-18302
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18302
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> Currently it happens sometimes that ArrayIndexOutOfBoundsException is thrown
> from asCql method when validation transaction statement. In addition, asCql
> does not return precisely the query user entered so the whole error message
> can be misleading.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[
https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700318#comment-17700318
]
David Capwell commented on CASSANDRA-18320:
---
bq. Actually, we do not need to wait for this to appear in 4.1.1 because this
is just test-related.
[~smiklosovic] this is only an issue in Simulator as it overrides the default
FileSystem. If you run embedded Cassandra and leverage this logic, then there
are edge cases for you, else you would never see this.
bq. I can unsurprisingly no longer reproduce
Yay! I ran _repeat jobs around 5 times and only ever saw OOM (not the "I can
not make progress" OOM, but a real one (likely direct memory given the stack
trace)). As far as I can tell, the loading logic fails due to this bug which
causes weird cascading issues; those issues are what was reported.
> Incompatible file system thrown while running Simulator
> ---
>
> Key: CASSANDRA-18320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18320
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> {code}
> java.io.UncheckedIOException
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831)
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047)
> at
> org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816)
> at
> org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370)
> at
> org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345)
> at
> org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
> at
> org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33)
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /cassandra/node1/commitlog
> at
> com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535)
> at
> com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517)
> at
> com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479)
> at
> com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465)
> at
> com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261)
> at java.nio.file.Files.delete(Files.java:1126)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252)
> {code}
--
This message was sent by Atlassian Jira
(
[jira] [Updated] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[
https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-18320:
--
Reviewers: Brandon Williams, Caleb Rackliffe (was: Caleb Rackliffe)
> Incompatible file system thrown while running Simulator
> ---
>
> Key: CASSANDRA-18320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18320
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> {code}
> java.io.UncheckedIOException
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831)
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047)
> at
> org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816)
> at
> org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370)
> at
> org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345)
> at
> org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
> at
> org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33)
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /cassandra/node1/commitlog
> at
> com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535)
> at
> com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517)
> at
> com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479)
> at
> com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465)
> at
> com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261)
> at java.nio.file.Files.delete(Files.java:1126)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[
https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700324#comment-17700324
]
David Capwell commented on CASSANDRA-18320:
---
Starting commit
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-4.1|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18320-cassandra-4.1-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18320-cassandra-4.1-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2350/]|
|trunk|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18320-trunk-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18320-trunk-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2351/]|
> Incompatible file system thrown while running Simulator
> ---
>
> Key: CASSANDRA-18320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18320
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> {code}
> java.io.UncheckedIOException
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831)
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047)
> at
> org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816)
> at
> org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370)
> at
> org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345)
> at
> org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
> at
> org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33)
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /cassandra/node1/commitlog
> at
> com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535)
> at
> com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517)
> at
> com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479)
> at
> com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465)
> at
> com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261)
> at java.
[jira] [Commented] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[
https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700330#comment-17700330
]
David Capwell commented on CASSANDRA-18302:
---
With this patch the following query gets a non-usable error
{code}
-- This Transaction is over peers table, below is the schema to help understand
the logic
-- CREATE TABLE system.peers_v2 (
-- peer inet,
-- peer_port int,
-- data_center text,
-- host_id uuid,
-- native_address inet,
-- native_port int,
-- preferred_ip inet,
-- preferred_port int,
-- rack text,
-- release_version text,
-- schema_version uuid,
-- tokens set,
-- PRIMARY KEY (peer, peer_port)
-- ) WITH CLUSTERING ORDER BY (peer_port ASC)
BEGIN TRANSACTION
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET a = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET b = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET c = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET d = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET e = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET f = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET g = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET h = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET i = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
-- load the local host peers
-- there is an assumption that a unique inet does not have multiple
ports, so LIMIT 1 is safe
-- this assumption may be a bad assumption... but you know what they say
right?
-- when you assume you...
-- Makes a ass...
-- Out of ...u...
-- And ..me...
LET j = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 );
SELECT * FROM system.peers WHERE tokens contains ('not exists') ALLOW
FILTERING;
COMMIT TRANSACTION;
{code}
and the error
{code}
com.datastax.driver.core.exceptions.InvalidQueryException: Range queries are
not allowed for reads within a transaction; returning select at 98:3 -
{code}
[jira] [Commented] (CASSANDRA-18319) Cassandra in Kubernetes: IP switch decommission issue
[
https://issues.apache.org/jira/browse/CASSANDRA-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700334#comment-17700334
]
Ines Potier commented on CASSANDRA-18319:
-
We checked and we typically sync with AWS-provided local atomic clocks, so
there should not be any skew
> Cassandra in Kubernetes: IP switch decommission issue
> -
>
> Key: CASSANDRA-18319
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18319
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ines Potier
>Priority: Normal
>
> We have recently encountered a recurring old IP reappearance issue while
> testing decommissions on some of our Kubernetes Cassandra staging clusters.
> *Issue Description*
> In Kubernetes, a Cassandra node can change IP at each pod bounce. We have
> noticed that this behavior, associated with a decommission operation, can get
> the cluster into an erroneous state.
> Consider the following situation: a Cassandra node {{node1}} , with
> {{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP
> ({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all
> other nodes’ nodetool status output includes a {{new_IP}} UN entry owning
> 20.5% of the token ring and no {{old_IP}} entry.
> Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not
> have a lot of data, and the decommission operation completes pretty quickly.
> Logs on other nodes start showing acknowledgment that {{node1}} has left and
> soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is
> deleted.
> After a minute delay, the cluster enters the erroneous state. An {{old_IP}}
> DN entry reappears in nodetool status, owning 20.5% of the token ring. No
> node owns this IP anymore and according to logs, {{old_IP}} is still
> associated with {{{}hostId1{}}}.
> *Issue Root Cause*
> By digging through Cassandra logs, and re-testing this scenario over and over
> again, we have reached the following conclusion:
> * Other nodes will continue exchanging gossip about {{old_IP}} , even after
> it becomes a fatClient.
> * The fatClient timeout and subsequent quarantine does not stop {{old_IP}}
> from reappearing in a node’s Gossip state, once its quarantine is over. We
> believe that this is due to a misalignment on all nodes’ {{old_IP}}
> expiration time.
> * Once {{new_IP}} has left the cluster, and {{old_IP}} next gossip state
> message is received by a node, StorageService will no longer face collisions
> (or will, but with an even older IP) for {{hostId1}} and its corresponding
> tokens. As a result, {{old_IP}} will regain ownership of 20.5% of the token
> ring.
> *Proposed fix*
> Following the above investigation, we were thinking about implementing the
> following fix:
> When a node receives a gossip status change with {{STATE_LEFT}} for a leaving
> endpoint {{{}new_IP{}}}, before evicting {{new_IP }}from the token ring,
> purge from Gossip (ie {{{}evictFromMembership{}}}) all endpoints that meet
> the following criteria:
> * {{endpointStateMap}} contains this endpoint
> * The endpoint is not currently a token owner
> ({{{}!tokenMetadata.isMember(endpoint){}}})
> * The endpoint’s {{hostId}} matches the {{hostId}} of {{new_IP}}
> * The endpoint is older than {{leaving_IP}}
> ({{{}Gossiper.instance.compareEndpointStartup{}}})
> * The endpoint’s token range (from {{{}endpointStateMap{}}}) intersects with
> {{{}new_IP{}}}’s
> This modification’s intention is to force nodes to realign on {{old_IP}}
> expiration, and expunge it from Gossip so it does not reappear after
> {{new_IP}} leaves the ring.
> Another approach we have also been considering is expunging {{old_IP}} at the
> moment of the StorageService collision resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18319) Cassandra in Kubernetes: IP switch decommission issue
[
https://issues.apache.org/jira/browse/CASSANDRA-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ines Potier updated CASSANDRA-18319:
Description:
We have recently encountered a recurring old IP reappearance issue while
testing decommissions on some of our Kubernetes Cassandra staging clusters.
*Issue Description*
In Kubernetes, a Cassandra node can change IP at each pod bounce. We have
noticed that this behavior, associated with a decommission operation, can get
the cluster into an erroneous state.
Consider the following situation: a Cassandra node {{node1}} , with
{{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP
({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all other
nodes’ nodetool status output includes a {{new_IP}} UN entry owning 20.5% of
the token ring and no {{old_IP}} entry.
Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not
have a lot of data, and the decommission operation completes pretty quickly.
Logs on other nodes start showing acknowledgment that {{node1}} has left and
soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is
deleted.
After a minute delay, the cluster enters the erroneous state. An {{old_IP}} DN
entry reappears in nodetool status, owning 20.5% of the token ring. No node
owns this IP anymore and according to logs, {{old_IP}} is still associated with
{{{}hostId1{}}}.
*Issue Root Cause*
By digging through Cassandra logs, and re-testing this scenario over and over
again, we have reached the following conclusion:
* Other nodes will continue exchanging gossip about {{old_IP}} , even after it
becomes a fatClient.
* The fatClient timeout and subsequent quarantine does not stop {{old_IP}}
from reappearing in a node’s Gossip state, once its quarantine is over. We
believe that this is due to a misalignment on all nodes’ {{old_IP}} expiration
time.
* Once {{new_IP}} has left the cluster, and {{old_IP}} next gossip state
message is received by a node, StorageService will no longer face collisions
(or will, but with an even older IP) for {{hostId1}} and its corresponding
tokens. As a result, {{old_IP}} will regain ownership of 20.5% of the token
ring.
*Proposed fix*
Following the above investigation, we were thinking about implementing the
following fix:
When a node receives a gossip status change with {{STATE_LEFT}} for a leaving
endpoint {{{}new_IP{}}}, before evicting {{{}new_IP from the token ring, purge
from Gossip (ie evictFromMembership{}}}) all endpoints that meet the following
criteria:
* {{endpointStateMap}} contains this endpoint
* The endpoint is not currently a token owner
({{{}!tokenMetadata.isMember(endpoint){}}})
* The endpoint’s {{hostId}} matches the {{hostId}} of {{new_IP}}
* The endpoint is older than {{leaving_IP}}
({{{}Gossiper.instance.compareEndpointStartup{}}})
* The endpoint’s token range (from {{{}endpointStateMap{}}}) intersects with
{{{}new_IP{}}}’s
This modification’s intention is to force nodes to realign on {{old_IP}}
expiration, and expunge it from Gossip so it does not reappear after {{new_IP}}
leaves the ring.
Another approach we have also been considering is expunging {{old_IP}} at the
moment of the StorageService collision resolution.
was:
We have recently encountered a recurring old IP reappearance issue while
testing decommissions on some of our Kubernetes Cassandra staging clusters.
*Issue Description*
In Kubernetes, a Cassandra node can change IP at each pod bounce. We have
noticed that this behavior, associated with a decommission operation, can get
the cluster into an erroneous state.
Consider the following situation: a Cassandra node {{node1}} , with
{{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP
({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all other
nodes’ nodetool status output includes a {{new_IP}} UN entry owning 20.5% of
the token ring and no {{old_IP}} entry.
Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not
have a lot of data, and the decommission operation completes pretty quickly.
Logs on other nodes start showing acknowledgment that {{node1}} has left and
soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is
deleted.
After a minute delay, the cluster enters the erroneous state. An {{old_IP}} DN
entry reappears in nodetool status, owning 20.5% of the token ring. No node
owns this IP anymore and according to logs, {{old_IP}} is still associated with
{{{}hostId1{}}}.
*Issue Root Cause*
By digging through Cassandra logs, and re-testing this scenario over and over
again, we have reached the following conclusion:
* Other nodes will continue exchanging gossip about {{old_IP}} , even after it
becomes a fatClient.
* The fatClient timeout and subsequent quarantine does not stop {{old_IP}}
from
[cassandra-website] branch asf-staging updated (7b6fcdaa -> 0116387e)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 7b6fcdaa generate docs for c4206294 new 0116387e generate docs for c4206294 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (7b6fcdaa) \ N -- N -- N refs/heads/asf-staging (0116387e) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4796442 -> 4796442 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
Stefan Miklosovic created CASSANDRA-18331: - Summary: Extend implicit allow-filtering to clustering keys as well Key: CASSANDRA-18331 URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 Project: Cassandra Issue Type: Improvement Reporter: Stefan Miklosovic Assignee: Stefan Miklosovic This was overlooked in CASSANDRA-18238. We should be able to do selects on vtables not only when selecting on regular columns but also on clustering ones. Currently we can do that on regulars only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18331: -- Change Category: Operability Complexity: Low Hanging Fruit Component/s: CQL/Semantics Fix Version/s: 5.x Priority: Low (was: Normal) Status: Open (was: Triage Needed) > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18331: -- Test and Documentation Plan: CI Status: Patch Available (was: In Progress) PR: [https://github.com/apache/cassandra/pull/2217/files] j11 [https://app.circleci.com/pipelines/github/instaclustr/cassandra/1990/workflows/140131d1-9f0a-45b5-b8d9-65c2b91c9be4] j8 [https://app.circleci.com/pipelines/github/instaclustr/cassandra/1990/workflows/6742ae5f-adbf-4159-abbc-0e894b015534] > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700342#comment-17700342 ] Stefan Miklosovic commented on CASSANDRA-18331: --- [~aleksey] would you mind to take a look at this, please? It was overlook in CASSANDRA-18238. At least it was not released anywhere yet. > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700342#comment-17700342 ] Stefan Miklosovic edited comment on CASSANDRA-18331 at 3/14/23 7:14 PM: [~aleksey] would you mind to take a look at this, please? It was overlooked in CASSANDRA-18238. At least it was not released anywhere yet. was (Author: smiklosovic): [~aleksey] would you mind to take a look at this, please? It was overlook in CASSANDRA-18238. At least it was not released anywhere yet. > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[
https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700354#comment-17700354
]
David Capwell commented on CASSANDRA-18320:
---
Circle CI is failing, so holding off until I can get a build working
> Incompatible file system thrown while running Simulator
> ---
>
> Key: CASSANDRA-18320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18320
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> {code}
> java.io.UncheckedIOException
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831)
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047)
> at
> org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816)
> at
> org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370)
> at
> org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345)
> at
> org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
> at
> org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33)
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /cassandra/node1/commitlog
> at
> com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535)
> at
> com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517)
> at
> com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479)
> at
> com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465)
> at
> com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261)
> at java.nio.file.Files.delete(Files.java:1126)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[
https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700354#comment-17700354
]
David Capwell edited comment on CASSANDRA-18320 at 3/14/23 7:42 PM:
Circle CI is failing, so holding off until I can get a build working... Jenkins
is chugging alone, so will see what happens
was (Author: dcapwell):
Circle CI is failing, so holding off until I can get a build working
> Incompatible file system thrown while running Simulator
> ---
>
> Key: CASSANDRA-18320
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18320
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.1.x, 5.x
>
>
> {code}
> java.io.UncheckedIOException
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831)
> at
> org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
> at java.util.ArrayList.forEach(ArrayList.java:1259)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
> at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155)
> at
> org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047)
> at
> org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816)
> at
> org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370)
> at
> org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345)
> at
> org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148)
> at
> org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33)
> Caused by: java.nio.file.DirectoryNotEmptyException:
> /cassandra/node1/commitlog
> at
> com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535)
> at
> com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517)
> at
> com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479)
> at
> com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465)
> at
> com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261)
> at java.nio.file.Files.delete(Files.java:1126)
> at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18319) Cassandra in Kubernetes: IP switch decommission issue
[
https://issues.apache.org/jira/browse/CASSANDRA-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700355#comment-17700355
]
Brandon Williams commented on CASSANDRA-18319:
--
bq. We believe that this is due to a misalignment on all nodes’ old_IP
expiration time.
Do you have any logs to support this theory?
> Cassandra in Kubernetes: IP switch decommission issue
> -
>
> Key: CASSANDRA-18319
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18319
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ines Potier
>Priority: Normal
>
> We have recently encountered a recurring old IP reappearance issue while
> testing decommissions on some of our Kubernetes Cassandra staging clusters.
> *Issue Description*
> In Kubernetes, a Cassandra node can change IP at each pod bounce. We have
> noticed that this behavior, associated with a decommission operation, can get
> the cluster into an erroneous state.
> Consider the following situation: a Cassandra node {{node1}} , with
> {{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP
> ({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all
> other nodes’ nodetool status output includes a {{new_IP}} UN entry owning
> 20.5% of the token ring and no {{old_IP}} entry.
> Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not
> have a lot of data, and the decommission operation completes pretty quickly.
> Logs on other nodes start showing acknowledgment that {{node1}} has left and
> soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is
> deleted.
> After a minute delay, the cluster enters the erroneous state. An {{old_IP}}
> DN entry reappears in nodetool status, owning 20.5% of the token ring. No
> node owns this IP anymore and according to logs, {{old_IP}} is still
> associated with {{{}hostId1{}}}.
> *Issue Root Cause*
> By digging through Cassandra logs, and re-testing this scenario over and over
> again, we have reached the following conclusion:
> * Other nodes will continue exchanging gossip about {{old_IP}} , even after
> it becomes a fatClient.
> * The fatClient timeout and subsequent quarantine does not stop {{old_IP}}
> from reappearing in a node’s Gossip state, once its quarantine is over. We
> believe that this is due to a misalignment on all nodes’ {{old_IP}}
> expiration time.
> * Once {{new_IP}} has left the cluster, and {{old_IP}} next gossip state
> message is received by a node, StorageService will no longer face collisions
> (or will, but with an even older IP) for {{hostId1}} and its corresponding
> tokens. As a result, {{old_IP}} will regain ownership of 20.5% of the token
> ring.
> *Proposed fix*
> Following the above investigation, we were thinking about implementing the
> following fix:
> When a node receives a gossip status change with {{STATE_LEFT}} for a leaving
> endpoint {{{}new_IP{}}}, before evicting {{{}new_IP from the token ring,
> purge from Gossip (ie evictFromMembership{}}}) all endpoints that meet the
> following criteria:
> * {{endpointStateMap}} contains this endpoint
> * The endpoint is not currently a token owner
> ({{{}!tokenMetadata.isMember(endpoint){}}})
> * The endpoint’s {{hostId}} matches the {{hostId}} of {{new_IP}}
> * The endpoint is older than {{leaving_IP}}
> ({{{}Gossiper.instance.compareEndpointStartup{}}})
> * The endpoint’s token range (from {{{}endpointStateMap{}}}) intersects with
> {{{}new_IP{}}}’s
> This modification’s intention is to force nodes to realign on {{old_IP}}
> expiration, and expunge it from Gossip so it does not reappear after
> {{new_IP}} leaves the ring.
> Another approach we have also been considering is expunging {{old_IP}} at the
> moment of the StorageService collision resolution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17611) Fix replace_address_test.TestReplaceAddress.test_fail_when_seed
[
https://issues.apache.org/jira/browse/CASSANDRA-17611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17611:
-
Resolution: Fixed
Status: Resolved (was: Open)
Unrelated environmental timeout.
> Fix replace_address_test.TestReplaceAddress.test_fail_when_seed
> ---
>
> Key: CASSANDRA-17611
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17611
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.27, 3.11.13
>
>
> Seen
> [with|https://ci-cassandra.apache.org/job/Cassandra-3.11/349/testReport/dtest.replace_address_test/TestReplaceAddress/test_fail_when_seed/]
> and [without
> vnode|https://ci-cassandra.apache.org/job/Cassandra-3.11/348/testReport/dtest-novnode.replace_address_test/TestReplaceAddress/test_fail_when_seed/]:
> {code:java}
> Error Message
> test teardown failure
> Stacktrace
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [MessagingService-Incoming-/127.0.0.1] 2022-04-28 22:04:36,765
> CassandraDaemon.java:244 - Exception in thread
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
> down at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:58)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:148)
> at
> org.apache.cassandra.net.MessagingService.receive(MessagingService.java:1035)
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:215)
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:195)
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:98),
> ERROR [MessagingService-Incoming-/127.0.0.1] 2022-04-28 22:04:36,765
> CassandraDaemon.java:244 - Exception in thread
> Thread[MessagingService-Incoming-/127.0.0.1,5,main]
> java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut
> down at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:58)
> at
> java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
> at
> org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:148)
> at
> org.apache.cassandra.net.MessagingService.receive(MessagingService.java:1035)
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:215)
> at
> org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:195)
> at
> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:98)]
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 05/09: Make it possible to run multiple Harry runners concurrently against the same keyspace
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit c0e749ee4f00d20df44cc9a243dac3e747bdd601
Author: Alex Petrov
AuthorDate: Thu Mar 9 17:28:44 2023 +0100
Make it possible to run multiple Harry runners concurrently against the
same keyspace
Patch by Alex Petrov; reviewed by Abe Ratnofsky for CASSANDRA-18315
---
harry-core/src/harry/core/Configuration.java | 51 +++---
harry-core/src/harry/generators/RngUtils.java | 12 +
harry-core/src/harry/model/OpSelectors.java| 19 +--
harry-core/src/harry/runner/Runner.java| 5 ++
.../{RecentValidator.java => RandomValidator.java} | 62 +++---
harry-core/src/harry/visitors/RecentValidator.java | 26 ++---
.../test/harry/model/IntegrationTestBase.java | 2 +-
.../model/QuiescentCheckerIntegrationTest.java | 4 +-
8 files changed, 102 insertions(+), 79 deletions(-)
diff --git a/harry-core/src/harry/core/Configuration.java
b/harry-core/src/harry/core/Configuration.java
index 88474b9..0c0e3f5 100644
--- a/harry-core/src/harry/core/Configuration.java
+++ b/harry-core/src/harry/core/Configuration.java
@@ -61,8 +61,9 @@ import harry.visitors.LoggingVisitor;
import harry.visitors.MutatingVisitor;
import harry.visitors.MutatingRowVisitor;
import harry.visitors.OperationExecutor;
-import harry.visitors.Visitor;
import harry.visitors.RecentValidator;
+import harry.visitors.Visitor;
+import harry.visitors.RandomValidator;
import harry.runner.Runner;
import harry.util.BitSet;
import org.reflections.Reflections;
@@ -291,7 +292,7 @@ public class Configuration
DataTrackerConfiguration data_tracker = new
DefaultDataTrackerConfiguration();
RunnerConfiguration runner;
SutConfiguration system_under_test;
-PDSelectorConfiguration partition_descriptor_selector = new
Configuration.DefaultPDSelectorConfiguration(10, 100);
+PDSelectorConfiguration partition_descriptor_selector = new
Configuration.DefaultPDSelectorConfiguration(10, 100, 0);
CDSelectorConfiguration clustering_descriptor_selector; // TODO:
sensible default value
public ConfigurationBuilder setSeed(long seed)
@@ -719,18 +720,30 @@ public class Configuration
{
public final int window_size;
public final int slide_after_repeats;
+public final long position_offset;
+
+@Deprecated
+public DefaultPDSelectorConfiguration(int window_size,
+ int slide_after_repeats)
+{
+this.window_size = window_size;
+this.slide_after_repeats = slide_after_repeats;
+this.position_offset = 0L;
+}
@JsonCreator
public DefaultPDSelectorConfiguration(@JsonProperty(value =
"window_size", defaultValue = "10") int window_size,
- @JsonProperty(value =
"slide_after_repeats", defaultValue = "100") int slide_after_repeats)
+ @JsonProperty(value =
"slide_after_repeats", defaultValue = "100") int slide_after_repeats,
+ @JsonProperty(value =
"position_offset", defaultValue = "0") long position_offset)
{
this.window_size = window_size;
this.slide_after_repeats = slide_after_repeats;
+this.position_offset = position_offset;
}
public OpSelectors.PdSelector make(OpSelectors.Rng rng)
{
-return new OpSelectors.DefaultPdSelector(rng, window_size,
slide_after_repeats);
+return new OpSelectors.DefaultPdSelector(rng, window_size,
slide_after_repeats, position_offset);
}
}
@@ -1083,27 +1096,49 @@ public class Configuration
public static class RecentPartitionsValidatorConfiguration implements
VisitorConfiguration
{
public final int partition_count;
-public final int trigger_after;
public final int queries;
public final Configuration.ModelConfiguration modelConfiguration;
// TODO: make query selector configurable
@JsonCreator
public
RecentPartitionsValidatorConfiguration(@JsonProperty("partition_count") int
partition_count,
-
@JsonProperty("trigger_after") int trigger_after,
@JsonProperty("queries_per_partition") int queries,
@JsonProperty("model")
Configuration.ModelConfiguration model)
{
this.partition_count = partition_count;
this.queries = queries;
-this.trigger_after = trigger_after;
this.modelConfiguration = model;
}
@Override
pub
[cassandra-harry] 06/09: Allow selecting subsets of columns and wilcard queries. Open API for hand-crafting both mutation and read queries Improve errors Fix a problem with corruptor sorting
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit a8b9869972ddd7adf68e12568d70b2c382ae1ad9
Author: Alex Petrov
AuthorDate: Thu Mar 9 17:28:44 2023 +0100
Allow selecting subsets of columns and wilcard queries.
Open API for hand-crafting both mutation and read queries
Improve errors
Fix a problem with corruptor sorting
Patch by Alex Petrov for CASSANDRA-17603.
---
.../src/harry/corruptor/ChangeValueCorruptor.java | 2 +-
.../harry/corruptor/QueryResponseCorruptor.java| 16 +-
harry-core/src/harry/data/ResultSetRow.java| 7 +
harry-core/src/harry/ddl/SchemaSpec.java | 8 +-
.../src/harry/generators/DataGenerators.java | 2 +
harry-core/src/harry/model/OpSelectors.java| 2 +-
harry-core/src/harry/model/QuiescentChecker.java | 96 --
harry-core/src/harry/model/SelectHelper.java | 142 +++--
harry-core/src/harry/operations/Query.java | 233 ++-
.../src/harry/operations/QueryGenerator.java | 241 +++
.../src/harry/reconciler/PartitionState.java | 266 +
harry-core/src/harry/reconciler/Reconciler.java| 234 +--
.../generators/DataGeneratorsIntegrationTest.java | 2 +-
.../ConcurrentQuiescentCheckerIntegrationTest.java | 2 +-
.../harry/model/HistoryBuilderIntegrationTest.java | 2 +-
.../test/harry/model/ModelTestBase.java| 2 +-
.../harry/model/QuerySelectorNegativeTest.java | 2 -
.../model/QuiescentCheckerIntegrationTest.java | 5 +-
.../QuiescentLocalStateCheckerIntegrationTest.java | 2 +-
.../harry/reconciler/SimpleReconcilerTest.java | 332 +
20 files changed, 1107 insertions(+), 491 deletions(-)
diff --git a/harry-core/src/harry/corruptor/ChangeValueCorruptor.java
b/harry-core/src/harry/corruptor/ChangeValueCorruptor.java
index 5f23a06..3074679 100644
--- a/harry-core/src/harry/corruptor/ChangeValueCorruptor.java
+++ b/harry-core/src/harry/corruptor/ChangeValueCorruptor.java
@@ -75,7 +75,7 @@ public class ChangeValueCorruptor implements RowCorruptor
final long oldV = row.vds[idx];
do
{
-corruptedVds[idx] =+ rng.next();
+corruptedVds[idx] =
schema.regularColumns.get(idx).type.generator().adjustEntropyDomain(rng.next());
}
// we need to find a value that sorts strictly greater than the
current one
while
(schema.regularColumns.get(idx).type.compareLexicographically(corruptedVds[idx],
oldV) <= 0);
diff --git a/harry-core/src/harry/corruptor/QueryResponseCorruptor.java
b/harry-core/src/harry/corruptor/QueryResponseCorruptor.java
index 62bf589..bf29754 100644
--- a/harry-core/src/harry/corruptor/QueryResponseCorruptor.java
+++ b/harry-core/src/harry/corruptor/QueryResponseCorruptor.java
@@ -84,11 +84,25 @@ public interface QueryResponseCorruptor
mismatch = true;
}
}
-assert mismatch || before.length != after.length;
+assert mismatch || before.length != after.length :
String.format("Could not corrupt.\n" +
+
"Before\n%s\n" +
+
"After\n%s\nkma",
+
toString(before),
+
toString(after));
return true;
}
}
return false;
}
+
+private static String toString(Object[][] obj)
+{
+StringBuilder sb = new StringBuilder();
+for (Object[] objects : obj)
+{
+sb.append(Arrays.toString(objects)).append("\n");
+}
+return sb.toString();
+}
}
}
diff --git a/harry-core/src/harry/data/ResultSetRow.java
b/harry-core/src/harry/data/ResultSetRow.java
index d6ff77b..8f37c82 100644
--- a/harry-core/src/harry/data/ResultSetRow.java
+++ b/harry-core/src/harry/data/ResultSetRow.java
@@ -48,6 +48,13 @@ public class ResultSetRow
this.slts = slts;
}
+public ResultSetRow clone()
+{
+return new ResultSetRow(pd, cd,
+Arrays.copyOf(sds, sds.length),
Arrays.copyOf(slts, slts.length),
+Arrays.copyOf(vds, vds.length),
Arrays.copyOf(lts, lts.length));
+}
+
public String toString()
{
return "resultSetRow("
diff --git a/harry-core/src/harry/ddl/SchemaSpec.java
b/harry-core/src/harry/ddl/SchemaSpec.java
index 8d8a62f..4a8a2de 100644
--- a/harry-core/src/harry/ddl/SchemaSp
[cassandra-harry] 01/09: Improvements:
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit d4ecaf80e69ea3bd3d8a9538ae3f70851997b6cf
Author: Abe Ratnofsky
AuthorDate: Mon Oct 17 12:54:44 2022 +0200
Improvements:
* formatting
* thread shutdown on failed run creation
* idempotent query execution
* retry delay
Patch by Abe Ratnofsky; reviewed by Alex Petrov for CASSANDRA-18315.
---
harry-core/src/harry/core/Configuration.java | 56 ++
harry-core/src/harry/ddl/ColumnSpec.java | 16 +++
harry-core/src/harry/ddl/SchemaGenerators.java | 41 ++--
harry-core/src/harry/generators/Bijections.java| 35 +-
harry-core/src/harry/generators/PCGFastPure.java | 2 +-
harry-core/src/harry/model/SelectHelper.java | 2 +-
.../src/harry/model/sut/SystemUnderTest.java | 12 -
harry-core/src/harry/runner/HarryRunner.java | 6 +++
harry-core/src/harry/runner/Runner.java| 9 +++-
harry-core/src/harry/visitors/MutatingVisitor.java | 6 ++-
.../harry/visitors/ParallelRecentValidator.java| 4 +-
harry-core/src/harry/visitors/RecentValidator.java | 2 +-
12 files changed, 145 insertions(+), 46 deletions(-)
diff --git a/harry-core/src/harry/core/Configuration.java
b/harry-core/src/harry/core/Configuration.java
index 3545e18..7cd8f72 100644
--- a/harry-core/src/harry/core/Configuration.java
+++ b/harry-core/src/harry/core/Configuration.java
@@ -65,6 +65,8 @@ public class Configuration
{
private static final ObjectMapper mapper;
+private static final String DEFAULT_KEYSPACE = "harry";
+
static
{
mapper = new ObjectMapper(new YAMLFactory()
@@ -220,34 +222,46 @@ public class Configuration
public static Run createRun(Configuration snapshot)
{
-validate(snapshot);
+SystemUnderTest sut = null;
+try
+{
+validate(snapshot);
-long seed = snapshot.seed;
+long seed = snapshot.seed;
-DataTracker tracker = snapshot.data_tracker == null ? new
DefaultDataTrackerConfiguration().make() : snapshot.data_tracker.make();
-OpSelectors.Rng rng = new OpSelectors.PCGFast(seed);
+DataTracker tracker = snapshot.data_tracker == null ? new
DefaultDataTrackerConfiguration().make() : snapshot.data_tracker.make();
+OpSelectors.Rng rng = new OpSelectors.PCGFast(seed);
-OpSelectors.MonotonicClock clock = snapshot.clock.make();
+OpSelectors.MonotonicClock clock = snapshot.clock.make();
-MetricReporter metricReporter = snapshot.metric_reporter.make();
+MetricReporter metricReporter = snapshot.metric_reporter.make();
-// TODO: validate that operation kind is compatible with schema, due
to statics etc
-SystemUnderTest sut = snapshot.system_under_test.make();
+// TODO: validate that operation kind is compatible with schema,
due to statics etc
+sut = snapshot.system_under_test.make();
-SchemaSpec schemaSpec = snapshot.schema_provider.make(seed, sut);
-schemaSpec.validate();
+SchemaSpec schemaSpec = snapshot.schema_provider.make(seed, sut);
+schemaSpec.validate();
-OpSelectors.PdSelector pdSelector =
snapshot.partition_descriptor_selector.make(rng);
-OpSelectors.DescriptorSelector descriptorSelector =
snapshot.clustering_descriptor_selector.make(rng, schemaSpec);
+OpSelectors.PdSelector pdSelector =
snapshot.partition_descriptor_selector.make(rng);
+OpSelectors.DescriptorSelector descriptorSelector =
snapshot.clustering_descriptor_selector.make(rng, schemaSpec);
-return new Run(rng,
- clock,
- pdSelector,
- descriptorSelector,
- schemaSpec,
- tracker,
- sut,
- metricReporter);
+return new Run(rng,
+ clock,
+ pdSelector,
+ descriptorSelector,
+ schemaSpec,
+ tracker,
+ sut,
+ metricReporter);
+}
+catch (Throwable t)
+{
+// Make sure to shut down all SUT threads if it has been started
+if (sut != null) {
+sut.shutdown();
+}
+throw t;
+}
}
public static Runner createRunner(Configuration config)
@@ -1058,7 +1072,7 @@ public class Configuration
{
public SchemaSpec make(long seed, SystemUnderTest sut)
{
-return SchemaGenerators.defaultSchemaSpecGen("harry", "table0")
+return SchemaGenerators.defaultS
[cassandra-harry] 03/09: Pull in token util from Cassandra to avoid circular dependency
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 6e0b450d9df3596d2eb79bfca92edd434cca7035
Author: Alex Petrov
AuthorDate: Thu Mar 9 17:17:41 2023 +0100
Pull in token util from Cassandra to avoid circular dependency
Patch by Alex Petrov; reviewed by Caleb Rackliffe and Abe Ratnofsky for
CASSANDRA-18315
---
harry-core/src/harry/util/TokenUtil.java | 150 +++
1 file changed, 150 insertions(+)
diff --git a/harry-core/src/harry/util/TokenUtil.java
b/harry-core/src/harry/util/TokenUtil.java
new file mode 100644
index 000..db210b6
--- /dev/null
+++ b/harry-core/src/harry/util/TokenUtil.java
@@ -0,0 +1,150 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package harry.util;
+
+import java.nio.ByteBuffer;
+
+/**
+ * Copied verbatim from Apache Cassandra codebase to avoid having a dependency
on Cassandra.
+ */
+public class TokenUtil
+{
+public static long token(ByteBuffer key)
+{
+if (key.remaining() == 0)
+return Long.MIN_VALUE;
+
+return normalize(getHash(key)[0]);
+}
+
+private static long normalize(long v)
+{
+// We exclude the MINIMUM value; see getToken()
+return v == Long.MIN_VALUE ? Long.MAX_VALUE : v;
+}
+
+private static long[] getHash(ByteBuffer key)
+{
+long[] hash = new long[2];
+hash3_x64_128(key, key.position(), key.remaining(), 0, hash);
+return hash;
+}
+
+public static void hash3_x64_128(ByteBuffer key, int offset, int length,
long seed, long[] result)
+{
+final int nblocks = length >> 4; // Process as 128-bit blocks.
+
+long h1 = seed;
+long h2 = seed;
+
+long c1 = 0x87c37b91114253d5L;
+long c2 = 0x4cf5ad432745937fL;
+
+//--
+// body
+
+for(int i = 0; i < nblocks; i++)
+{
+long k1 = getBlock(key, offset, i * 2 + 0);
+long k2 = getBlock(key, offset, i * 2 + 1);
+
+k1 *= c1; k1 = rotl64(k1,31); k1 *= c2; h1 ^= k1;
+
+h1 = rotl64(h1,27); h1 += h2; h1 = h1*5+0x52dce729;
+
+k2 *= c2; k2 = rotl64(k2,33); k2 *= c1; h2 ^= k2;
+
+h2 = rotl64(h2,31); h2 += h1; h2 = h2*5+0x38495ab5;
+}
+
+//--
+// tail
+
+// Advance offset to the unprocessed tail of the data.
+offset += nblocks * 16;
+
+long k1 = 0;
+long k2 = 0;
+
+switch(length & 15)
+{
+case 15: k2 ^= ((long) key.get(offset+14)) << 48;
+case 14: k2 ^= ((long) key.get(offset+13)) << 40;
+case 13: k2 ^= ((long) key.get(offset+12)) << 32;
+case 12: k2 ^= ((long) key.get(offset+11)) << 24;
+case 11: k2 ^= ((long) key.get(offset+10)) << 16;
+case 10: k2 ^= ((long) key.get(offset+9)) << 8;
+case 9: k2 ^= ((long) key.get(offset+8)) << 0;
+k2 *= c2; k2 = rotl64(k2,33); k2 *= c1; h2 ^= k2;
+
+case 8: k1 ^= ((long) key.get(offset+7)) << 56;
+case 7: k1 ^= ((long) key.get(offset+6)) << 48;
+case 6: k1 ^= ((long) key.get(offset+5)) << 40;
+case 5: k1 ^= ((long) key.get(offset+4)) << 32;
+case 4: k1 ^= ((long) key.get(offset+3)) << 24;
+case 3: k1 ^= ((long) key.get(offset+2)) << 16;
+case 2: k1 ^= ((long) key.get(offset+1)) << 8;
+case 1: k1 ^= ((long) key.get(offset));
+k1 *= c1; k1 = rotl64(k1,31); k1 *= c2; h1 ^= k1;
+};
+
+//--
+// finalization
+
+h1 ^= length; h2 ^= length;
+
+h1 += h2;
+h2 += h1;
+
+h1 = fmix(h1);
+h2 = fmix(h2);
+
+h1 += h2;
+h2 += h1;
+
+result[0] = h1;
+result[1] = h2;
+}
+
+protected static long getBlock(ByteBuffer key, int offset, int index)
+{
+int i_8 = index << 3;
+int blockOffset = offset + i_8;
+return ((long) key.get(blockOffset + 0) & 0xff) + (((long)
key.get(blockOffs
[cassandra-harry] 07/09: Minor API improvements
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git
commit 8a3748c29048b8005dab098ad031b8f75efd942e
Author: Alex Petrov
AuthorDate: Tue Mar 14 13:08:04 2023 +0100
Minor API improvements
Patch by Alex Petrov; reviewed by Abe Ratnofsky for CASSANDRA-18315
---
conf/default.yaml | 3 -
conf/external.yaml | 3 -
harry-core/src/harry/core/Configuration.java | 79 --
.../harry/model/RepairingLocalStateValidator.java | 43 +-
harry-core/src/harry/runner/Runner.java| 4 -
.../src/harry/visitors/AllPartitionsValidator.java | 44 +-
.../src/harry/visitors/CorruptingVisitor.java | 3 +-
.../harry/visitors/ParallelRecentValidator.java| 55 +++-
.../src/harry/visitors/ParallelValidator.java | 24 ++
harry-core/src/harry/visitors/QueryLogger.java | 97 ++
.../harry/visitors/RandomPartitionValidator.java | 10 +--
harry-core/src/harry/visitors/RandomValidator.java | 71 +---
harry-core/src/harry/visitors/RecentValidator.java | 48 ++-
harry-core/src/harry/visitors/Sampler.java | 56 +
.../src/harry/runner/external/MiniStress.java | 5 +-
.../ConcurrentQuiescentCheckerIntegrationTest.java | 3 +-
.../harry/model/InJVMTokenAwareExecutorTest.java | 2 +-
.../model/QuiescentCheckerIntegrationTest.java | 7 +-
.../QuiescentLocalStateCheckerIntegrationTest.java | 2 +-
19 files changed, 273 insertions(+), 286 deletions(-)
diff --git a/conf/default.yaml b/conf/default.yaml
index a004747..4a20782 100644
--- a/conf/default.yaml
+++ b/conf/default.yaml
@@ -87,18 +87,15 @@ runner:
row_visitor:
mutating: {}
- sampler:
- trigger_after: 1000
sample_partitions: 10
- parallel_validate_recent_partitions:
partition_count: 100
queries_per_partition: 2
concurrency: 20
- trigger_after: 1
model:
quiescent_checker: {}
- validate_all_partitions:
concurrency: 20
- trigger_after: 10
model:
quiescent_checker: {}
diff --git a/conf/external.yaml b/conf/external.yaml
index b75547e..ae5fda2 100644
--- a/conf/external.yaml
+++ b/conf/external.yaml
@@ -109,18 +109,15 @@ runner:
row_visitor:
mutating: {}
- sampler:
- trigger_after: 1000
sample_partitions: 10
- parallel_validate_recent_partitions:
partition_count: 100
queries_per_partition: 2
concurrency: 20
- trigger_after: 1
model:
quiescent_checker: {}
- validate_all_partitions:
concurrency: 20
- trigger_after: 10
model:
quiescent_checker: {}
diff --git a/harry-core/src/harry/core/Configuration.java
b/harry-core/src/harry/core/Configuration.java
index 0c0e3f5..9427c89 100644
--- a/harry-core/src/harry/core/Configuration.java
+++ b/harry-core/src/harry/core/Configuration.java
@@ -34,7 +34,6 @@ import java.util.Objects;
import java.util.Set;
import java.util.concurrent.TimeUnit;
import java.util.function.Consumer;
-import java.util.stream.Collectors;
import com.fasterxml.jackson.annotation.JsonCreator;
import com.fasterxml.jackson.annotation.JsonProperty;
@@ -53,17 +52,9 @@ import harry.model.QuiescentChecker;
import harry.model.clock.ApproximateMonotonicClock;
import harry.model.sut.SystemUnderTest;
import harry.runner.LockingDataTracker;
-import harry.visitors.AllPartitionsValidator;
-import harry.visitors.CorruptingVisitor;
+import harry.visitors.*;
import harry.runner.DataTracker;
import harry.runner.DefaultDataTracker;
-import harry.visitors.LoggingVisitor;
-import harry.visitors.MutatingVisitor;
-import harry.visitors.MutatingRowVisitor;
-import harry.visitors.OperationExecutor;
-import harry.visitors.RecentValidator;
-import harry.visitors.Visitor;
-import harry.visitors.RandomValidator;
import harry.runner.Runner;
import harry.util.BitSet;
import org.reflections.Reflections;
@@ -500,8 +491,6 @@ public class Configuration
}
}
-
-
@JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include =
JsonTypeInfo.As.WRAPPER_OBJECT)
public interface ClockConfiguration extends
OpSelectors.MonotonicClockFactory
{
@@ -1054,24 +1043,22 @@ public class Configuration
public static class AllPartitionsValidatorConfiguration implements
VisitorConfiguration
{
public final int concurrency;
-public final int trigger_after;
-
-@JsonProperty("model")
-public final Configuration.ModelConfiguration modelConfiguration;
+public final QueryLoggerConfiguration query_logger;
+public final Configuration
