[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700023#comment-17700023 ] Stefan Miklosovic commented on CASSANDRA-18320: --- Actually, we do not need to wait for this to appear in 4.1.1 because this is just test-related. > Incompatible file system thrown while running Simulator > --- > > Key: CASSANDRA-18320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18320 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: David Capwell >Priority: Normal > Fix For: 4.1.x, 5.x > > > {code} > java.io.UncheckedIOException > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831) > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047) > at > org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816) > at > org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370) > at > org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345) > at > org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148) > at > org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33) > Caused by: java.nio.file.DirectoryNotEmptyException: > /cassandra/node1/commitlog > at > com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535) > at > com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517) > at > com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479) > at > com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465) > at > com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261) > at java.nio.file.Files.delete(Files.java:1126) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18326) Debian package repository misconfiguration
Tibor Repasi created CASSANDRA-18326: Summary: Debian package repository misconfiguration Key: CASSANDRA-18326 URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 Project: Cassandra Issue Type: Bug Components: Packaging Reporter: Tibor Repasi Debian apt is failing on current jfrog repository access for 40x releases with: {code} W: Conflicting distribution: https://debian.cassandra.apache.org 40x InRelease (expected 40x but got 40) E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its 'Codename' value from '40x' to '40' N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details. {code} This is caused by the typo in [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] containing {code} Codename: 40 {code} but it is expected to be {code} Codename: 40x {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski commented on CASSANDRA-18143: --- 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 4.0 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18143: -- Reviewers: Jacek Lewandowski, Stefan Miklosovic, Stefan Miklosovic (was: Jacek Lewandowski) Jacek Lewandowski, Stefan Miklosovic, Stefan Miklosovic (was: Jacek Lewandowski, Stefan Miklosovic) Status: Review In Progress (was: Patch Available) > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18143: -- Test and Documentation Plan: ci Status: Patch Available (was: In Progress) > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 10:00 AM: - 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 4.1 5.0 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 4.0 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700067#comment-17700067 ] Stefan Miklosovic commented on CASSANDRA-18326: --- [~brandon.williams] would you help us with this one, please? > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Priority: Normal > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 10:31 AM: - 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 5.0 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18037) Use snake case for the names CQL native functions
[ https://issues.apache.org/jira/browse/CASSANDRA-18037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18037: -- Change Category: Semantic Complexity: Normal Status: Open (was: Triage Needed) > Use snake case for the names CQL native functions > - > > Key: CASSANDRA-18037 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18037 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Syntax >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > > Most native functions are named all lower case, without underscore nor > hyphen to separate words. That's the case, for example, of "intasblob" or > "blobasint". > We also have some functions using camel case, as in "castAsInt" or > "castAsTimestamp". Note that the came cased names require quoting due to > CQL's case insensitivity. > Differently to CQL native functions, system keyspaces, tables and columns > consistently use snake case. For example, we have "system_schema", > "dropped_columns", "default_time_to_live". > As discussed in [this > thread|https://lists.apache.org/thread/k9ml1k4fg6o7mfby1nr3y0mnq9r90dym], we > should adopt snake_case for CQL native function names. Also we should provide > aliases for the current function names, so we don't break compatibility. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams reassigned CASSANDRA-18326: Assignee: Brandon Williams > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700095#comment-17700095 ] Brandon Williams commented on CASSANDRA-18326: -- Yep, this is my fault my manually uploading the packages in CASSANDRA-18307. I thought I had fixed everything from the '40' mistake, but it looks like I missed this. > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700095#comment-17700095 ] Brandon Williams edited comment on CASSANDRA-18326 at 3/14/23 11:05 AM: Yep, this is my fault from my manually uploading the packages in CASSANDRA-18307. I thought I had fixed everything from the '40' mistake, but it looks like I missed this. was (Author: brandon.williams): Yep, this is my fault my manually uploading the packages in CASSANDRA-18307. I thought I had fixed everything from the '40' mistake, but it looks like I missed this. > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700100#comment-17700100 ] Brandon Williams commented on CASSANDRA-18320: -- This looks good to me and I can unsurprisingly no longer reproduce, +1. > Incompatible file system thrown while running Simulator > --- > > Key: CASSANDRA-18320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18320 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: David Capwell >Priority: Normal > Fix For: 4.1.x, 5.x > > > {code} > java.io.UncheckedIOException > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831) > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047) > at > org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816) > at > org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370) > at > org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345) > at > org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148) > at > org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33) > Caused by: java.nio.file.DirectoryNotEmptyException: > /cassandra/node1/commitlog > at > com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535) > at > com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517) > at > com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479) > at > com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465) > at > com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261) > at java.nio.file.Files.delete(Files.java:1126) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700103#comment-17700103 ] Brandon Williams commented on CASSANDRA-18326: -- I saw that indeed 'Release' was the old file and re-uploaded that which seems to have solved it; I'm going to go ahead and do the whole debian directory again since it is idempotent and I don't know if anything else didn't take. > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18327) Fix link to .tar.gz download in documentation
Arnout Engelen created CASSANDRA-18327: -- Summary: Fix link to .tar.gz download in documentation Key: CASSANDRA-18327 URL: https://issues.apache.org/jira/browse/CASSANDRA-18327 Project: Cassandra Issue Type: Improvement Components: Documentation Reporter: Arnout Engelen On [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball] the reader is instructed to: {{curl -OL [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}} However, that mirror no longer exists. It would be nicer to use a URL such as [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz], but that requires having the current Cassandra version available as a variable to asciidoc and inserting in into a code snippet, which I'm not sure is easy to do. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18327) Fix link to .tar.gz download in documentation
[ https://issues.apache.org/jira/browse/CASSANDRA-18327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18327: - Change Category: Operability Complexity: Normal Status: Open (was: Triage Needed) > Fix link to .tar.gz download in documentation > - > > Key: CASSANDRA-18327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18327 > Project: Cassandra > Issue Type: Improvement > Components: Documentation >Reporter: Arnout Engelen >Priority: Normal > > On > [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball] > the reader is instructed to: > > {{curl -OL > [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}} > However, that mirror no longer exists. > It would be nicer to use a URL such as > [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz], > but that requires having the current Cassandra version available as a > variable to asciidoc and inserting in into a code snippet, which I'm not sure > is easy to do. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18327) Fix link to .tar.gz download in documentation
[ https://issues.apache.org/jira/browse/CASSANDRA-18327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arnout Engelen updated CASSANDRA-18327: --- Description: On [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball] the reader is instructed to: {{curl -OL [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}} However, that mirror no longer exists. It would be nicer to use a URL such as [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz], but that requires having the current Cassandra version available as a variable to asciidoc and inserting in into a code snippet, which I'm not sure is easy to do. Alternatively we could point people to [https://cassandra.apache.org/_/download.html] instead of showing a curl command. was: On [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball] the reader is instructed to: {{curl -OL [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}} However, that mirror no longer exists. It would be nicer to use a URL such as [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz], but that requires having the current Cassandra version available as a variable to asciidoc and inserting in into a code snippet, which I'm not sure is easy to do. > Fix link to .tar.gz download in documentation > - > > Key: CASSANDRA-18327 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18327 > Project: Cassandra > Issue Type: Improvement > Components: Documentation >Reporter: Arnout Engelen >Priority: Normal > > On > [https://cassandra.apache.org/doc/latest/cassandra/getting_started/installing.html#installing-the-binary-tarball] > the reader is instructed to: > > {{curl -OL > [http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz]}} > However, that mirror no longer exists. > It would be nicer to use a URL such as > [https://www.apache.org/dyn/closer.lua/download/cassandra/4.1.0/apache-cassandra-4.1.0-bin.tar.gz], > but that requires having the current Cassandra version available as a > variable to asciidoc and inserting in into a code snippet, which I'm not sure > is easy to do. > Alternatively we could point people to > [https://cassandra.apache.org/_/download.html] instead of showing a curl > command. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700114#comment-17700114 ] Brandon Williams commented on CASSANDRA-18326: -- bq. do the whole debian directory This is done now, [~rtib] please confirm everything works for you. > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18326: - Bug Category: Parent values: Correctness(12982) Complexity: Normal Discovered By: User Report Fix Version/s: NA Severity: Normal Status: Open (was: Triage Needed) > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
Andres de la Peña created CASSANDRA-18328: - Summary: Remove deprecated CQL functions dateof and unixtimestampof Key: CASSANDRA-18328 URL: https://issues.apache.org/jira/browse/CASSANDRA-18328 Project: Cassandra Issue Type: Task Components: CQL/Syntax Reporter: Andres de la Peña Assignee: Andres de la Peña The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on Cassandra 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa], almost eight years ago. They were deprecated in favour of the then new {{totimestamp}} and {{tounixtimestamp}} functions. A note about their deprecation was added to [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423], and they were marked as deprecated on [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions]. They are also listed as deprecated on [the new doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions]. We can finally remove those functions in 5.0, since they have been deprecated for so long. Discussion thread [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[ https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18328: -- Fix Version/s: 5.x > Remove deprecated CQL functions dateof and unixtimestampof > -- > > Key: CASSANDRA-18328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18328 > Project: Cassandra > Issue Type: Task > Components: CQL/Syntax >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 5.x > > > The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on > Cassandra > 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa], > almost eight years ago. They were deprecated in favour of the then new > {{totimestamp}} and {{tounixtimestamp}} functions. > A note about their deprecation was added to > [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423], > and they were marked as deprecated on > [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions]. > They are also listed as deprecated on [the new > doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions]. > We can finally remove those functions in 5.0, since they have been deprecated > for so long. > Discussion thread > [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[ https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18328: -- Epic Link: CASSANDRA-18306 > Remove deprecated CQL functions dateof and unixtimestampof > -- > > Key: CASSANDRA-18328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18328 > Project: Cassandra > Issue Type: Task > Components: CQL/Syntax >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 5.x > > > The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on > Cassandra > 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa], > almost eight years ago. They were deprecated in favour of the then new > {{totimestamp}} and {{tounixtimestamp}} functions. > A note about their deprecation was added to > [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423], > and they were marked as deprecated on > [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions]. > They are also listed as deprecated on [the new > doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions]. > We can finally remove those functions in 5.0, since they have been deprecated > for so long. > Discussion thread > [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700139#comment-17700139 ] Tibor Repasi commented on CASSANDRA-18326: -- Hi [~brandon.williams] , it looks good to me. Apt update took it and recent version is available now. > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18326) Debian package repository misconfiguration
[ https://issues.apache.org/jira/browse/CASSANDRA-18326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18326: - Resolution: Fixed Status: Resolved (was: Open) Thanks for confirming! > Debian package repository misconfiguration > -- > > Key: CASSANDRA-18326 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18326 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Tibor Repasi >Assignee: Brandon Williams >Priority: Normal > Fix For: NA > > > Debian apt is failing on current jfrog repository access for 40x releases > with: > {code} > W: Conflicting distribution: https://debian.cassandra.apache.org 40x > InRelease (expected 40x but got 40) > E: Repository 'https://debian.cassandra.apache.org 40x InRelease' changed its > 'Codename' value from '40x' to '40' > N: This must be accepted explicitly before updates for this repository can be > applied. See apt-secure(8) manpage for details. > {code} > This is caused by the typo in > [dists/40x/Release|https://debian.cassandra.apache.org/dists/40x/Release] > containing > {code} > Codename: 40 > {code} > but it is expected to be > {code} > Codename: 40x > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 12:46 PM: - 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 5.0 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[ https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700151#comment-17700151 ] Jacek Lewandowski commented on CASSANDRA-18153: --- [~abonacin] can you provide a full reproduce example? > Memtable being flushed without hostId in version "me" and newer during > CommitLogReplay > -- > > Key: CASSANDRA-18153 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18153 > Project: Cassandra > Issue Type: Bug > Components: Local/SSTable >Reporter: Adriano Bonacin >Assignee: Adriano Bonacin >Priority: Normal > Time Spent: 50m > Remaining Estimate: 0h > > On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store > HostID in the new "me" SSTable version. > But SSTables flushed during CommitLogReplay miss this HostID info. > > In the next Cassandra startup, if these SSTables were still present, > system.log will show: > {{WARN Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored}} > {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }} > > And debug.log will show a list of SSTables, witch can include "md" and "me" > version (before upgradesstables): > > {{Ignored commitLogIntervals from the following sstables: > [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}} > > https://issues.apache.org/jira/browse/CASSANDRA-16619 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18329) Upgrade jamm
Ekaterina Dimitrova created CASSANDRA-18329: --- Summary: Upgrade jamm Key: CASSANDRA-18329 URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 Project: Cassandra Issue Type: Task Reporter: Ekaterina Dimitrova Jamm is currently under maintenance that will solve JDK11 issues and enable it to work with post JDK11+ versions up to JDK17. This ticket will serve as a placeholder for upgrading Jamm in Cassandra when the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18329) Upgrade jamm
[ https://issues.apache.org/jira/browse/CASSANDRA-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-18329: Change Category: Operability Complexity: Normal Component/s: Jamm Status: Open (was: Triage Needed) > Upgrade jamm > > > Key: CASSANDRA-18329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 > Project: Cassandra > Issue Type: Task > Components: Jamm >Reporter: Ekaterina Dimitrova >Priority: Normal > > Jamm is currently under maintenance that will solve JDK11 issues and enable > it to work with post JDK11+ versions up to JDK17. > This ticket will serve as a placeholder for upgrading Jamm in Cassandra when > the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18329) Upgrade jamm
[ https://issues.apache.org/jira/browse/CASSANDRA-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-18329: Epic Link: CASSANDRA-16895 > Upgrade jamm > > > Key: CASSANDRA-18329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 > Project: Cassandra > Issue Type: Task > Components: Jamm >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > Jamm is currently under maintenance that will solve JDK11 issues and enable > it to work with post JDK11+ versions up to JDK17. > This ticket will serve as a placeholder for upgrading Jamm in Cassandra when > the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18329) Upgrade jamm
[ https://issues.apache.org/jira/browse/CASSANDRA-18329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-18329: Fix Version/s: 4.0.x 4.1.x 5.x > Upgrade jamm > > > Key: CASSANDRA-18329 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18329 > Project: Cassandra > Issue Type: Task > Components: Jamm >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > Jamm is currently under maintenance that will solve JDK11 issues and enable > it to work with post JDK11+ versions up to JDK17. > This ticket will serve as a placeholder for upgrading Jamm in Cassandra when > the new Jamm release is out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700164#comment-17700164 ] Stefan Miklosovic commented on CASSANDRA-18143: --- All good. 4.0 should be restarted though. On successful build there I am +1 and we can commit. Thanks. > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 1:35 PM: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a, https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/5ed83a49-a560-46ae-a8db-708a55082a66 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 was (Author: jlewandowski): 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700171#comment-17700171 ] Jacek Lewandowski commented on CASSANDRA-18143: --- 4.0 unit tests retried and passed. All looking good, merging, thanks for the review > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-18143: -- Status: Ready to Commit (was: Review In Progress) > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700045#comment-17700045 ] Jacek Lewandowski edited comment on CASSANDRA-18143 at 3/14/23 1:36 PM: 3.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.0 https://github.com/apache/cassandra/pull/2214 3.11 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-3.11 4.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.0 4.1 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143-4.1 5.0 https://github.com/jacek-lewandowski/cassandra/tree/CASSANDRA-18143 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a, https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/5ed83a49-a560-46ae-a8db-708a55082a66 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 was (Author: jlewandowski): 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a, https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/5ed83a49-a560-46ae-a8db-708a55082a66 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 Tests: 3.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/617/workflows/c4a9facb-4918-4b78-9019-cb98e55a983a 3.11 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/613/workflows/ca64b8de-6d25-474f-a8be-e1644ffacedb 4.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/614/workflows/8df14d46-21dd-4ff5-b40d-bb9e71af6a5a 4.1 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/615/workflows/9872e6b8-8432-4126-bfa7-0767faa95391 5.0 https://app.circleci.com/pipelines/github/jacek-lewandowski/cassandra/616/workflows/d584f8c4-0481-490e-aeb5-b58b3e57b0d6 > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (8ba3888fcf -> deb8abd2fd)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 8ba3888fcf Merge branch 'cassandra-4.0' into cassandra-4.1 add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 add 5104008db9 Merge branch 'cassandra-3.11' into cassandra-4.0 add deb8abd2fd Merge branch 'cassandra-4.0' into cassandra-4.1 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (02048cffea -> 8df69c8016)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 02048cffea Remove -l / -m / -h designation and have two options: free or paid tier circle config add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.1' into trunk
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit d1ba381d96cf1f864f1f9d0c59c5efa4cf660f49 Merge: 7ad746cc83 deb8abd2fd Author: Jacek Lewandowski AuthorDate: Tue Mar 14 15:19:22 2023 +0100 Merge branch 'cassandra-4.1' into trunk * cassandra-4.1: Fix the ordering of sstables when running sstableupgrade tool CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 13 + src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 16 insertions(+), 2 deletions(-) diff --cc src/java/org/apache/cassandra/db/Directories.java index cf07e995ac,9f80ce0619..e7c2535a6e --- a/src/java/org/apache/cassandra/db/Directories.java +++ b/src/java/org/apache/cassandra/db/Directories.java @@@ -59,17 -49,11 +59,18 @@@ import org.apache.cassandra.db.lifecycl import org.apache.cassandra.io.FSDiskFullWriteError; import org.apache.cassandra.io.FSError; import org.apache.cassandra.io.FSNoDiskAvailableForWriteError; +import org.apache.cassandra.io.FSReadError; import org.apache.cassandra.io.FSWriteError; +import org.apache.cassandra.io.sstable.Component; +import org.apache.cassandra.io.sstable.Descriptor; +import org.apache.cassandra.io.sstable.SSTable; +import org.apache.cassandra.io.sstable.SSTableId; ++import org.apache.cassandra.io.sstable.SSTableIdFactory; +import org.apache.cassandra.io.util.File; +import org.apache.cassandra.io.util.FileStoreUtils; import org.apache.cassandra.io.util.FileUtils; -import org.apache.cassandra.io.sstable.*; -import org.apache.cassandra.schema.SchemaConstants; import org.apache.cassandra.io.util.PathUtils; +import org.apache.cassandra.schema.SchemaConstants; import org.apache.cassandra.schema.TableMetadata; import org.apache.cassandra.service.snapshot.SnapshotManifest; import org.apache.cassandra.service.snapshot.TableSnapshot; diff --cc src/java/org/apache/cassandra/tools/StandaloneUpgrader.java index 657ec05d65,a7c099c7e5..a7a9dcdbb4 --- a/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java +++ b/src/java/org/apache/cassandra/tools/StandaloneUpgrader.java @@@ -84,11 -76,11 +84,11 @@@ public class StandaloneUpgrade Collection readers = new ArrayList<>(); - // Upgrade sstables - for (Map.Entry> entry : lister.list().entrySet()) + // Upgrade sstables in id order + for (Map.Entry> entry : lister.sortedList()) { Set components = entry.getValue(); -if (!components.contains(Component.DATA) || !components.contains(Component.PRIMARY_INDEX)) +if (!components.containsAll(entry.getKey().getFormat().primaryComponents())) continue; try - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (7ad746cc83 -> d1ba381d96)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 7ad746cc83 Revert "Switch checkstyle to j11." add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 add 5104008db9 Merge branch 'cassandra-3.11' into cassandra-4.0 add deb8abd2fd Merge branch 'cassandra-4.0' into cassandra-4.1 new d1ba381d96 Merge branch 'cassandra-4.1' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 13 + src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 16 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (4b3cb0a772 -> 58d06d34ab)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 4b3cb0a772 Merge branch 'cassandra-3.0' into cassandra-3.11 add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (3cfb222122 -> 5104008db9)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 3cfb222122 Fix BufferPool.memoryInUse counter add 8df69c8016 Fix the ordering of sstables when running sstableupgrade tool add 58d06d34ab Merge branch 'cassandra-3.0' into cassandra-3.11 add 5104008db9 Merge branch 'cassandra-3.11' into cassandra-4.0 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + src/java/org/apache/cassandra/db/Directories.java | 12 src/java/org/apache/cassandra/tools/StandaloneUpgrader.java | 4 ++-- 3 files changed, 15 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18143) upgradesstables does not always upgrade tables in proper order.
[ https://issues.apache.org/jira/browse/CASSANDRA-18143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-18143: -- Fix Version/s: 3.0.29 3.11.15 4.0.9 4.1.1 5.0 Since Version: 3.0.0 Source Control Link: https://github.com/apache/cassandra/commit/8df69c8016fb84e59e558e74508a0767ad3a7cc0 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed, thanks [~claude] and [~smiklosovic] > upgradesstables does not always upgrade tables in proper order. > --- > > Key: CASSANDRA-18143 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18143 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Claude Warren >Assignee: Claude Warren >Priority: Normal > Fix For: 3.0.29, 3.11.15, 4.0.9, 4.1.1, 5.0 > > > The SSTableUpgrader accepts tools in the hash order provided by > Directories.SSTableLister rather than ordering them to ensure that they are > upgraded in the proper order. > They should be ordered by their id. The comparator for SSTableId is available > in SSTableIdFactory.COMPARATOR. > > Dev discussion thread: > https://lists.apache.org/thread/w6pm5hbdxt295mtvlckv0joyk8x4o8nf -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18260) Add details to Error message: Not enough space for compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-18260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Henrik Ingo reassigned CASSANDRA-18260: --- Assignee: Henrik Ingo > Add details to Error message: Not enough space for compaction > -- > > Key: CASSANDRA-18260 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18260 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Logging >Reporter: Brad Schoening >Assignee: Henrik Ingo >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > > When compaction fails, the log message should list a) the free space > available on disk at that point in time and b) perhaps the number and/or size > of the source sstables being compacted. > Free space can change from one moment to the next, so when the below > compaction failed because it needed 161GB, upon checking the server a few > minutes later, it had 184GB free. Similarly, the error message mentions it > was writing out one SSTable on this STCS table, but its not clear if it was > combining X -> 1 tables, or something else. > [INFO ] [CompactionExecutor:77758] cluster_id=87 ip_address=127.1.1.1 > CompactionTask.java:241 - Compacted (8a1cffe0-abb5-11ed-b3fc-8d2ac2c52f0d) 1 > sstables to [...] to level=0. 86.997GiB to 86.997GiB (~99% of original) in > 1,508,155ms. Read Throughput = 59.069MiB/s, Write Throughput = 59.069MiB/s, > Row Throughput = ~20,283/s. 21,375 total partitions merged to 21,375. > Partition merge counts were \{1:21375, } > [ERROR] [CompactionExecutor:4] cluster_id=87 ip_address=127.1.1.1 > CassandraDaemon.java:581 - Exception in thread > Thread[CompactionExecutor:4,1,main] > java.lang.RuntimeException: Not enough space for compaction, estimated > sstables = 1, expected write size = 161228934093 > at > org.apache.cassandra.db.compaction.CompactionTask.buildCompactionCandidatesForAvailableDiskSpace(CompactionTask.java:386) > at > org.apache.cassandra.db.compaction.CompactionTask.runMayThrow(CompactionTask.java:126) > at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) > at > org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:77) > at > org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:100) > at > org.apache.cassandra.db.compaction.CompactionManager$7.execute(CompactionManager.java:613) > at > org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:377) > at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[ https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700237#comment-17700237 ] Jacek Lewandowski commented on CASSANDRA-18153: --- I don't know exactly how to reproduce this, but assuming this is a possible scenario, I want to propose a mix of something brought by all of us. In particular: - in all the places where we try to get host id, we try first from token metadata and then from system.local - if we start a fresh instance (that is, there is no commit log files to replay), we conditionally create a host id in the system.local table and flush that table; we do that along with storing other local node metadata, before touching schema or the commit log With that, we are guaranteed that host id is stored in sstable and not as a mutation in the commit log before we actually start to replay anything. > Memtable being flushed without hostId in version "me" and newer during > CommitLogReplay > -- > > Key: CASSANDRA-18153 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18153 > Project: Cassandra > Issue Type: Bug > Components: Local/SSTable >Reporter: Adriano Bonacin >Assignee: Adriano Bonacin >Priority: Normal > Time Spent: 50m > Remaining Estimate: 0h > > On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store > HostID in the new "me" SSTable version. > But SSTables flushed during CommitLogReplay miss this HostID info. > > In the next Cassandra startup, if these SSTables were still present, > system.log will show: > {{WARN Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored}} > {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }} > > And debug.log will show a list of SSTables, witch can include "md" and "me" > version (before upgradesstables): > > {{Ignored commitLogIntervals from the following sstables: > [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}} > > https://issues.apache.org/jira/browse/CASSANDRA-16619 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[ https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700237#comment-17700237 ] Jacek Lewandowski edited comment on CASSANDRA-18153 at 3/14/23 2:53 PM: I don't know exactly how to reproduce this, but assuming this is a possible scenario, I want to propose a mix of something brought by all of us. In particular: - in all the places where we try to get host id, we try first from token metadata and then from system.local - if we start a fresh instance (that is, there is no commit log files to replay), we conditionally create a host id in the system.local table and flush that table; we do that along with storing other local node metadata, before touching schema or the commit log https://issues.apache.org/jira/browse/CASSANDRA-18153 (3.11) With that, we are guaranteed that host id is stored in sstable and not as a mutation in the commit log before we actually start to replay anything. was (Author: jlewandowski): I don't know exactly how to reproduce this, but assuming this is a possible scenario, I want to propose a mix of something brought by all of us. In particular: - in all the places where we try to get host id, we try first from token metadata and then from system.local - if we start a fresh instance (that is, there is no commit log files to replay), we conditionally create a host id in the system.local table and flush that table; we do that along with storing other local node metadata, before touching schema or the commit log With that, we are guaranteed that host id is stored in sstable and not as a mutation in the commit log before we actually start to replay anything. > Memtable being flushed without hostId in version "me" and newer during > CommitLogReplay > -- > > Key: CASSANDRA-18153 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18153 > Project: Cassandra > Issue Type: Bug > Components: Local/SSTable >Reporter: Adriano Bonacin >Assignee: Adriano Bonacin >Priority: Normal > Time Spent: 50m > Remaining Estimate: 0h > > On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store > HostID in the new "me" SSTable version. > But SSTables flushed during CommitLogReplay miss this HostID info. > > In the next Cassandra startup, if these SSTables were still present, > system.log will show: > {{WARN Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored}} > {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }} > > And debug.log will show a list of SSTables, witch can include "md" and "me" > version (before upgradesstables): > > {{Ignored commitLogIntervals from the following sstables: > [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}} > > https://issues.apache.org/jira/browse/CASSANDRA-16619 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18153) Memtable being flushed without hostId in version "me" and newer during CommitLogReplay
[ https://issues.apache.org/jira/browse/CASSANDRA-18153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700237#comment-17700237 ] Jacek Lewandowski edited comment on CASSANDRA-18153 at 3/14/23 2:53 PM: I don't know exactly how to reproduce this, but assuming this is a possible scenario, I want to propose a mix of something brought by all of us. In particular: - in all the places where we try to get host id, we try first from token metadata and then from system.local - if we start a fresh instance (that is, there is no commit log files to replay), we conditionally create a host id in the system.local table and flush that table; we do that along with storing other local node metadata, before touching schema or the commit log With that, we are guaranteed that host id is stored in sstable and not as a mutation in the commit log before we actually start to replay anything. https://github.com/apache/cassandra/pull/2215 (3.11) was (Author: jlewandowski): I don't know exactly how to reproduce this, but assuming this is a possible scenario, I want to propose a mix of something brought by all of us. In particular: - in all the places where we try to get host id, we try first from token metadata and then from system.local - if we start a fresh instance (that is, there is no commit log files to replay), we conditionally create a host id in the system.local table and flush that table; we do that along with storing other local node metadata, before touching schema or the commit log https://issues.apache.org/jira/browse/CASSANDRA-18153 (3.11) With that, we are guaranteed that host id is stored in sstable and not as a mutation in the commit log before we actually start to replay anything. > Memtable being flushed without hostId in version "me" and newer during > CommitLogReplay > -- > > Key: CASSANDRA-18153 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18153 > Project: Cassandra > Issue Type: Bug > Components: Local/SSTable >Reporter: Adriano Bonacin >Assignee: Adriano Bonacin >Priority: Normal > Time Spent: 50m > Remaining Estimate: 0h > > On ticket CASSANDRA-16619 some files were changed to allow Cassandra to store > HostID in the new "me" SSTable version. > But SSTables flushed during CommitLogReplay miss this HostID info. > > In the next Cassandra startup, if these SSTables were still present, > system.log will show: > {{WARN Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored}} > {{WARN }}{{{}Origin of 3 sstables is unknown or doesn't match the local node; > commitLogIntervals for them were ignored{}}}{{{}{}}}{{ }} > > And debug.log will show a list of SSTables, witch can include "md" and "me" > version (before upgradesstables): > > {{Ignored commitLogIntervals from the following sstables: > [/var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/me-3-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-1-big-Data.db, > > /var/lib/cassandra/data/system/compaction_history-b4dbb7b4dc493fb5b3bfce6e434832ca/md-2-big-Data.db]}} > > https://issues.apache.org/jira/browse/CASSANDRA-16619 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: disable jdk switching
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/trunk by this push: new efdd2223 disable jdk switching efdd2223 is described below commit efdd222343e76a9660716af2126ef7b348e18624 Author: Brandon Williams AuthorDate: Thu Mar 2 11:18:26 2023 -0600 disable jdk switching Patch by brandonwilliams; reviewed by edimitrova and mck for CASSANDRA-18106 --- upgrade_tests/upgrade_base.py | 21 - upgrade_tests/upgrade_through_versions_test.py | 3 --- 2 files changed, 24 deletions(-) diff --git a/upgrade_tests/upgrade_base.py b/upgrade_tests/upgrade_base.py index dbd2fc57..30f51b25 100644 --- a/upgrade_tests/upgrade_base.py +++ b/upgrade_tests/upgrade_base.py @@ -17,25 +17,6 @@ from dtest import Tester, create_ks logger = logging.getLogger(__name__) -def switch_jdks(major_version_int): -""" -Changes the jdk version globally, by setting JAVA_HOME = JAVA[N]_HOME. -This means the environment must have JAVA[N]_HOME set to switch to jdk version N. -""" -new_java_home = 'JAVA{}_HOME'.format(major_version_int) - -try: -os.environ[new_java_home] -except KeyError: -raise RuntimeError("You need to set {} to run these tests!".format(new_java_home)) - -# don't change if the same version was requested -current_java_home = os.environ.get('JAVA_HOME') -if current_java_home != os.environ[new_java_home]: -logger.debug("Switching jdk to version {} (JAVA_HOME is changing from {} to {})".format(major_version_int, current_java_home or 'undefined', os.environ[new_java_home])) -os.environ['JAVA_HOME'] = os.environ[new_java_home] - - @pytest.mark.upgrade_test @pytest.mark.skipif(sys.platform == 'win32', reason='Skip upgrade tests on Windows') class UpgradeTester(Tester, metaclass=ABCMeta): @@ -73,7 +54,6 @@ class UpgradeTester(Tester, metaclass=ABCMeta): previous_java_home = os.environ['JAVA_HOME'] previous_cassandra_version = os.environ['CASSANDRA_VERSION'] if 'CASSANDRA_VERSION' in os.environ else None -switch_jdks(self.UPGRADE_PATH.starting_meta.java_version) os.environ['CASSANDRA_VERSION'] = self.UPGRADE_PATH.starting_version yield @@ -170,7 +150,6 @@ class UpgradeTester(Tester, metaclass=ABCMeta): node1.mark_log_for_errors() logger.debug('upgrading node1 to {}'.format(self.UPGRADE_PATH.upgrade_version)) -switch_jdks(self.UPGRADE_PATH.upgrade_meta.java_version) node1.set_install_dir(version=self.UPGRADE_PATH.upgrade_version) self.install_legacy_parsing(node1) diff --git a/upgrade_tests/upgrade_through_versions_test.py b/upgrade_tests/upgrade_through_versions_test.py index d711a10b..022ccc2e 100644 --- a/upgrade_tests/upgrade_through_versions_test.py +++ b/upgrade_tests/upgrade_through_versions_test.py @@ -20,7 +20,6 @@ from cassandra.query import SimpleStatement from dtest import RUN_STATIC_UPGRADE_MATRIX, Tester from tools.misc import generate_ssl_stores, new_node -from .upgrade_base import switch_jdks from .upgrade_manifest import (build_upgrade_pairs, current_2_1_x, current_2_2_x, current_3_0_x, indev_3_11_x, @@ -328,7 +327,6 @@ class TestUpgrade(Tester): cluster = self.cluster cluster.set_install_dir(version=self.test_version_metas[0].version) self.install_nodetool_legacy_parsing() -switch_jdks(self.test_version_metas[0].java_version) self.fixture_dtest_setup.reinitialize_cluster_for_different_version() logger.debug("Versions to test (%s): %s" % (type(self), str([v.version for v in self.test_version_metas]))) @@ -508,7 +506,6 @@ class TestUpgrade(Tester): and upgrade all nodes. """ logger.debug('Upgrading {nodes} to {version}'.format(nodes=[n.name for n in nodes] if nodes is not None else 'all nodes', version=version_meta.version)) -switch_jdks(version_meta.java_version) logger.debug("JAVA_HOME: " + os.environ.get('JAVA_HOME')) if not partial: nodes = self.cluster.nodelist() - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Remove JAVA8/JAVA11_HOME from circle configs
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 5c51f05dab Remove JAVA8/JAVA11_HOME from circle configs 5c51f05dab is described below commit 5c51f05dab0562c3b95b60224e4698e2a30d2ce7 Author: Brandon Williams AuthorDate: Fri Mar 10 09:47:56 2023 -0600 Remove JAVA8/JAVA11_HOME from circle configs Patch by brandonwilliams; reviewed by edimitrova and mck for CASSANDRA-18106 --- .circleci/config.yml | 190 --- .circleci/config.yml.FREE| 190 --- .circleci/config.yml.PAID| 190 --- .circleci/config_template.yml| 2 - .circleci/config_template.yml.PAID.patch | 68 +-- 5 files changed, 34 insertions(+), 606 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index bb41b759fc..74e5d21c86 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -95,8 +95,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -186,8 +184,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -252,8 +248,6 @@ jobs: path: /tmp/cassandra/pylib environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -363,8 +357,6 @@ jobs: destination: dtest_j8_with_vnodes_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -519,8 +511,6 @@ jobs: destination: dtest_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -606,8 +596,6 @@ jobs: destination: dtest_j8_large_with_vnodes_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -725,8 +713,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -799,8 +785,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -890,8 +874,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -1001,8 +983,6 @@ jobs: destination: dtest_j11_without_vnodes_logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -1093,8 +1073,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR: /home/cassandra/cassandra-dtest @@ -1212,8 +1190,6 @@ jobs: destination: logs environment: - ANT_HOME: /usr/share/ant -- JAVA11_HOME: /usr/lib/jvm/java-11-openjdk-amd64 -- JAVA8_HOME: /usr/lib/jvm/java-8-openjdk-amd64 - LANG: en_US.UTF-8 - KEEP_TEST_DIR: true - DEFAULT_DIR:
[jira] [Updated] (CASSANDRA-18106) Update CCM for JDK17 and revise current JDK detection strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-18106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-18106: - Fix Version/s: 5.0 (was: 5.x) Source Control Link: https://github.com/apache/cassandra/commit/5c51f05dab0562c3b95b60224e4698e2a30d2ce7 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed and tagged CCM, cassandra-dtest, and the circle configs. Thanks! > Update CCM for JDK17 and revise current JDK detection strategy > -- > > Key: CASSANDRA-18106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18106 > Project: Cassandra > Issue Type: Task > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 5.0 > > Attachments: Screenshot 2023-03-03 at 09.24.50.png > > > As part of CASSANDRA-16895 initial POC an initial version of CCM patch was > created. This needs to be revisited and reviewed > Recently we closed CASSANDRA-18039 which brought questions, probably we need > to revise how we detect JDK versions in CCM and whether it is correct. To the > best of my knowledge there are certain tests in the repo around that and they > pass so my guess is we need to revise just the strategy and maybe document it > explicitly or consider if we want any changes to be applied. Also, we need to > be careful with breaking changes. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18317) Properly synchronize CQLSSTableWriter#build on the Schema.instance
[ https://issues.apache.org/jira/browse/CASSANDRA-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-18317: Reviewers: Alex Petrov, Yifan Cai (was: Yifan Cai) Status: Review In Progress (was: Needs Committer) > Properly synchronize CQLSSTableWriter#build on the Schema.instance > -- > > Key: CASSANDRA-18317 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18317 > Project: Cassandra > Issue Type: Bug > Components: Tool/bulk load >Reporter: Francisco Guerrero >Assignee: Francisco Guerrero >Priority: Normal > Time Spent: 0.5h > Remaining Estimate: 0h > > The {{CQLSSTableWriter#build}} method should properly synchronize on the > {{Schema.instance}} class to prevent concurrent Schema operations fail, [when > the offline tool also updates schema]. > For example, a table creation operation, which modifies the keyspaces tables > metadata, might end up missing the update when a concurrent call to the > {{CQLSSTableWriter#build}} method is accessing the {{Schema}} instance. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18330) Delivery of CEP-21: Transactional Cluster Metadata
Sam Tunnicliffe created CASSANDRA-18330: --- Summary: Delivery of CEP-21: Transactional Cluster Metadata Key: CASSANDRA-18330 URL: https://issues.apache.org/jira/browse/CASSANDRA-18330 Project: Cassandra Issue Type: Epic Reporter: Sam Tunnicliffe -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[ https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700292#comment-17700292 ] Andres de la Peña commented on CASSANDRA-18328: --- ||PR||CI|| |[trunk|https://github.com/apache/cassandra/pull/2216]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/a13238cb-37b3-44af-a5e0-38daac07b285] [j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/040bc564-7f99-4794-80ea-380c5a4b3123]| |[dtests|https://github.com/apache/cassandra-dtest/pull/214]|| > Remove deprecated CQL functions dateof and unixtimestampof > -- > > Key: CASSANDRA-18328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18328 > Project: Cassandra > Issue Type: Task > Components: CQL/Syntax >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 5.x > > > The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on > Cassandra > 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa], > almost eight years ago. They were deprecated in favour of the then new > {{totimestamp}} and {{tounixtimestamp}} functions. > A note about their deprecation was added to > [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423], > and they were marked as deprecated on > [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions]. > They are also listed as deprecated on [the new > doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions]. > We can finally remove those functions in 5.0, since they have been deprecated > for so long. > Discussion thread > [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18330) Delivery of CEP-21: Transactional Cluster Metadata
[ https://issues.apache.org/jira/browse/CASSANDRA-18330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-18330: Change Category: Semantic Complexity: Byzantine Component/s: Cluster/Membership Cluster/Schema Assignee: Sam Tunnicliffe Status: Open (was: Triage Needed) > Delivery of CEP-21: Transactional Cluster Metadata > -- > > Key: CASSANDRA-18330 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18330 > Project: Cassandra > Issue Type: Epic > Components: Cluster/Membership, Cluster/Schema >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[ https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18328: -- Change Category: Semantic Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > Remove deprecated CQL functions dateof and unixtimestampof > -- > > Key: CASSANDRA-18328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18328 > Project: Cassandra > Issue Type: Task > Components: CQL/Syntax >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 5.x > > > The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on > Cassandra > 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa], > almost eight years ago. They were deprecated in favour of the then new > {{totimestamp}} and {{tounixtimestamp}} functions. > A note about their deprecation was added to > [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423], > and they were marked as deprecated on > [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions]. > They are also listed as deprecated on [the new > doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions]. > We can finally remove those functions in 5.0, since they have been deprecated > for so long. > Discussion thread > [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18328) Remove deprecated CQL functions dateof and unixtimestampof
[ https://issues.apache.org/jira/browse/CASSANDRA-18328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andres de la Peña updated CASSANDRA-18328: -- Test and Documentation Plan: ||PR||CI|| |[trunk|https://github.com/apache/cassandra/pull/2216]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/a13238cb-37b3-44af-a5e0-38daac07b285] [j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/2712/workflows/040bc564-7f99-4794-80ea-380c5a4b3123]| |[dtests|https://github.com/apache/cassandra-dtest/pull/214]|| Status: Patch Available (was: Open) > Remove deprecated CQL functions dateof and unixtimestampof > -- > > Key: CASSANDRA-18328 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18328 > Project: Cassandra > Issue Type: Task > Components: CQL/Syntax >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 5.x > > > The CQL functions {{dateof}} and {{unixtimestampof}} were [deprecated on > Cassandra > 2.2.0|https://github.com/apache/cassandra/commit/c08aaabd95d4872593c29807de6ec1485cefa7fa], > almost eight years ago. They were deprecated in favour of the then new > {{totimestamp}} and {{tounixtimestamp}} functions. > A note about their deprecation was added to > [{{NEWS.txt}}|https://github.com/apache/cassandra/blob/trunk/NEWS.txt#L1421-L1423], > and they were marked as deprecated on > [{{CQL.textile}}|https://github.com/apache/cassandra/blob/trunk/doc/cql3/CQL.textile#time-conversion-functions]. > They are also listed as deprecated on [the new > doc|https://github.com/apache/cassandra/blob/trunk/doc/modules/cassandra/pages/cql/functions.adoc#time-conversion-functions]. > We can finally remove those functions in 5.0, since they have been deprecated > for so long. > Discussion thread > [here|https://lists.apache.org/thread/0gs824fpsngn5dr0yq2x1h8qsflj5sr0]. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[ https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700303#comment-17700303 ] Caleb Rackliffe commented on CASSANDRA-18302: - Let's look at the particular example from {{testSetMapElementFromMapElementReference}}, assuming the one-line fix I mentioned [above|https://issues.apache.org/jira/browse/CASSANDRA-18302?focusedCommentId=17699878&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17699878] is in place... {noformat} SELECT int_map[?] FROM distributed_test_keyspace.tbl0 WHERE k = 0 {noformat} ...becomes... {noformat} SELECT int_map['one'] FROM distributed_test_keyspace.tbl0 WHERE k = 0 {noformat} The only difference here between the original query and the {{asCQL()}} output is the value binding for {{'one'}}. Combined w/ a line number, it seems pretty easy to see what's going on. (In this case, the only reason I think we found the problem is that we forgot to use {{LazyToString}} in {{createNamedReads()}}? CC [~dcapwell]) Are there other examples where the disconnect is more problematic? Aside from all that, the {{asCQL()}} and {{toCQLString()}} complex has been around for a while, and the former seems like it needs the fix above independent of Accord. It would be nice to avoid more new complexity than necessary, but I guess what we have to hash out is how much new complexity is worth having error messages w/ the _exact_ CQL strings supplied by the user. Interested in other opinions here... > Fix AIOOBE and improve validation messages for transaction statements > - > > Key: CASSANDRA-18302 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18302 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently it happens sometimes that ArrayIndexOutOfBoundsException is thrown > from asCql method when validation transaction statement. In addition, asCql > does not return precisely the query user entered so the whole error message > can be misleading. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[ https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700303#comment-17700303 ] Caleb Rackliffe edited comment on CASSANDRA-18302 at 3/14/23 4:43 PM: -- Let's look at the particular example from {{testSetMapElementFromMapElementReference}}, assuming the one-line fix I mentioned [above|https://issues.apache.org/jira/browse/CASSANDRA-18302?focusedCommentId=17699878&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17699878] is in place... {noformat} SELECT int_map[?] FROM distributed_test_keyspace.tbl0 WHERE k = 0 {noformat} ...becomes... {noformat} SELECT int_map['one'] FROM distributed_test_keyspace.tbl0 WHERE k = 0 {noformat} The only difference here between the original query and the {{asCQL()}} output is the value binding for {{'one'}}. Combined w/ a line number, it seems pretty easy to see what's going on. (In this case, the only reason I think we found the problem is that we forgot to use {{LazyToString}} in {{createNamedReads()}}? CC [~dcapwell]) Are there other examples where the disconnect is more problematic? Aside from all that, the {{asCQL()}} and {{toCQLString()}} complex has been around for a while, and the former seems like it needs the fix above independent of Accord. It would be nice to avoid more new complexity than necessary, but I guess what we have to hash out is how much new complexity (and up-front, non-lazy processing) is worth having error messages w/ the _exact_ CQL strings supplied by the user. Interested in other opinions here... was (Author: maedhroz): Let's look at the particular example from {{testSetMapElementFromMapElementReference}}, assuming the one-line fix I mentioned [above|https://issues.apache.org/jira/browse/CASSANDRA-18302?focusedCommentId=17699878&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17699878] is in place... {noformat} SELECT int_map[?] FROM distributed_test_keyspace.tbl0 WHERE k = 0 {noformat} ...becomes... {noformat} SELECT int_map['one'] FROM distributed_test_keyspace.tbl0 WHERE k = 0 {noformat} The only difference here between the original query and the {{asCQL()}} output is the value binding for {{'one'}}. Combined w/ a line number, it seems pretty easy to see what's going on. (In this case, the only reason I think we found the problem is that we forgot to use {{LazyToString}} in {{createNamedReads()}}? CC [~dcapwell]) Are there other examples where the disconnect is more problematic? Aside from all that, the {{asCQL()}} and {{toCQLString()}} complex has been around for a while, and the former seems like it needs the fix above independent of Accord. It would be nice to avoid more new complexity than necessary, but I guess what we have to hash out is how much new complexity is worth having error messages w/ the _exact_ CQL strings supplied by the user. Interested in other opinions here... > Fix AIOOBE and improve validation messages for transaction statements > - > > Key: CASSANDRA-18302 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18302 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently it happens sometimes that ArrayIndexOutOfBoundsException is thrown > from asCql method when validation transaction statement. In addition, asCql > does not return precisely the query user entered so the whole error message > can be misleading. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 03/25: [CEP-21] Include current epoch in internode header
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 56d289b4fbe104cbf9a3320c0ed7b93991dcaa2d Author: Sam Tunnicliffe AuthorDate: Wed Mar 1 10:03:44 2023 + [CEP-21] Include current epoch in internode header Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../org/apache/cassandra/db/CounterMutation.java | 6 + src/java/org/apache/cassandra/db/Mutation.java | 5 + src/java/org/apache/cassandra/net/Message.java | 130 + .../org/apache/cassandra/net/MessagingService.java | 6 +- 4 files changed, 121 insertions(+), 26 deletions(-) diff --git a/src/java/org/apache/cassandra/db/CounterMutation.java b/src/java/org/apache/cassandra/db/CounterMutation.java index 4f91b83ca2..86086902aa 100644 --- a/src/java/org/apache/cassandra/db/CounterMutation.java +++ b/src/java/org/apache/cassandra/db/CounterMutation.java @@ -51,6 +51,7 @@ import static java.util.concurrent.TimeUnit.*; import static org.apache.cassandra.net.MessagingService.VERSION_30; import static org.apache.cassandra.net.MessagingService.VERSION_3014; import static org.apache.cassandra.net.MessagingService.VERSION_40; +import static org.apache.cassandra.net.MessagingService.VERSION_50; import static org.apache.cassandra.utils.Clock.Global.nanoTime; public class CounterMutation implements IMutation @@ -334,6 +335,7 @@ public class CounterMutation implements IMutation private int serializedSize30; private int serializedSize3014; private int serializedSize40; +private int serializedSize50; public int serializedSize(int version) { @@ -351,6 +353,10 @@ public class CounterMutation implements IMutation if (serializedSize40 == 0) serializedSize40 = (int) serializer.serializedSize(this, VERSION_40); return serializedSize40; +case VERSION_50: +if (serializedSize50 == 0) +serializedSize50 = (int) serializer.serializedSize(this, VERSION_50); +return serializedSize50; default: throw new IllegalStateException("Unknown serialization version: " + version); } diff --git a/src/java/org/apache/cassandra/db/Mutation.java b/src/java/org/apache/cassandra/db/Mutation.java index ad43b16d48..9a5cdabb50 100644 --- a/src/java/org/apache/cassandra/db/Mutation.java +++ b/src/java/org/apache/cassandra/db/Mutation.java @@ -317,6 +317,7 @@ public class Mutation implements IMutation, Supplier private int serializedSize30; private int serializedSize3014; private int serializedSize40; +private int serializedSize50; public int serializedSize(int version) { @@ -334,6 +335,10 @@ public class Mutation implements IMutation, Supplier if (serializedSize40 == 0) serializedSize40 = (int) serializer.serializedSize(this, VERSION_40); return serializedSize40; +case VERSION_50: +if (serializedSize50 == 0) +serializedSize50 = (int) serializer.serializedSize(this, VERSION_50); +return serializedSize50; default: throw new IllegalStateException("Unknown serialization version: " + version); } diff --git a/src/java/org/apache/cassandra/net/Message.java b/src/java/org/apache/cassandra/net/Message.java index fa14134933..03562540b4 100644 --- a/src/java/org/apache/cassandra/net/Message.java +++ b/src/java/org/apache/cassandra/net/Message.java @@ -25,6 +25,7 @@ import java.util.HashMap; import java.util.Map; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicInteger; +import java.util.function.Supplier; import com.google.common.annotations.VisibleForTesting; import com.google.common.primitives.Ints; @@ -40,11 +41,11 @@ import org.apache.cassandra.io.util.DataInputBuffer; import org.apache.cassandra.io.util.DataInputPlus; import org.apache.cassandra.io.util.DataOutputPlus; import org.apache.cassandra.locator.InetAddressAndPort; +import org.apache.cassandra.tcm.Epoch; +import org.apache.cassandra.tcm.ClusterMetadata; import org.apache.cassandra.tracing.Tracing; import org.apache.cassandra.tracing.Tracing.TraceType; -import org.apache.cassandra.utils.MonotonicClockTranslation; -import org.apache.cassandra.utils.NoSpamLogger; -import org.apache.cassandra.utils.TimeUUID; +import org.apache.cassandra.utils.*; import static java.util.concurrent.TimeUnit.MINUTES; import static java.util.concurrent.TimeUnit.NANOSECONDS; @@ -67,6 +68,8 @@ public class Message private static final Logger logger = LoggerFactory.getLogger(Message.class); private static final NoSpamLogger noSpam1m = NoSpamLogger.getLogger(logger, 1, TimeUnit.MINUTES); +pr
[cassandra] 05/25: [CEP-21] Add basics of ownership and data placement
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 2434132cc7685729b5bb24446d9257b23bc5c113 Author: Sam Tunnicliffe AuthorDate: Wed Mar 1 12:52:29 2023 + [CEP-21] Add basics of ownership and data placement Introduce new classes for representing placement of data ranges on replicas, along with the movement of data via transitions from one placement to the next. Eventually, these placements will be statically calculated in response to events with alter either the topology of the cluster (i.e. adding/removing/moving nodes) or the replication profile of the data itself (i.e. creating/altering keyspaces). These triggering events will be distributed and enacted consistently using the global log. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../cassandra/dht/OrderPreservingPartitioner.java | 24 + src/java/org/apache/cassandra/dht/Range.java | 27 ++ src/java/org/apache/cassandra/dht/Token.java | 27 ++ .../apache/cassandra/locator/RangesByEndpoint.java | 69 +++ .../cassandra/tcm/ownership/DataPlacement.java | 211 + .../cassandra/tcm/ownership/DataPlacements.java| 216 + .../org/apache/cassandra/tcm/ownership/Delta.java | 125 + .../tcm/ownership/GlobalPlacementDelta.java| 163 +++ .../cassandra/tcm/ownership/MovementMap.java | 89 .../cassandra/tcm/ownership/PlacementDeltas.java | 244 ++ .../cassandra/tcm/ownership/PlacementForRange.java | 525 + .../cassandra/tcm/ownership/PlacementProvider.java | 53 +++ .../tcm/ownership/PlacementTransitionPlan.java | 128 + .../cassandra/tcm/ownership/ReplicationMap.java| 106 + .../cassandra/tcm/sequences/LockedRanges.java | 430 + 15 files changed, 2437 insertions(+) diff --git a/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java b/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java index 2d4def95d0..fb9d0011a7 100644 --- a/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java +++ b/src/java/org/apache/cassandra/dht/OrderPreservingPartitioner.java @@ -45,6 +45,15 @@ public class OrderPreservingPartitioner implements IPartitioner private static final String rndchars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"; public static final StringToken MINIMUM = new StringToken(""); +public static final StringToken MAXIMUM = new StringToken("") { +public int compareTo(Token o) +{ +if (o == MAXIMUM) +return 0; + +return 1; +} +}; public static final BigInteger CHAR_MASK = new BigInteger("65535"); @@ -116,6 +125,11 @@ public class OrderPreservingPartitioner implements IPartitioner return MINIMUM; } +public StringToken getMaximumToken() +{ +return MAXIMUM; +} + public StringToken getRandomToken() { return getRandomToken(ThreadLocalRandom.current()); @@ -208,6 +222,16 @@ public class OrderPreservingPartitioner implements IPartitioner { return ByteSource.of(token, version); } + +@Override +public int compareTo(Token o) +{ +// todo (rebase): I have no recollection of why this is needed - investigate +if (o == MAXIMUM) +return -1; + +return super.compareTo(o); +} } public StringToken getToken(ByteBuffer key) diff --git a/src/java/org/apache/cassandra/dht/Range.java b/src/java/org/apache/cassandra/dht/Range.java index 2c468990d2..581602cccd 100644 --- a/src/java/org/apache/cassandra/dht/Range.java +++ b/src/java/org/apache/cassandra/dht/Range.java @@ -17,6 +17,7 @@ */ package org.apache.cassandra.dht; +import java.io.IOException; import java.io.Serializable; import java.util.*; import java.util.function.Predicate; @@ -25,6 +26,11 @@ import com.google.common.collect.Iterables; import org.apache.commons.lang3.ObjectUtils; import org.apache.cassandra.db.PartitionPosition; +import org.apache.cassandra.io.util.DataInputPlus; +import org.apache.cassandra.io.util.DataOutputPlus; +import org.apache.cassandra.net.MessagingService; +import org.apache.cassandra.tcm.serialization.MetadataSerializer; +import org.apache.cassandra.tcm.serialization.Version; import org.apache.cassandra.utils.Pair; /** @@ -38,6 +44,7 @@ import org.apache.cassandra.utils.Pair; */ public class Range> extends AbstractBounds implements Comparable>, Serializable { +public static final Serializer serializer = new Serializer(); public static final long serialVersionUID = 1L; public Range(T left, T right) @@ -626,4 +633,24 @@ public class Range> extends AbstractBounds imp
[cassandra] 08/25: [CEP-21] Start to remove and deprecate gossip functionality
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 5a09d9441902bcb99051ca23bd19191ee85424f8 Author: Sam Tunnicliffe AuthorDate: Thu Mar 2 14:00:55 2023 + [CEP-21] Start to remove and deprecate gossip functionality WIP commit (i.e. does not compile) beginning the process of removing gossip as the source of truth regarding membership, ownership, topology and data placement. This task will be split over mutiple commits. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../statements/schema/AlterKeyspaceStatement.java | 24 +- .../apache/cassandra/db/filter/ColumnFilter.java | 98 +-- .../org/apache/cassandra/gms/ApplicationState.java | 38 +- .../org/apache/cassandra/gms/EndpointState.java| 27 +- .../org/apache/cassandra/gms/FailureDetector.java | 12 +- .../org/apache/cassandra/gms/GossipDigestAck.java | 2 +- .../cassandra/gms/GossipDigestAckVerbHandler.java | 11 +- .../cassandra/gms/GossipDigestSynVerbHandler.java | 5 +- src/java/org/apache/cassandra/gms/Gossiper.java| 728 + .../org/apache/cassandra/gms/GossiperEvent.java| 2 +- src/java/org/apache/cassandra/gms/NewGossiper.java | 143 .../org/apache/cassandra/gms/VersionedValue.java | 96 ++- .../schema/SystemDistributedKeyspace.java | 45 +- .../cassandra/tcm/compatibility/GossipHelper.java | 303 + .../apache/cassandra/tracing/TraceKeyspace.java| 14 +- 15 files changed, 774 insertions(+), 774 deletions(-) diff --git a/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java b/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java index 9efccd5464..250b18ffed 100644 --- a/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java +++ b/src/java/org/apache/cassandra/cql3/statements/schema/AlterKeyspaceStatement.java @@ -48,6 +48,7 @@ import org.apache.cassandra.schema.Keyspaces; import org.apache.cassandra.schema.Keyspaces.KeyspacesDiff; import org.apache.cassandra.service.ClientState; import org.apache.cassandra.tcm.ClusterMetadata; +import org.apache.cassandra.tcm.membership.NodeId; import org.apache.cassandra.transport.Event.SchemaChange; import org.apache.cassandra.transport.Event.SchemaChange.Change; import org.apache.cassandra.utils.FBUtilities; @@ -91,7 +92,7 @@ public final class AlterKeyspaceStatement extends AlterSchemaStatement newKeyspace.params.validate(keyspaceName, state); newKeyspace.replicationStrategy.validate(); - +validateNoRangeMovements(); validateTransientReplication(keyspace.replicationStrategy, newKeyspace.replicationStrategy); // Because we used to not properly validate unrecognized options, we only log a warning if we find one. @@ -140,14 +141,19 @@ public final class AlterKeyspaceStatement extends AlterSchemaStatement if (allow_alter_rf_during_range_movement) return; -Stream unreachableNotAdministrativelyInactive = -Gossiper.instance.getUnreachableMembers().stream().filter(endpoint -> !FBUtilities.getBroadcastAddressAndPort().equals(endpoint) && - !Gossiper.instance.isAdministrativelyInactiveState(endpoint)); -Stream endpoints = Stream.concat(Gossiper.instance.getLiveMembers().stream(), - unreachableNotAdministrativelyInactive); -List notNormalEndpoints = endpoints.filter(endpoint -> !FBUtilities.getBroadcastAddressAndPort().equals(endpoint) && - !Gossiper.instance.getEndpointStateForEndpoint(endpoint).isNormalState()) - .collect(Collectors.toList()); +ClusterMetadata metadata = ClusterMetadata.current(); +NodeId nodeId = metadata.directory.peerId(FBUtilities.getBroadcastAddressAndPort()); +Set notNormalEndpoints = metadata.directory.states.entrySet().stream().filter(e -> !e.getKey().equals(nodeId)).filter(e -> { +switch (e.getValue()) +{ +case BOOTSTRAPPING: +case LEAVING: +case MOVING: +return true; +default: +return false; +} +}).map(e -> metadata.directory.endpoint(e.getKey())).collect(Collectors.toSet()); if (!notNormalEndpoints.isEmpty()) { throw new ConfigurationException("Cannot alter RF while some endpoints are not in normal state (no range movements): " + notNormalEndpoints); diff --git a/src/java/org/apache/cassandra/db/filt
[cassandra] 13/25: [CEP-21] Update ColumnFamilyStore (4/7)
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit da5b8ec9cedc29122ec1a3e84f42f8af11f6b650 Author: Sam Tunnicliffe AuthorDate: Thu Mar 2 19:20:39 2023 + [CEP-21] Update ColumnFamilyStore (4/7) Part 4 of 7 modifications to ColumnFamilyStore, mostly related to: * ShardBoundaries * DiskBoundaries Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../org/apache/cassandra/db/ColumnFamilyStore.java | 28 +++-- .../org/apache/cassandra/db/DiskBoundaries.java| 23 ++-- .../apache/cassandra/db/DiskBoundaryManager.java | 84 -- .../cassandra/db/memtable/ShardBoundaries.java | 13 ++- .../org/apache/cassandra/tcm/ClusterMetadata.java | 124 + 5 files changed, 168 insertions(+), 104 deletions(-) diff --git a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java index 0affba5814..99e0bcde68 100644 --- a/src/java/org/apache/cassandra/db/ColumnFamilyStore.java +++ b/src/java/org/apache/cassandra/db/ColumnFamilyStore.java @@ -90,14 +90,14 @@ import org.apache.cassandra.db.compaction.CompactionStrategyManager; import org.apache.cassandra.db.compaction.OperationType; import org.apache.cassandra.db.filter.ClusteringIndexFilter; import org.apache.cassandra.db.filter.DataLimits; +import org.apache.cassandra.db.memtable.Flushing; +import org.apache.cassandra.db.memtable.Memtable; +import org.apache.cassandra.db.memtable.ShardBoundaries; import org.apache.cassandra.db.lifecycle.LifecycleNewTracker; import org.apache.cassandra.db.lifecycle.LifecycleTransaction; import org.apache.cassandra.db.lifecycle.SSTableSet; import org.apache.cassandra.db.lifecycle.Tracker; import org.apache.cassandra.db.lifecycle.View; -import org.apache.cassandra.db.memtable.Flushing; -import org.apache.cassandra.db.memtable.Memtable; -import org.apache.cassandra.db.memtable.ShardBoundaries; import org.apache.cassandra.db.partitions.CachedPartition; import org.apache.cassandra.db.partitions.PartitionUpdate; import org.apache.cassandra.db.repair.CassandraTableRepairManager; @@ -162,6 +162,7 @@ import org.apache.cassandra.service.snapshot.SnapshotManifest; import org.apache.cassandra.service.snapshot.TableSnapshot; import org.apache.cassandra.streaming.TableStreamManager; import org.apache.cassandra.tcm.ClusterMetadata; +import org.apache.cassandra.tcm.Epoch; import org.apache.cassandra.utils.ByteBufferUtil; import org.apache.cassandra.utils.DefaultValue; import org.apache.cassandra.utils.ExecutorUtils; @@ -520,6 +521,7 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean, Memtable.Owner data.addInitialSSTablesWithoutUpdatingSize(sstables, this); } +// compaction strategy should be created after the CFS has been prepared compactionStrategyManager = new CompactionStrategyManager(this); if (maxCompactionThreshold.value() <= 0 || minCompactionThreshold.value() <=0) @@ -1460,17 +1462,16 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean, Memtable.Owner if (shardBoundaries == null || shardBoundaries.shardCount() != shardCount || -shardBoundaries.ringVersion != -1 && shardBoundaries.ringVersion != StorageService.instance.getTokenMetadata().getRingVersion()) +!shardBoundaries.epoch.equals(Epoch.EMPTY) && !shardBoundaries.epoch.equals(metadata.epoch)) { List weightedRanges; -long ringVersion; +Epoch epoch; if (!SchemaConstants.isLocalSystemKeyspace(keyspace.getName()) -&& getPartitioner() == metadata.current().partitioner) +&& getPartitioner() == metadata.current().tokenMap.partitioner()) { DiskBoundaryManager.VersionedRangesAtEndpoint versionedLocalRanges = DiskBoundaryManager.getVersionedLocalRanges(this); Set> localRanges = versionedLocalRanges.rangesAtEndpoint.ranges(); -ringVersion = versionedLocalRanges.ringVersion; - +epoch = versionedLocalRanges.epoch; if (!localRanges.isEmpty()) { @@ -1494,12 +1495,12 @@ public class ColumnFamilyStore implements ColumnFamilyStoreMBean, Memtable.Owner // Local tables need to cover the full token range and don't care about ring changes. // We also end up here if the table's partitioner is not the database's, which can happen in tests. weightedRanges = fullWeightedRange(); -ringVersion = -1; +epoch = Epoch.EMPTY; } List boundaries = getPartitioner().splitter().get().splitOwnedRanges(shardCount, weightedRanges, false);
[cassandra] 18/25: [CEP-21] Add missing implementations to concurrent utils
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit bb615495bac125b00caeea93919680dfced6fb6f Author: Sam Tunnicliffe AuthorDate: Fri Mar 3 16:37:47 2023 + [CEP-21] Add missing implementations to concurrent utils Adds a handful of implementations to subclasses in the org.apache.cassandra.utils.concurrent package Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../cassandra/utils/concurrent/AbstractFuture.java | 33 ++ .../cassandra/utils/concurrent/AsyncFuture.java| 11 .../apache/cassandra/utils/concurrent/Future.java | 10 +++ .../cassandra/utils/concurrent/LoadingMap.java | 7 + .../cassandra/utils/concurrent/SyncFuture.java | 11 5 files changed, 72 insertions(+) diff --git a/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java b/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java index 83cd7d3f8b..a6c5aeda8a 100644 --- a/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java +++ b/src/java/org/apache/cassandra/utils/concurrent/AbstractFuture.java @@ -383,6 +383,39 @@ public abstract class AbstractFuture implements Future return result; } +/** + * Support {@link com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture, AsyncFunction, Executor)} natively + * + * See {@link #addListener(GenericFutureListener)} for ordering semantics. + */ +@Override +public Future andThenAsync(Function> andThen) +{ +return andThenAsync(andThen, null); +} + +/** + * Support {@link com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture, AsyncFunction, Executor)} natively + * + * See {@link #addListener(GenericFutureListener)} for ordering semantics. + */ +protected Future andThenAsync(AbstractFuture result, Function> andThen, @Nullable Executor executor) +{ +addListener(() -> { +try +{ +if (isSuccess()) andThen.apply(getNow()).addListener(propagate(result)); +else result.tryFailure(cause()); +} +catch (Throwable t) +{ +result.tryFailure(t); +throw t; +} +}, executor); +return result; +} + /** * Add a listener to be invoked once this future completes. * Listeners are submitted to {@link #notifyExecutor} in the order they are added (or the specified executor diff --git a/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java b/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java index 0ef35d5dde..89cea2dbe3 100644 --- a/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java +++ b/src/java/org/apache/cassandra/utils/concurrent/AsyncFuture.java @@ -144,6 +144,17 @@ public class AsyncFuture extends AbstractFuture return flatMap(new AsyncFuture<>(), flatMapper, executor); } +/** + * Support {@link com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture, AsyncFunction, Executor)} natively + * + * See {@link #addListener(GenericFutureListener)} for ordering semantics. + */ +@Override +public Future andThenAsync(Function> andThen, @Nullable Executor executor) +{ +return andThenAsync(new AsyncFuture<>(), andThen, executor); +} + /** * Wait for this future to complete {@link Awaitable#await()} */ diff --git a/src/java/org/apache/cassandra/utils/concurrent/Future.java b/src/java/org/apache/cassandra/utils/concurrent/Future.java index fae5d43ef1..909d140d17 100644 --- a/src/java/org/apache/cassandra/utils/concurrent/Future.java +++ b/src/java/org/apache/cassandra/utils/concurrent/Future.java @@ -177,6 +177,16 @@ public interface Future extends io.netty.util.concurrent.Future, Listenabl */ Future flatMap(Function> flatMapper, Executor executor); +/** + * Support {@link com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture, AsyncFunction, Executor)} natively + */ + Future andThenAsync(Function> andThen); + +/** + * Support {@link com.google.common.util.concurrent.Futures#transformAsync(ListenableFuture, AsyncFunction, Executor)} natively + */ + Future andThenAsync(Function> andThen, Executor executor); + /** * Invoke {@code runnable} on completion, using {@code executor}. * diff --git a/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java b/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java index 399eb0e3b5..1d41a97db8 100644 --- a/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java +++ b/src/java/org/apache/cassandra/utils/concurrent/LoadingMap.java @@ -18,6
[cassandra] 17/25: [CEP-21] New nodetool commands
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7bda86e1fe57be880155dd7cb652d415f0650f58 Author: Sam Tunnicliffe AuthorDate: Fri Mar 3 15:41:59 2023 + [CEP-21] New nodetool commands Adds new nodetool commands to: * list members of the CMS * initiate a snapshot of ClusterMetadata via submitting a SealPeriod operation Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- src/java/org/apache/cassandra/tools/NodeProbe.java | 5 src/java/org/apache/cassandra/tools/NodeTool.java | 2 ++ .../cassandra/tools/nodetool/DescribeCMS.java | 34 ++ .../cassandra/tools/nodetool/SealPeriod.java | 33 + 4 files changed, 74 insertions(+) diff --git a/src/java/org/apache/cassandra/tools/NodeProbe.java b/src/java/org/apache/cassandra/tools/NodeProbe.java index e983e2fbef..ab051843f1 100644 --- a/src/java/org/apache/cassandra/tools/NodeProbe.java +++ b/src/java/org/apache/cassandra/tools/NodeProbe.java @@ -327,6 +327,11 @@ public class NodeProbe implements AutoCloseable return ssProxy.forceKeyspaceCleanup(jobs, keyspaceName, tables); } +public void sealPeriod() +{ +ssProxy.sealPeriod(); +} + public int scrub(boolean disableSnapshot, boolean skipCorrupted, boolean checkData, boolean reinsertOverflowedTTL, int jobs, String keyspaceName, String... tables) throws IOException, ExecutionException, InterruptedException { return ssProxy.scrub(disableSnapshot, skipCorrupted, checkData, reinsertOverflowedTTL, jobs, keyspaceName, tables); diff --git a/src/java/org/apache/cassandra/tools/NodeTool.java b/src/java/org/apache/cassandra/tools/NodeTool.java index 5500240477..35a673c406 100644 --- a/src/java/org/apache/cassandra/tools/NodeTool.java +++ b/src/java/org/apache/cassandra/tools/NodeTool.java @@ -107,6 +107,7 @@ public class NodeTool DataPaths.class, Decommission.class, DescribeCluster.class, +DescribeCMS.class, DescribeRing.class, DisableAuditLog.class, DisableAutoCompaction.class, @@ -190,6 +191,7 @@ public class NodeTool ResumeHandoff.class, Ring.class, Scrub.class, +SealPeriod.class, SetAuthCacheConfig.class, SetBatchlogReplayThrottle.class, SetCacheCapacity.class, diff --git a/src/java/org/apache/cassandra/tools/nodetool/DescribeCMS.java b/src/java/org/apache/cassandra/tools/nodetool/DescribeCMS.java new file mode 100644 index 00..ce34af4bf8 --- /dev/null +++ b/src/java/org/apache/cassandra/tools/nodetool/DescribeCMS.java @@ -0,0 +1,34 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.cassandra.tools.nodetool; + +import io.airlift.airline.Command; +import org.apache.cassandra.tools.NodeProbe; +import org.apache.cassandra.tools.NodeTool; + +@Command(name = "describecms", description = "Describe the current Cluster Metadata Service") +public class DescribeCMS extends NodeTool.NodeToolCmd +{ +@Override +protected void execute(NodeProbe probe) +{ +System.out.println("ClusterMetadataService:"); +System.out.println(probe.getStorageService().describeCMS()); +} +} diff --git a/src/java/org/apache/cassandra/tools/nodetool/SealPeriod.java b/src/java/org/apache/cassandra/tools/nodetool/SealPeriod.java new file mode 100644 index 00..65bd699ca3 --- /dev/null +++ b/src/java/org/apache/cassandra/tools/nodetool/SealPeriod.java @@ -0,0 +1,33 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may
[cassandra] branch cep-21-tcm created (now 2d133ba464)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git at 2d133ba464 TMP - use bundled version of harry This branch includes the following new commits: new 7b6b04a8bf [CEP-21] Add distributed metadata log new 728a9d9a07 [CEP-21] Move schema management to TCM new 56d289b4fb [CEP-21] Include current epoch in internode header new 35910f2425 [CEP-21] Add rudimentary cluster membership to TCM new 2434132cc7 [CEP-21] Add basics of ownership and data placement new 57ad145f19 [CEP-21] Produce placements equivalent to current replication strategies new ee17642571 [CEP-21] Initial transforms for modifying ownership new 5a09d94419 [CEP-21] Start to remove and deprecate gossip functionality new d4d35a7d79 [CEP-21] Replace fake initial implementation of CMS membership new 6fcff5b04e [CEP-21] Major changes to StorageService to remove TMD and Gossip (1/7) new 8425d3177d [CEP-21] Placement and ownership related changes (2/7) new ad7edf5331 [CEP-21] Consistent read/write path (3/7) new da5b8ec9ce [CEP-21] Update ColumnFamilyStore (4/7) new 205331198f [CEP-21] Fix misc compilation errors (5/7) new f528d235c8 [CEP-21] Remove TokenMetadata (6/7) new 9a1546b089 [CEP-21] Restore operation mode reporting (7/7) new 7bda86e1fe [CEP-21] New nodetool commands new bb615495ba [CEP-21] Add missing implementations to concurrent utils new e7d451fd6b [CEP-21] Option to record thread creation stacktrace new 6438a5f17b [CEP-21] Upgrade support new 4892a4e0ea [CEP-21] Update snitches to use Directory new 85748281f4 [CEP-21] Modify CassandraDaemon new e2036cd719 [CEP-21] Test / build config changes new 11e8ded97b [CEP-21] Bulk update of unit and dtests new 2d133ba464 TMP - use bundled version of harry The 25 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 21/25: [CEP-21] Update snitches to use Directory
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 4892a4e0ea39968308771864fc08b5097501243d Author: Sam Tunnicliffe AuthorDate: Fri Mar 3 19:02:57 2023 + [CEP-21] Update snitches to use Directory Minimal changes to IEndpointSnitch implementations to have them pull location info from Directory. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../cassandra/locator/AlibabaCloudSnitch.java | 40 ++ .../apache/cassandra/locator/CloudstackSnitch.java | 40 ++ .../org/apache/cassandra/locator/Ec2Snitch.java| 37 ++-- .../cassandra/locator/GoogleCloudSnitch.java | 39 ++--- .../locator/GossipingPropertyFileSnitch.java | 40 +- 5 files changed, 51 insertions(+), 145 deletions(-) diff --git a/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java b/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java index 729e1b3763..45d7fcd857 100644 --- a/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java +++ b/src/java/org/apache/cassandra/locator/AlibabaCloudSnitch.java @@ -25,13 +25,11 @@ import java.net.MalformedURLException; import java.net.SocketTimeoutException; import java.net.URL; import java.nio.charset.StandardCharsets; -import java.util.Map; -import org.apache.cassandra.db.SystemKeyspace; + import org.apache.cassandra.exceptions.ConfigurationException; -import org.apache.cassandra.gms.ApplicationState; -import org.apache.cassandra.gms.EndpointState; -import org.apache.cassandra.gms.Gossiper; import org.apache.cassandra.io.util.FileUtils; +import org.apache.cassandra.tcm.ClusterMetadata; +import org.apache.cassandra.tcm.membership.NodeId; import org.apache.cassandra.utils.FBUtilities; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -47,9 +45,6 @@ public class AlibabaCloudSnitch extends AbstractNetworkTopologySnitch { protected static final Logger logger = LoggerFactory.getLogger(AlibabaCloudSnitch.class); protected static final String ZONE_NAME_QUERY_URL = "http://100.100.100.200/latest/meta-data/zone-id";; -private static final String DEFAULT_DC = "UNKNOWN-DC"; -private static final String DEFAULT_RACK = "UNKNOWN-RACK"; -private Map> savedEndpoints; protected String ecsZone; protected String ecsRegion; @@ -112,17 +107,10 @@ public class AlibabaCloudSnitch extends AbstractNetworkTopologySnitch { if (endpoint.equals(FBUtilities.getBroadcastAddressAndPort())) return ecsZone; -EndpointState state = Gossiper.instance.getEndpointStateForEndpoint(endpoint); -if (state == null || state.getApplicationState(ApplicationState.RACK) == null) -{ -if (savedEndpoints == null) -savedEndpoints = SystemKeyspace.loadDcRackInfo(); -if (savedEndpoints.containsKey(endpoint)) -return savedEndpoints.get(endpoint).get("rack"); -return DEFAULT_RACK; -} -return state.getApplicationState(ApplicationState.RACK).value; - + +ClusterMetadata metadata = ClusterMetadata.current(); +NodeId nodeId = metadata.directory.peerId(endpoint); +return metadata.directory.location(nodeId).rack; } @Override @@ -130,17 +118,9 @@ public class AlibabaCloudSnitch extends AbstractNetworkTopologySnitch { if (endpoint.equals(FBUtilities.getBroadcastAddressAndPort())) return ecsRegion; -EndpointState state = Gossiper.instance.getEndpointStateForEndpoint(endpoint); -if (state == null || state.getApplicationState(ApplicationState.DC) == null) -{ -if (savedEndpoints == null) -savedEndpoints = SystemKeyspace.loadDcRackInfo(); -if (savedEndpoints.containsKey(endpoint)) -return savedEndpoints.get(endpoint).get("data_center"); -return DEFAULT_DC; -} -return state.getApplicationState(ApplicationState.DC).value; - +ClusterMetadata metadata = ClusterMetadata.current(); +NodeId nodeId = metadata.directory.peerId(endpoint); +return metadata.directory.location(nodeId).datacenter; } } diff --git a/src/java/org/apache/cassandra/locator/CloudstackSnitch.java b/src/java/org/apache/cassandra/locator/CloudstackSnitch.java index d8579534fb..a4535d6374 100644 --- a/src/java/org/apache/cassandra/locator/CloudstackSnitch.java +++ b/src/java/org/apache/cassandra/locator/CloudstackSnitch.java @@ -25,7 +25,6 @@ import java.net.HttpURLConnection; import java.net.URL; import java.net.URI; import java.nio.charset.StandardCharsets; -import java.util.Map; import java.util.regex.Pattern; import java.util.regex.Matcher; @@ -33,12 +
[cassandra] 09/25: [CEP-21] Replace fake initial implementation of CMS membership
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit d4d35a7d790df44fc151c52f405fc10484f84930 Author: Sam Tunnicliffe AuthorDate: Thu Mar 2 14:42:04 2023 + [CEP-21] Replace fake initial implementation of CMS membership WIP commit (i.e. does not compile) replacing initial toy implementation of CMS membership with proper implementation. Membership of the CMS is determined by ownership of keyspaces with the META replication strategy (more precisely, by being a member of the _read_ placements for meta strategy keyspaces, a node is considered a member of the CMS). Also implements more of the "real" [pre]initialization of the CMS, in preparation for supporting upgrading a running cluster from a gossip based system. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../org/apache/cassandra/auth/AuthKeyspace.java| 55 +++--- src/java/org/apache/cassandra/config/Config.java | 2 +- src/java/org/apache/cassandra/gms/Gossiper.java| 6 +- .../org/apache/cassandra/locator/MetaStrategy.java | 4 +- .../schema/SystemDistributedKeyspace.java | 114 +-- .../org/apache/cassandra/tcm/ClusterMetadata.java | 63 ++ src/java/org/apache/cassandra/tcm/Discovery.java | 2 +- .../apache/cassandra/tcm/PaxosBackedProcessor.java | 7 +- .../org/apache/cassandra/tcm/RemoteProcessor.java | 2 +- .../org/apache/cassandra/tcm/Transformation.java | 7 + .../cassandra/tcm/compatibility/GossipHelper.java | 3 - .../org/apache/cassandra/tcm/log/LocalLog.java | 4 +- .../apache/cassandra/tcm/sequences/AddToCMS.java | 217 + .../tcm/sequences/InProgressSequences.java | 1 + ...lize.java => BaseMembershipTransformation.java} | 57 ++ .../tcm/transformations/cms/FinishAddMember.java | 80 .../tcm/transformations/cms/Initialize.java| 58 +- .../tcm/transformations/cms/PreInitialize.java | 29 ++- .../tcm/transformations/cms/RemoveMember.java | 59 ++ .../tcm/transformations/cms/StartAddMember.java| 84 .../apache/cassandra/tracing/TraceKeyspace.java| 51 +++-- 21 files changed, 690 insertions(+), 215 deletions(-) diff --git a/src/java/org/apache/cassandra/auth/AuthKeyspace.java b/src/java/org/apache/cassandra/auth/AuthKeyspace.java index 67fc9c1a06..12760706fc 100644 --- a/src/java/org/apache/cassandra/auth/AuthKeyspace.java +++ b/src/java/org/apache/cassandra/auth/AuthKeyspace.java @@ -37,7 +37,7 @@ public final class AuthKeyspace { } -private static final int DEFAULT_RF = CassandraRelevantProperties.SYSTEM_AUTH_DEFAULT_RF.getInt(); +public static final int DEFAULT_RF = CassandraRelevantProperties.SYSTEM_AUTH_DEFAULT_RF.getInt(); /** * Generation is used as a timestamp for automatic table creation on startup. @@ -57,49 +57,54 @@ public final class AuthKeyspace public static final long SUPERUSER_SETUP_DELAY = Long.getLong("cassandra.superuser_setup_delay_ms", 1); +public static String ROLES_CQL = "CREATE TABLE IF NOT EXISTS %s (" + + "role text," + + "is_superuser boolean," + + "can_login boolean," + + "salted_hash text," + + "member_of set," + + "PRIMARY KEY(role))"; private static final TableMetadata Roles = parse(ROLES, "role definitions", - "CREATE TABLE %s (" - + "role text," - + "is_superuser boolean," - + "can_login boolean," - + "salted_hash text," - + "member_of set," - + "PRIMARY KEY(role))"); + ROLES_CQL); +public static String ROLE_MEMBERS_CQL = "CREATE TABLE IF NOT EXISTS %s (" ++ "role text," ++ "member text," ++ "PRIMARY KEY(role, member))"; private static final TableMetadata RoleMembers = parse(ROLE_MEMBERS, "role memberships lookup table", - "CREATE TABLE %s (" - + "role text," - + "member text," - + "PRIMARY KEY(role, member))"); + ROLE_MEMBERS_CQL); +public static String ROLE_PERMISSIONS_CQL = "CREATE TABLE IF NOT EXISTS %s (" ++ "role text," ++ "resource text," ++ "permissions set," ++ "PRIMARY KEY(role, res
[cassandra] 20/25: [CEP-21] Upgrade support
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 6438a5f17b9d19aa2a0954521f3049a282fb93ff Author: Sam Tunnicliffe AuthorDate: Fri Mar 3 17:40:14 2023 + [CEP-21] Upgrade support Following an upgrade, nodes in an existing cluster will enter a minimal modification mode. In this state, the set of allowed cluster metadata modifications is constrained to include only the addition, removal and replacement of nodes, to allow failed hosts to be replaced during the upgrade. In this mode the CMS has no members and each peer maintains its own ClusterMetadata independently. This metadata is intitialised at startup from system tables and gossip is used to propagate the permitted metadata changes. When the operator is ready, one node is chosen for promotion to the initial CMS, which is done manually via nodetool. At this point, the candidate node will propose itself as the initial CMS and attempt to gain consensus from the rest of the cluster. If successful, it verifies that all peers have an identical view of cluster metadata and initialises the distributed log with a snapshot of that metadata. Once this process is complete all future cluster metadata updates are performed via the CMS using the global log and reverting to the previous method of metadata management is not supported. Further members can and should be added to the CMS via the nodetool command. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../db/commitlog/CommitLogDescriptor.java | 5 +- .../apache/cassandra/db/virtual/PeersTable.java| 187 ++ .../cassandra/db/virtual/SystemViewsKeyspace.java | 1 + .../apache/cassandra/hints/HintsDescriptor.java| 5 +- .../cassandra/locator/InetAddressAndPort.java | 14 +- src/java/org/apache/cassandra/net/InboundSink.java | 28 ++- .../apache/cassandra/net/ResponseVerbHandler.java | 2 + src/java/org/apache/cassandra/net/Verb.java| 11 +- .../cassandra/tcm/ClusterMetadataService.java | 121 +++- src/java/org/apache/cassandra/tcm/Commit.java | 14 +- src/java/org/apache/cassandra/tcm/Discovery.java | 1 + src/java/org/apache/cassandra/tcm/Startup.java | 116 ++- .../tcm/listeners/LegacyStateListener.java | 91 + .../org/apache/cassandra/tcm/log/LocalLog.java | 28 ++- .../tcm/migration/ClusterMetadataHolder.java | 66 +++ .../apache/cassandra/tcm/migration/Election.java | 212 - .../cassandra/tcm/migration/GossipCMSListener.java | 7 +- .../cassandra/tcm/migration/GossipProcessor.java | 41 src/java/org/apache/cassandra/tools/NodeTool.java | 1 + .../apache/cassandra/tools/nodetool/AddToCMS.java | 39 20 files changed, 953 insertions(+), 37 deletions(-) diff --git a/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java b/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java index ed2af1bd0d..41444fecde 100644 --- a/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java +++ b/src/java/org/apache/cassandra/db/commitlog/CommitLogDescriptor.java @@ -62,13 +62,14 @@ public class CommitLogDescriptor // We don't support anything pre-3.0 public static final int VERSION_30 = 6; public static final int VERSION_40 = 7; +public static final int VERSION_50 = 8; /** * Increment this number if there is a changes in the commit log disc layout or MessagingVersion changes. * Note: make sure to handle {@link #getMessagingVersion()} */ @VisibleForTesting -public static final int current_version = VERSION_40; +public static final int current_version = VERSION_50; final int version; public final long id; @@ -222,6 +223,8 @@ public class CommitLogDescriptor return MessagingService.VERSION_30; case VERSION_40: return MessagingService.VERSION_40; +case VERSION_50: +return MessagingService.VERSION_50; default: throw new IllegalStateException("Unknown commitlog version " + version); } diff --git a/src/java/org/apache/cassandra/db/virtual/PeersTable.java b/src/java/org/apache/cassandra/db/virtual/PeersTable.java new file mode 100644 index 00..157bed2f60 --- /dev/null +++ b/src/java/org/apache/cassandra/db/virtual/PeersTable.java @@ -0,0 +1,187 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0
[cassandra] 14/25: [CEP-21] Fix misc compilation errors (5/7)
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 205331198fd1c45e5257a93137e01b22a1af4c4c Author: Sam Tunnicliffe AuthorDate: Thu Mar 2 19:33:16 2023 + [CEP-21] Fix misc compilation errors (5/7) Part 5 of 7 only compilation errors in non-test code are directly related to TokenMetadata Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../org/apache/cassandra/locator/PropertyFileSnitch.java | 12 +--- .../org/apache/cassandra/tcm/ClusterMetadataService.java | 14 -- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java index 3a9b161356..4ceef2e84e 100644 --- a/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java +++ b/src/java/org/apache/cassandra/locator/PropertyFileSnitch.java @@ -31,6 +31,7 @@ import org.slf4j.LoggerFactory; import org.apache.cassandra.exceptions.ConfigurationException; import org.apache.cassandra.service.StorageService; +import org.apache.cassandra.tcm.ClusterMetadataService; import org.apache.cassandra.utils.FBUtilities; import org.apache.cassandra.utils.ResourceWatcher; import org.apache.cassandra.utils.WrappedRunnable; @@ -213,14 +214,9 @@ public class PropertyFileSnitch extends AbstractNetworkTopologySnitch endpointMap = reloadedMap; if (StorageService.instance != null) // null check tolerates circular dependency; see CASSANDRA-4145 { -if (isUpdate) -StorageService.instance.updateTopology(); -else - StorageService.instance.getTokenMetadata().invalidateCachedRings(); +//if (isUpdate) +//StorageService.instance.updateTopology(); } - -if (gossipStarted) -StorageService.instance.gossipSnitchInfo(); } /** @@ -232,6 +228,8 @@ public class PropertyFileSnitch extends AbstractNetworkTopologySnitch */ private static boolean livenessCheck(HashMap reloadedMap, String[] reloadedDefaultDCRack) { +if (ClusterMetadataService.instance() == null) +return false; // If the default has changed we must check all live hosts but hopefully we will find a live // host quickly and interrupt the loop. Otherwise we only check the live hosts that were either // in the old set or in the new set diff --git a/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java b/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java index 37381354aa..4cf0db438a 100644 --- a/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java +++ b/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java @@ -35,7 +35,6 @@ import org.apache.cassandra.net.IVerbHandler; import org.apache.cassandra.net.Message; import org.apache.cassandra.net.NoPayload; import org.apache.cassandra.schema.DistributedSchema; -import org.apache.cassandra.service.StorageService; import org.apache.cassandra.tcm.log.Entry; import org.apache.cassandra.tcm.log.LocalLog; import org.apache.cassandra.tcm.log.Replication; @@ -390,6 +389,17 @@ public class ClusterMetadataService }); } +public void initRecentlySealedPeriodsIndex() +{ +Sealed.initIndexFromSystemTables(); +} + +public boolean isMigrating() +{ +return false; +//return Election.instance.isMigrating(); +} + /** * Switchable implementations that allow us to go between local and remote implementation whenever we need it. * When the node becomes a member of CMS, it switches back to being a regular member of a cluster, and all @@ -469,6 +479,6 @@ public class ClusterMetadataService public enum State { -LOCAL, REMOTE +LOCAL, REMOTE, GOSSIP } } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 22/25: [CEP-21] Modify CassandraDaemon
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 85748281f4a7216581a94f1ca73183921534dca6 Author: Sam Tunnicliffe AuthorDate: Fri Mar 3 19:43:03 2023 + [CEP-21] Modify CassandraDaemon Alter CassandraDaemon intialization to accomodate TCM and replay of the cluster metadata log. This is something of a WIP and there is clearly scope to further clean up this part of the code. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../apache/cassandra/service/CassandraDaemon.java | 79 ++ 1 file changed, 36 insertions(+), 43 deletions(-) diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java b/src/java/org/apache/cassandra/service/CassandraDaemon.java index 4aea5d47aa..38a94a7d7b 100644 --- a/src/java/org/apache/cassandra/service/CassandraDaemon.java +++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java @@ -71,7 +71,8 @@ import org.apache.cassandra.gms.Gossiper; import org.apache.cassandra.io.sstable.SSTableHeaderFix; import org.apache.cassandra.io.util.File; import org.apache.cassandra.io.util.FileUtils; -import org.apache.cassandra.locator.InetAddressAndPort; +import org.apache.cassandra.tcm.ClusterMetadataService; +import org.apache.cassandra.tcm.Startup; import org.apache.cassandra.metrics.CassandraMetricsRegistry; import org.apache.cassandra.metrics.DefaultNameFactory; import org.apache.cassandra.net.StartupClusterConnectivityChecker; @@ -79,11 +80,10 @@ import org.apache.cassandra.schema.Schema; import org.apache.cassandra.schema.SchemaConstants; import org.apache.cassandra.schema.TableMetadata; import org.apache.cassandra.security.ThreadAwareSecurityManager; -import org.apache.cassandra.streaming.StreamManager; -import org.apache.cassandra.service.paxos.PaxosState; import org.apache.cassandra.tcm.ClusterMetadata; import org.apache.cassandra.tcm.InProgressSequence; -import org.apache.cassandra.tcm.Startup; +import org.apache.cassandra.streaming.StreamManager; +import org.apache.cassandra.service.paxos.PaxosState; import org.apache.cassandra.utils.FBUtilities; import org.apache.cassandra.utils.JMXServerUtils; import org.apache.cassandra.utils.JVMStabilityInspector; @@ -96,7 +96,6 @@ import org.apache.cassandra.utils.logging.LoggingSupportFactory; import org.apache.cassandra.utils.logging.VirtualTableAppender; import static java.util.concurrent.TimeUnit.NANOSECONDS; -import static org.apache.cassandra.config.CassandraRelevantProperties.CASSANDRA_FOREGROUND; import static org.apache.cassandra.config.CassandraRelevantProperties.CASSANDRA_JMX_REMOTE_PORT; import static org.apache.cassandra.config.CassandraRelevantProperties.CASSANDRA_PID_FILE; import static org.apache.cassandra.config.CassandraRelevantProperties.COM_SUN_MANAGEMENT_JMXREMOTE_PORT; @@ -113,6 +112,7 @@ import static org.apache.cassandra.config.CassandraRelevantProperties.JAVA_VM_NA public class CassandraDaemon { public static final String MBEAN_NAME = "org.apache.cassandra.db:type=NativeAccess"; +public static boolean SKIP_GC_INSPECTOR = Boolean.getBoolean("cassandra.startup.skip_gc_inspector"); private static final Logger logger; @@ -259,9 +259,7 @@ public class CassandraDaemon NativeLibrary.tryMlockall(); DatabaseDescriptor.createAllDirectories(); - Keyspace.setInitialized(); - CommitLog.instance.start(); try @@ -272,7 +270,6 @@ public class CassandraDaemon { throw new AssertionError("Can't initialize cluster metadata service"); } - QueryProcessor.registerStatementInvalidatingListener(); //TODO disabled b/c this involves checking schema but log replay hasn't run yet so it hasn't been constructed @@ -295,20 +292,9 @@ public class CassandraDaemon SystemKeyspaceMigrator41.migrate(); +// TODO (TM/alexp) // Populate token metadata before flushing, for token-aware sstable partitioning (#6696) -//StorageService.instance.populateTokenMetadata(); - -//try -//{ -//// load schema from disk -//Schema.instance.loadFromDisk(); -//} -//catch (Exception e) -//{ -//logger.error("Error while loading schema: ", e); -//throw e; -//} - +// StorageService.instance.populateTokenMetadata(); setupVirtualKeyspaces(); SSTableHeaderFix.fixNonFrozenUDTIfUpgradeFrom30(); @@ -322,13 +308,12 @@ public class CassandraDaemon exitOrFail(e.returnCode, e.getMessage(), e.getCause()); } -Keyspace.setInitialized(); - // initialize keyspaces for (String keyspaceName : Schema.instance.getKeyspaces()) { if (logger.isDebugEnabled())
[cassandra] 23/25: [CEP-21] Test / build config changes
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e2036cd719b051c0cb9b58709f29028f0a51b0be Author: Sam Tunnicliffe AuthorDate: Mon Mar 6 10:09:36 2023 + [CEP-21] Test / build config changes Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .build/cassandra-build-deps-template.xml | 4 .build/parent-pom-template.xml | 5 + test/conf/logback-dtest.xml | 5 +++-- test/conf/logback-simulator.xml | 1 - 4 files changed, 12 insertions(+), 3 deletions(-) diff --git a/.build/cassandra-build-deps-template.xml b/.build/cassandra-build-deps-template.xml index 93387996ef..a317f861a5 100644 --- a/.build/cassandra-build-deps-template.xml +++ b/.build/cassandra-build-deps-template.xml @@ -62,6 +62,10 @@ com.google.jimfs jimfs + + com.google.guava + guava-testlib + com.puppycrawl.tools checkstyle diff --git a/.build/parent-pom-template.xml b/.build/parent-pom-template.xml index 4edd92e65c..f016c98679 100644 --- a/.build/parent-pom-template.xml +++ b/.build/parent-pom-template.xml @@ -297,6 +297,11 @@ + +com.google.guava +guava-testlib +27.0-jre + com.google.jimfs jimfs diff --git a/test/conf/logback-dtest.xml b/test/conf/logback-dtest.xml index 52eaf335de..48d9859b67 100644 --- a/test/conf/logback-dtest.xml +++ b/test/conf/logback-dtest.xml @@ -27,7 +27,7 @@ ./build/test/logs/${cassandra.testtag}/${suitename}/${cluster_id}/${instance_id}/system.log - %-5level [%thread] ${instance_id} %date{ISO8601} %msg%n + %-5level [%thread] ${instance_id} %date{ISO8601} %F:%L - %msg%n true @@ -50,7 +50,8 @@ - + + diff --git a/test/conf/logback-simulator.xml b/test/conf/logback-simulator.xml index 25c9de8a6a..3a9fabc380 100644 --- a/test/conf/logback-simulator.xml +++ b/test/conf/logback-simulator.xml @@ -40,7 +40,6 @@ - - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 25/25: TMP - use bundled version of harry
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 2d133ba46499d9e4c4a5c0d2ab85df69b5f424a1 Author: Sam Tunnicliffe AuthorDate: Thu Mar 9 19:09:48 2023 + TMP - use bundled version of harry --- .build/build-resolver.xml | 3 +++ .build/parent-pom-template.xml| 5 +++-- .gitignore| 1 + build.xml | 2 +- lib/harry-0.0.2-internal-20221121.14211-2.jar | Bin 0 -> 435204 bytes 5 files changed, 8 insertions(+), 3 deletions(-) diff --git a/.build/build-resolver.xml b/.build/build-resolver.xml index 60122b854f..25c1c86519 100644 --- a/.build/build-resolver.xml +++ b/.build/build-resolver.xml @@ -259,5 +259,8 @@ + + + diff --git a/.build/parent-pom-template.xml b/.build/parent-pom-template.xml index f016c98679..7e0bb51dcf 100644 --- a/.build/parent-pom-template.xml +++ b/.build/parent-pom-template.xml @@ -487,8 +487,9 @@ org.apache.cassandra harry-core -0.0.1 -test +0.0.2-internal-20221121.14211-2 +system + ${test.lib}/harry-0.0.2-internal-20221121.14211-2.jar org.reflections diff --git a/.gitignore b/.gitignore index 916ab0de99..7e32b06b4e 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,7 @@ pylib/src/ **/cqlshlib.xml !lib/cassandra-driver-internal-only-*.zip !lib/puresasl-*.zip +!lib/harry-0.0.2-internal-20221121.14211-2.jar # C* debs build-stamp diff --git a/build.xml b/build.xml index 197f720472..ee1f5892cd 100644 --- a/build.xml +++ b/build.xml @@ -389,7 +389,7 @@ - + diff --git a/lib/harry-0.0.2-internal-20221121.14211-2.jar b/lib/harry-0.0.2-internal-20221121.14211-2.jar new file mode 100644 index 00..44bf1be4a3 Binary files /dev/null and b/lib/harry-0.0.2-internal-20221121.14211-2.jar differ - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 19/25: [CEP-21] Option to record thread creation stacktrace
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e7d451fd6b0516a0566dcb466ceba677d08759be Author: Sam Tunnicliffe AuthorDate: Fri Mar 3 17:52:36 2023 + [CEP-21] Option to record thread creation stacktrace Adds a property for use in tests and debugging which preserves the stacktrace of when a thread is created by NamedThreadFactory. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../cassandra/concurrent/NamedThreadFactory.java | 38 +- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java index 9816649424..1df561e609 100644 --- a/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java +++ b/src/java/org/apache/cassandra/concurrent/NamedThreadFactory.java @@ -17,6 +17,7 @@ */ package org.apache.cassandra.concurrent; +import java.util.Arrays; import java.util.concurrent.ThreadFactory; import java.util.concurrent.atomic.AtomicInteger; @@ -33,6 +34,9 @@ import org.apache.cassandra.utils.JVMStabilityInspector; public class NamedThreadFactory implements ThreadFactory { +public static final String PRESERVE_THREAD_CREATION_STACKTRACE_PROPERTY_NAME = "cassandra.test.preserve_thread_creation_stacktrace"; +public static final Boolean PRESERVE_THREAD_CREATION_STACKTRACE = Boolean.getBoolean(PRESERVE_THREAD_CREATION_STACKTRACE_PROPERTY_NAME); + private static final AtomicInteger anonymousCounter = new AtomicInteger(); private static volatile String globalPrefix; @@ -159,11 +163,43 @@ public class NamedThreadFactory implements ThreadFactory public static Thread createThread(ThreadGroup threadGroup, Runnable runnable, String name, boolean daemon) { String prefix = globalPrefix; -Thread thread = new FastThreadLocalThread(threadGroup, runnable, prefix != null ? prefix + name : name); +Thread thread; +String threadName = prefix != null ? prefix + name : name; +if (PRESERVE_THREAD_CREATION_STACKTRACE) +thread = new InspectableFastThreadLocalThread(threadGroup, runnable, threadName); +else +thread = new FastThreadLocalThread(threadGroup, runnable, threadName); thread.setDaemon(daemon); return thread; } +public static class InspectableFastThreadLocalThread extends FastThreadLocalThread +{ +public StackTraceElement[] creationTrace; + +private void setStack() +{ +creationTrace = Thread.currentThread().getStackTrace(); +creationTrace = Arrays.copyOfRange(creationTrace, 2, creationTrace.length); +} + +public InspectableFastThreadLocalThread() { super(); setStack(); } + +public InspectableFastThreadLocalThread(Runnable target) { super(target); setStack(); } + +public InspectableFastThreadLocalThread(ThreadGroup group, Runnable target) { super(group, target); setStack(); } + +public InspectableFastThreadLocalThread(String name) { super(name); setStack(); } + +public InspectableFastThreadLocalThread(ThreadGroup group, String name) { super(group, name); setStack(); } + +public InspectableFastThreadLocalThread(Runnable target, String name) { super(target, name); setStack(); } + +public InspectableFastThreadLocalThread(ThreadGroup group, Runnable target, String name) { super(group, target, name); setStack(); } + +public InspectableFastThreadLocalThread(ThreadGroup group, Runnable target, String name, long stackSize) { super(group, target, name, stackSize); setStack(); } + +} public static T setupThread(T thread, int priority, ClassLoader contextClassLoader, Thread.UncaughtExceptionHandler uncaughtExceptionHandler) { thread.setPriority(priority); - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 04/25: [CEP-21] Add rudimentary cluster membership to TCM
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 35910f2425e61d275eed511691a96c208e686fbf Author: Sam Tunnicliffe AuthorDate: Wed Mar 1 10:53:51 2023 + [CEP-21] Add rudimentary cluster membership to TCM Adds a new Directory component to ClusterMetadata to manage member identity, state location and addressing. This duplicates some of the functions of TokenMetadata, Topology et al but with updates performed consistently via the global log. Although it isn't actually used for anything yet it is a prerequisite for managing data ownership through TCM, which will eventually replace TokenMetadata completely. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../apache/cassandra/service/StorageService.java | 3 + .../org/apache/cassandra/tcm/ClusterMetadata.java | 68 ++- .../org/apache/cassandra/tcm/MetadataKeys.java | 1 + .../org/apache/cassandra/tcm/Transformation.java | 2 + .../cassandra/tcm/compatibility/GossipHelper.java | 2 + .../apache/cassandra/tcm/membership/Directory.java | 578 + .../apache/cassandra/tcm/membership/Location.java | 83 +++ .../cassandra/tcm/membership/NodeAddresses.java| 125 + .../apache/cassandra/tcm/membership/NodeId.java| 89 .../NodeState.java}| 35 +- .../cassandra/tcm/membership/NodeVersion.java | 118 + .../cassandra/tcm/transformations/Register.java| 208 .../apache/cassandra/utils/CassandraVersion.java | 1 + .../apache/cassandra/utils/btree/BTreeBiMap.java | 102 .../cassandra/utils/btree/BTreeMultimap.java | 214 .../org/apache/cassandra/utils/btree/BTreeSet.java | 45 +- 16 files changed, 1639 insertions(+), 35 deletions(-) diff --git a/src/java/org/apache/cassandra/service/StorageService.java b/src/java/org/apache/cassandra/service/StorageService.java index f9362a018d..48c6168742 100644 --- a/src/java/org/apache/cassandra/service/StorageService.java +++ b/src/java/org/apache/cassandra/service/StorageService.java @@ -105,6 +105,8 @@ import org.apache.cassandra.service.disk.usage.DiskUsageBroadcaster; import org.apache.cassandra.tcm.ClusterMetadata; import org.apache.cassandra.tcm.ClusterMetadataService; import org.apache.cassandra.tcm.Startup; +import org.apache.cassandra.tcm.membership.NodeId; +import org.apache.cassandra.tcm.transformations.Register; import org.apache.cassandra.utils.concurrent.Future; import org.apache.cassandra.schema.TableId; import org.apache.cassandra.utils.concurrent.FutureCombiner; @@ -1015,6 +1017,7 @@ public class StorageService extends NotificationBroadcasterSupport implements IE throw new RuntimeException(e); } +NodeId self = Register.maybeRegister(); completeInitialization(); } diff --git a/src/java/org/apache/cassandra/tcm/ClusterMetadata.java b/src/java/org/apache/cassandra/tcm/ClusterMetadata.java index 4d9f5a28cf..90d5343d2c 100644 --- a/src/java/org/apache/cassandra/tcm/ClusterMetadata.java +++ b/src/java/org/apache/cassandra/tcm/ClusterMetadata.java @@ -38,6 +38,12 @@ import org.apache.cassandra.locator.InetAddressAndPort; import org.apache.cassandra.schema.DistributedSchema; import org.apache.cassandra.tcm.extensions.ExtensionKey; import org.apache.cassandra.tcm.extensions.ExtensionValue; +import org.apache.cassandra.tcm.membership.Directory; +import org.apache.cassandra.tcm.membership.Location; +import org.apache.cassandra.tcm.membership.NodeAddresses; +import org.apache.cassandra.tcm.membership.NodeId; +import org.apache.cassandra.tcm.membership.NodeState; +import org.apache.cassandra.tcm.membership.NodeVersion; import org.apache.cassandra.tcm.serialization.MetadataSerializer; import org.apache.cassandra.tcm.serialization.Version; import org.apache.cassandra.tcm.transformations.cms.EntireRange; @@ -58,16 +64,28 @@ public class ClusterMetadata public final ImmutableMap, ExtensionValue> extensions; public final DistributedSchema schema; +public final Directory directory; public final EndpointsForRange cmsReplicas; public final ImmutableSet cmsMembers; public ClusterMetadata(IPartitioner partitioner) +{ +this(partitioner, Directory.EMPTY); +} + +private ClusterMetadata(IPartitioner partitioner, Directory directory) +{ +this(partitioner, directory, DistributedSchema.first()); +} + +private ClusterMetadata(IPartitioner partitioner, Directory directory, DistributedSchema schema) { this(Epoch.EMPTY, Period.EMPTY, true, partitioner, - DistributedSchema.first(), + schema, + directory, ImmutableSet.of(), Immutable
[cassandra] 06/25: [CEP-21] Produce placements equivalent to current replication strategies
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 57ad145f1946961a4da506a9b4c35fa95725916d Author: Sam Tunnicliffe AuthorDate: Wed Mar 1 12:53:19 2023 + [CEP-21] Produce placements equivalent to current replication strategies Minimal modifications to AbstractReplicationStrategy implementations to support the production of DataPlacements using ClusterMetadata while retaining calculateNaturalReplicas. Also adds tests to compare the output of both methods and assert their equivalence. Eventually, the original implementations based on TokenMetadata will be retired and will be retained in the test source to guard against regressions. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../locator/AbstractReplicaCollection.java | 40 ++- .../locator/AbstractReplicationStrategy.java | 10 +- .../apache/cassandra/locator/LocalStrategy.java| 50 +-- .../org/apache/cassandra/locator/MetaStrategy.java | 23 +- .../cassandra/locator/NetworkTopologyStrategy.java | 70 ++-- .../apache/cassandra/locator/SimpleStrategy.java | 41 ++- .../apache/cassandra/locator/TokenMetadata.java| 159 + .../apache/cassandra/service/StorageService.java | 11 +- .../service/reads/range/ReplicaPlanIterator.java | 3 +- .../org/apache/cassandra/tcm/ClusterMetadata.java | 54 ++- .../org/apache/cassandra/tcm/MetadataKeys.java | 5 +- .../AsEndpoints.java} | 21 +- .../AsLocations.java} | 27 +- .../AsTokenMap.java} | 24 +- .../cassandra/tcm/compatibility/GossipHelper.java | 2 + .../tcm/compatibility/TokenRingUtils.java | 150 .../apache/cassandra/tcm/membership/Directory.java | 4 +- .../tcm/ownership/PrimaryRangeComparator.java | 51 +++ .../apache/cassandra/tcm/ownership/TokenMap.java | 302 .../tcm/ownership/UniformRangePlacement.java | 324 + .../tcm/transformations/cms/EntireRange.java | 8 +- .../org/apache/cassandra/utils/BiMultiValMap.java | 16 + .../locator/NetworkTopologyStrategyTest.java | 3 +- .../cassandra/locator/TokenMetadataTest.java | 3 +- .../cassandra/service/LeaveAndBootstrapTest.java | 3 +- .../org/apache/cassandra/service/MoveTest.java | 3 +- .../cassandra/tcm/membership/MembershipUtils.java | 10 + .../cassandra/tcm/ownership/OwnershipUtils.java| 139 .../tcm/ownership/PlacementDeltasTest.java | 201 +++ .../UniformRangePlacementEquivalenceTest.java | 388 + .../tcm/ownership/UniformRangePlacementTest.java | 290 +++ 31 files changed, 2233 insertions(+), 202 deletions(-) diff --git a/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java b/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java index cc210622d1..fa8d17facd 100644 --- a/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java +++ b/src/java/org/apache/cassandra/locator/AbstractReplicaCollection.java @@ -22,6 +22,7 @@ import com.carrotsearch.hppc.ObjectIntHashMap; import com.google.common.annotations.VisibleForTesting; import com.google.common.base.Preconditions; import com.google.common.collect.Iterables; +import com.google.common.collect.Iterators; import java.util.AbstractList; import java.util.AbstractMap; @@ -100,6 +101,18 @@ public abstract class AbstractReplicaCollection map) +{ +Replica[] contents = new Replica[size]; +for (int i = 0; i < contents.length; i++) +{ +if (this.contents[i] != null) +contents[i] = map.apply(this.contents[i]); +} + +return new ReplicaList(contents, begin, contents.length); +} + public void add(Replica replica) { // can only add to full array - if we have sliced it, we must be a snapshot @@ -591,33 +604,30 @@ public abstract class AbstractReplicaCollection - * It's not clear whether {@link AbstractReplicaCollection} should implement the order sensitive {@link Object#equals(Object) equals} - * of {@link java.util.List} or the order oblivious {@link Object#equals(Object) equals} of {@link java.util.Set}. We never rely on equality - * in the database so rather then leave in a potentially surprising implementation we have it throw {@link UnsupportedOperationException}. - * - * - * Don't implement this and pick one behavior over the other. If you want equality you can static import {@link com.google.common.collect.Iterables#elementsEqual(Iterable, Iterable)} - * and use that to get order sensitive equals. + * Implements order sensitive
[cassandra] 12/25: [CEP-21] Consistent read/write path (3/7)
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit ad7edf53314a89cbec94adf13cdc679cc4006c85 Author: Sam Tunnicliffe AuthorDate: Thu Mar 2 19:07:12 2023 + [CEP-21] Consistent read/write path (3/7) Part 3 of 7 adds the ability to detect version mismatches between peers on the read/write path and to handle such divergence. Lagging peers will attempt to catch up from the CMS if the coordinator in a r/w operation has seen newer metadata. Coordinators may fail writes if the cluster metadata changes while the write is in flight, if the consistency level can no longer be satisfied by the original replica plan. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- src/java/org/apache/cassandra/db/ReadCommand.java | 9 + .../cassandra/db/ReadCommandVerbHandler.java | 80 +++-- src/java/org/apache/cassandra/db/ReadResponse.java | 7 + .../cassandra/db/partitions/PartitionUpdate.java | 18 ++ .../org/apache/cassandra/locator/ReplicaPlan.java | 271 ++-- .../org/apache/cassandra/locator/ReplicaPlans.java | 350 - .../apache/cassandra/net/ResponseVerbHandler.java | 14 + .../service/AbstractWriteResponseHandler.java | 4 + .../cassandra/service/WriteResponseHandler.java| 12 + .../org/apache/cassandra/service/paxos/Paxos.java | 34 +- .../cassandra/service/reads/ReadCallback.java | 35 +-- .../service/reads/ReplicaFilteringProtection.java | 3 +- .../cassandra/service/reads/ResponseResolver.java | 1 - .../service/reads/range/ReplicaPlanIterator.java | 7 +- .../reads/repair/BlockingPartitionRepair.java | 54 ++-- .../service/reads/repair/BlockingReadRepair.java | 9 +- .../service/reads/repair/NoopReadRepair.java | 2 +- .../service/reads/repair/ReadOnlyReadRepair.java | 2 +- .../cassandra/service/reads/repair/ReadRepair.java | 2 +- .../reads/repair/RowIteratorMergeListener.java | 20 +- 20 files changed, 744 insertions(+), 190 deletions(-) diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java b/src/java/org/apache/cassandra/db/ReadCommand.java index 0978b8ef05..de218d9b67 100644 --- a/src/java/org/apache/cassandra/db/ReadCommand.java +++ b/src/java/org/apache/cassandra/db/ReadCommand.java @@ -38,6 +38,7 @@ import org.apache.cassandra.config.*; import org.apache.cassandra.db.filter.*; import org.apache.cassandra.exceptions.QueryCancelledException; import org.apache.cassandra.net.MessageFlag; +import org.apache.cassandra.net.MessagingService; import org.apache.cassandra.net.ParamType; import org.apache.cassandra.net.Verb; import org.apache.cassandra.db.partitions.*; @@ -66,6 +67,7 @@ import org.apache.cassandra.schema.TableMetadata; import org.apache.cassandra.schema.SchemaProvider; import org.apache.cassandra.service.ActiveRepairService; import org.apache.cassandra.service.ClientWarn; +import org.apache.cassandra.tcm.Epoch; import org.apache.cassandra.tracing.Tracing; import org.apache.cassandra.utils.FBUtilities; import org.apache.cassandra.utils.ObjectSizes; @@ -1070,6 +1072,8 @@ public abstract class ReadCommand extends AbstractReadQuery if (command.isDigestQuery()) out.writeUnsignedVInt32(command.digestVersion()); command.metadata().id.serialize(out); +if (version >= MessagingService.VERSION_50) +Epoch.serializer.serialize(command.metadata().epoch, out); out.writeInt(command.nowInSec()); ColumnFilter.serializer.serialize(command.columnFilter(), out, version); RowFilter.serializer.serialize(command.rowFilter(), out, version); @@ -1097,6 +1101,10 @@ public abstract class ReadCommand extends AbstractReadQuery boolean hasIndex = hasIndex(flags); int digestVersion = isDigest ? in.readUnsignedVInt32() : 0; TableMetadata metadata = schema.getExistingTableMetadata(TableId.deserialize(in)); +Epoch schemaVersion = null; +if (version >= MessagingService.VERSION_50) +schemaVersion = Epoch.serializer.deserialize(in); +assert schemaVersion == null || metadata.epoch.equals(schemaVersion) : metadata.epoch + " " + schemaVersion; // TODO: handle etc int nowInSec = in.readInt(); ColumnFilter columnFilter = ColumnFilter.serializer.deserialize(in, version, metadata); RowFilter rowFilter = RowFilter.serializer.deserialize(in, version, metadata); @@ -1128,6 +1136,7 @@ public abstract class ReadCommand extends AbstractReadQuery return 2 // kind + flags + (command.isDigestQuery() ? TypeSizes.sizeofUnsignedVInt(command.digestVersion()) : 0) + command.metadata().id.serializedSize() +
[cassandra] 16/25: [CEP-21] Restore operation mode reporting (7/7)
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 9a1546b089b21d724242ec88b6362607c57338db Author: Sam Tunnicliffe AuthorDate: Fri Mar 3 10:03:41 2023 + [CEP-21] Restore operation mode reporting (7/7) Part 7 of 7 brings StorageService.operationMode back into sync with previous behaviour. Many external coordination tools depend on accessing this state via JMX, so this is an important external interface. This commit also adds a virtual version of the system.local table, as we can fully construct the data for this from ClusterMetadata, meaning we no longer the on-disk system table, though this is retained for now. In future, more system tables can be virtualised (system.peers, system_schema, etc). Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../org/apache/cassandra/db/SystemKeyspace.java| 32 - .../apache/cassandra/db/virtual/LocalTable.java| 145 + .../cassandra/db/virtual/SystemViewsKeyspace.java | 1 + 3 files changed, 173 insertions(+), 5 deletions(-) diff --git a/src/java/org/apache/cassandra/db/SystemKeyspace.java b/src/java/org/apache/cassandra/db/SystemKeyspace.java index 4879260406..947aafbd07 100644 --- a/src/java/org/apache/cassandra/db/SystemKeyspace.java +++ b/src/java/org/apache/cassandra/db/SystemKeyspace.java @@ -97,7 +97,6 @@ import org.apache.cassandra.schema.TableMetadata; import org.apache.cassandra.schema.Tables; import org.apache.cassandra.schema.Types; import org.apache.cassandra.schema.Views; -import org.apache.cassandra.service.StorageService; import org.apache.cassandra.service.paxos.Ballot; import org.apache.cassandra.service.paxos.Commit; import org.apache.cassandra.service.paxos.Commit.Accepted; @@ -108,8 +107,10 @@ import org.apache.cassandra.service.paxos.PaxosState; import org.apache.cassandra.service.paxos.uncommitted.PaxosRows; import org.apache.cassandra.service.paxos.uncommitted.PaxosUncommittedIndex; import org.apache.cassandra.streaming.StreamOperation; +import org.apache.cassandra.tcm.ClusterMetadata; import org.apache.cassandra.tcm.Epoch; import org.apache.cassandra.tcm.Sealed; +import org.apache.cassandra.tcm.membership.NodeState; import org.apache.cassandra.transport.ProtocolVersion; import org.apache.cassandra.utils.ByteBufferUtil; import org.apache.cassandra.utils.CassandraVersion; @@ -598,7 +599,27 @@ public final class SystemKeyspace NEEDS_BOOTSTRAP, COMPLETED, IN_PROGRESS, -DECOMMISSIONED +DECOMMISSIONED; + +public static BootstrapState fromNodeState(NodeState nodeState) +{ +if (nodeState == null) // todo, handle this properly +return DECOMMISSIONED; +switch (nodeState) +{ +case REGISTERED: +return NEEDS_BOOTSTRAP; +case BOOTSTRAPPING: +return IN_PROGRESS; +case JOINED: +case LEAVING: +case MOVING: +return COMPLETED; +case LEFT: +default: +return DECOMMISSIONED; +} +} } public static void persistLocalMetadata() @@ -912,11 +933,11 @@ public final class SystemKeyspace executeInternal(format(req, LOCAL, LOCAL), version); } -private static Set tokensAsSet(Collection tokens) +public static Set tokensAsSet(Collection tokens) { if (tokens.isEmpty()) return Collections.emptySet(); -Token.TokenFactory factory = StorageService.instance.getTokenFactory(); +Token.TokenFactory factory = ClusterMetadata.current().partitioner.getTokenFactory(); Set s = new HashSet<>(tokens.size()); for (Token tk : tokens) s.add(factory.toString(tk)); @@ -925,7 +946,7 @@ public final class SystemKeyspace private static Collection deserializeTokens(Collection tokensStrings) { -Token.TokenFactory factory = StorageService.instance.getTokenFactory(); +Token.TokenFactory factory = ClusterMetadata.current().partitioner.getTokenFactory(); List tokens = new ArrayList<>(tokensStrings.size()); for (String tk : tokensStrings) tokens.add(factory.fromString(tk)); @@ -1261,6 +1282,7 @@ public final class SystemKeyspace * Read the host ID from the system keyspace, creating (and storing) one if * none exists. */ +// TODO: this method should not exist. Only CMS can give out host ids. public static synchronized UUID getOrInitializeLocalHostId() { UUID hostId = getLocalHostId(); diff --git a/src/java/org/apache/cassandra/db/virtual/LocalTable.java b/src/java/org/apache/cassandra/db/v
[cassandra] 15/25: [CEP-21] Remove TokenMetadata (6/7)
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git commit f528d235c89ab2a5e19b227b01d70fe11b2fbfcc Author: Sam Tunnicliffe AuthorDate: Thu Mar 2 19:47:53 2023 + [CEP-21] Remove TokenMetadata (6/7) Part 6 of 7 Completely remove TokenMetadata, the intention is to bring it back in a stripped down form, available to tests only, so we can continue to verify equivalence between old and new code. Test code is still extremely broken at this point, but non-test code is buildable again, though almost certainly not actually runnable. Co-authored-by: Marcus Eriksson Co-authored-by: Alex Petrov Co-authored-by: Sam Tunnicliffe --- .../apache/cassandra/locator/TokenMetadata.java| 1626 .../locator/TokenMetadataDiagnostics.java | 46 - .../cassandra/locator/TokenMetadataEvent.java | 62 - .../apache/cassandra/service/StorageService.java |1 - .../apache/cassandra/stress/CompactionStress.java | 18 +- 5 files changed, 11 insertions(+), 1742 deletions(-) diff --git a/src/java/org/apache/cassandra/locator/TokenMetadata.java b/src/java/org/apache/cassandra/locator/TokenMetadata.java deleted file mode 100644 index abbc76d482..00 --- a/src/java/org/apache/cassandra/locator/TokenMetadata.java +++ /dev/null @@ -1,1626 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one - * or more contributor license agreements. See the NOTICE file - * distributed with this work for additional information - * regarding copyright ownership. The ASF licenses this file - * to you under the Apache License, Version 2.0 (the - * "License"); you may not use this file except in compliance - * with the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package org.apache.cassandra.locator; - -import java.nio.ByteBuffer; -import java.util.*; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; -import java.util.concurrent.atomic.AtomicReference; -import java.util.concurrent.locks.ReadWriteLock; -import java.util.concurrent.locks.ReentrantReadWriteLock; -import java.util.function.Supplier; -import java.util.stream.Collectors; - -import javax.annotation.concurrent.GuardedBy; - -import com.google.common.annotations.VisibleForTesting; -import com.google.common.collect.*; - -import org.apache.commons.lang3.StringUtils; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import org.apache.cassandra.config.DatabaseDescriptor; -import org.apache.cassandra.db.DecoratedKey; -import org.apache.cassandra.db.Keyspace; -import org.apache.cassandra.dht.IPartitioner; -import org.apache.cassandra.dht.Range; -import org.apache.cassandra.dht.Token; -import org.apache.cassandra.gms.FailureDetector; -import org.apache.cassandra.locator.ReplicaCollection.Builder.Conflict; -import org.apache.cassandra.service.StorageService; -import org.apache.cassandra.tcm.compatibility.AsEndpoints; -import org.apache.cassandra.tcm.compatibility.AsLocations; -import org.apache.cassandra.tcm.compatibility.AsTokenMap; -import org.apache.cassandra.tcm.compatibility.TokenRingUtils; -import org.apache.cassandra.tcm.membership.Location; -import org.apache.cassandra.tcm.membership.NodeId; -import org.apache.cassandra.utils.BiMultiValMap; -import org.apache.cassandra.utils.Pair; -import org.apache.cassandra.utils.SortedBiMultiValMap; - -import static org.apache.cassandra.config.CassandraRelevantProperties.LINE_SEPARATOR; -import static org.apache.cassandra.utils.Clock.Global.currentTimeMillis; - -public class TokenMetadata implements AsEndpoints, AsLocations, AsTokenMap -{ -private static final Logger logger = LoggerFactory.getLogger(TokenMetadata.class); - -/** - * Maintains token to endpoint map of every node in the cluster. - * Each Token is associated with exactly one Address, but each Address may have - * multiple tokens. Hence, the BiMultiValMap collection. - */ -private final BiMultiValMap tokenToEndpointMap; - -/** Maintains endpoint to host ID map of every node in the cluster */ -private final BiMap endpointToHostIdMap; - -// Prior to CASSANDRA-603, we just had Map pendingRanges, -// which was added to when a node began bootstrap and removed from when it finished. -// -// This is inadequate when multiple changes are allowed simultaneously. For example, -// suppose that there is a ring of nodes A, C and E, with replication factor 3.
[jira] [Updated] (CASSANDRA-18330) Delivery of CEP-21: Transactional Cluster Metadata
[ https://issues.apache.org/jira/browse/CASSANDRA-18330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-18330: Complexity: Challenging (was: Byzantine) > Delivery of CEP-21: Transactional Cluster Metadata > -- > > Key: CASSANDRA-18330 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18330 > Project: Cassandra > Issue Type: Epic > Components: Cluster/Membership, Cluster/Schema >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[ https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700311#comment-17700311 ] David Capwell commented on CASSANDRA-18302: --- bq. Therefore, using asCQL may confuse the users I guess my main question is how much of a concern is this? Right now when users violate a data pattern (too large read, tombstone, etc.) we call org.apache.cassandra.db.AbstractReadQuery#toCQLString which is yet another way to go from internal type to CQL, org.apache.cassandra.cql3.statements.SelectStatement#asCQL was only used before for CoordinatorReadSizeWarnThresholdBytes and CoordinatorReadSizeAbortThresholdBytes, and in those cases we log the CQL only and don't send back to the user (by design, we only show the partition token); when I did this it was more to inform the operators of what is going on so they are more informed when talking with users, so the "exact" string was never important. So, that then gets to the current error messages for txn only, so if we look at a few examples {code} cqlsh> BEGIN TRANSACTION SELECT * FROM system.peers; COMMIT TRANSACTION; InvalidRequest: Error from server: code=2200 [Invalid query] message="Normal SELECT must specify either all primary key elements or all partition key elements and LIMIT 1. In both cases partition key elements must be always specified with equality operators; CQL SELECT * FROM system.peers" cqlsh> BEGIN TRANSACTION SELECT tokens FROM system.peers; COMMIT TRANSACTION; InvalidRequest: Error from server: code=2200 [Invalid query] message="Normal SELECT must specify either all primary key elements or all partition key elements and LIMIT 1. In both cases partition key elements must be always specified with equality operators; CQL SELECT tokens FROM system.peers" cqlsh> BEGIN TRANSACTION SELECT * FROM system.peers WHERE tokens contains ('not exists'); COMMIT TRANSACTION; InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot execute this query as it might involve data filtering and thus may have unpredictable performance. If you want to execute this query despite the performance unpredictability, use ALLOW FILTERING" -- this shows a bug with quoting cqlsh> BEGIN TRANSACTION SELECT * FROM system.peers WHERE tokens contains ('not exists') ALLOW FILTERING; COMMIT TRANSACTION; InvalidRequest: Error from server: code=2200 [Invalid query] message="Normal SELECT must specify either all primary key elements or all partition key elements and LIMIT 1. In both cases partition key elements must be always specified with equality operators; CQL SELECT * FROM system.peers WHERE tokens CONTAINS not exists" {code} in these examples there are some that are not 100% correct but give the main thing users need. I do feel adding line/column to the error would help with transactions, but the "correct" string is less of a concern I have. This gets us to trade offs. We limit the txn string to 4k and in this case we fail to log the CQL to users for large enough transactions, so for larger transactions we are failing to show the CQL.We are also forced to hold this string in the case it is prepared, but we already have the non-truncated string (as you point out), so this causes us to double this memory which is also defined by users, so not something we really can control and limit well (if you have 1k prepared statements that are all truncated you have ~200mb (1k txn * 4096 bytes * 48 bytes for string object) of memory just for this case which we hope to be rare. > Fix AIOOBE and improve validation messages for transaction statements > - > > Key: CASSANDRA-18302 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18302 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: Jacek Lewandowski >Assignee: Jacek Lewandowski >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Currently it happens sometimes that ArrayIndexOutOfBoundsException is thrown > from asCql method when validation transaction statement. In addition, asCql > does not return precisely the query user entered so the whole error message > can be misleading. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700318#comment-17700318 ] David Capwell commented on CASSANDRA-18320: --- bq. Actually, we do not need to wait for this to appear in 4.1.1 because this is just test-related. [~smiklosovic] this is only an issue in Simulator as it overrides the default FileSystem. If you run embedded Cassandra and leverage this logic, then there are edge cases for you, else you would never see this. bq. I can unsurprisingly no longer reproduce Yay! I ran _repeat jobs around 5 times and only ever saw OOM (not the "I can not make progress" OOM, but a real one (likely direct memory given the stack trace)). As far as I can tell, the loading logic fails due to this bug which causes weird cascading issues; those issues are what was reported. > Incompatible file system thrown while running Simulator > --- > > Key: CASSANDRA-18320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18320 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: David Capwell >Priority: Normal > Fix For: 4.1.x, 5.x > > > {code} > java.io.UncheckedIOException > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831) > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047) > at > org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816) > at > org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370) > at > org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345) > at > org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148) > at > org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33) > Caused by: java.nio.file.DirectoryNotEmptyException: > /cassandra/node1/commitlog > at > com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535) > at > com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517) > at > com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479) > at > com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465) > at > com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261) > at java.nio.file.Files.delete(Files.java:1126) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252) > {code} -- This message was sent by Atlassian Jira (
[jira] [Updated] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-18320: -- Reviewers: Brandon Williams, Caleb Rackliffe (was: Caleb Rackliffe) > Incompatible file system thrown while running Simulator > --- > > Key: CASSANDRA-18320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18320 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: David Capwell >Priority: Normal > Fix For: 4.1.x, 5.x > > > {code} > java.io.UncheckedIOException > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831) > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047) > at > org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816) > at > org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370) > at > org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345) > at > org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148) > at > org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33) > Caused by: java.nio.file.DirectoryNotEmptyException: > /cassandra/node1/commitlog > at > com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535) > at > com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517) > at > com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479) > at > com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465) > at > com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261) > at java.nio.file.Files.delete(Files.java:1126) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700324#comment-17700324 ] David Capwell commented on CASSANDRA-18320: --- Starting commit CI Results (pending): ||Branch||Source||Circle CI||Jenkins|| |cassandra-4.1|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18320-cassandra-4.1-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18320-cassandra-4.1-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2350/]| |trunk|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18320-trunk-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18320-trunk-FA990C99-FC38-4C68-91A6-96C4A8303CE3]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2351/]| > Incompatible file system thrown while running Simulator > --- > > Key: CASSANDRA-18320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18320 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: David Capwell >Priority: Normal > Fix For: 4.1.x, 5.x > > > {code} > java.io.UncheckedIOException > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831) > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047) > at > org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816) > at > org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370) > at > org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345) > at > org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148) > at > org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33) > Caused by: java.nio.file.DirectoryNotEmptyException: > /cassandra/node1/commitlog > at > com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535) > at > com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517) > at > com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479) > at > com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465) > at > com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261) > at java.
[jira] [Commented] (CASSANDRA-18302) Fix AIOOBE and improve validation messages for transaction statements
[ https://issues.apache.org/jira/browse/CASSANDRA-18302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700330#comment-17700330 ] David Capwell commented on CASSANDRA-18302: --- With this patch the following query gets a non-usable error {code} -- This Transaction is over peers table, below is the schema to help understand the logic -- CREATE TABLE system.peers_v2 ( -- peer inet, -- peer_port int, -- data_center text, -- host_id uuid, -- native_address inet, -- native_port int, -- preferred_ip inet, -- preferred_port int, -- rack text, -- release_version text, -- schema_version uuid, -- tokens set, -- PRIMARY KEY (peer, peer_port) -- ) WITH CLUSTERING ORDER BY (peer_port ASC) BEGIN TRANSACTION -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET a = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET b = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET c = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET d = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET e = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET f = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET g = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET h = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET i = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); -- load the local host peers -- there is an assumption that a unique inet does not have multiple ports, so LIMIT 1 is safe -- this assumption may be a bad assumption... but you know what they say right? -- when you assume you... -- Makes a ass... -- Out of ...u... -- And ..me... LET j = ( SELECT * FROM system.peers_v2 WHERE peer = '127.0.0.1' LIMIT 1 ); SELECT * FROM system.peers WHERE tokens contains ('not exists') ALLOW FILTERING; COMMIT TRANSACTION; {code} and the error {code} com.datastax.driver.core.exceptions.InvalidQueryException: Range queries are not allowed for reads within a transaction; returning select at 98:3 - {code}
[jira] [Commented] (CASSANDRA-18319) Cassandra in Kubernetes: IP switch decommission issue
[ https://issues.apache.org/jira/browse/CASSANDRA-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700334#comment-17700334 ] Ines Potier commented on CASSANDRA-18319: - We checked and we typically sync with AWS-provided local atomic clocks, so there should not be any skew > Cassandra in Kubernetes: IP switch decommission issue > - > > Key: CASSANDRA-18319 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18319 > Project: Cassandra > Issue Type: Bug >Reporter: Ines Potier >Priority: Normal > > We have recently encountered a recurring old IP reappearance issue while > testing decommissions on some of our Kubernetes Cassandra staging clusters. > *Issue Description* > In Kubernetes, a Cassandra node can change IP at each pod bounce. We have > noticed that this behavior, associated with a decommission operation, can get > the cluster into an erroneous state. > Consider the following situation: a Cassandra node {{node1}} , with > {{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP > ({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all > other nodes’ nodetool status output includes a {{new_IP}} UN entry owning > 20.5% of the token ring and no {{old_IP}} entry. > Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not > have a lot of data, and the decommission operation completes pretty quickly. > Logs on other nodes start showing acknowledgment that {{node1}} has left and > soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is > deleted. > After a minute delay, the cluster enters the erroneous state. An {{old_IP}} > DN entry reappears in nodetool status, owning 20.5% of the token ring. No > node owns this IP anymore and according to logs, {{old_IP}} is still > associated with {{{}hostId1{}}}. > *Issue Root Cause* > By digging through Cassandra logs, and re-testing this scenario over and over > again, we have reached the following conclusion: > * Other nodes will continue exchanging gossip about {{old_IP}} , even after > it becomes a fatClient. > * The fatClient timeout and subsequent quarantine does not stop {{old_IP}} > from reappearing in a node’s Gossip state, once its quarantine is over. We > believe that this is due to a misalignment on all nodes’ {{old_IP}} > expiration time. > * Once {{new_IP}} has left the cluster, and {{old_IP}} next gossip state > message is received by a node, StorageService will no longer face collisions > (or will, but with an even older IP) for {{hostId1}} and its corresponding > tokens. As a result, {{old_IP}} will regain ownership of 20.5% of the token > ring. > *Proposed fix* > Following the above investigation, we were thinking about implementing the > following fix: > When a node receives a gossip status change with {{STATE_LEFT}} for a leaving > endpoint {{{}new_IP{}}}, before evicting {{new_IP }}from the token ring, > purge from Gossip (ie {{{}evictFromMembership{}}}) all endpoints that meet > the following criteria: > * {{endpointStateMap}} contains this endpoint > * The endpoint is not currently a token owner > ({{{}!tokenMetadata.isMember(endpoint){}}}) > * The endpoint’s {{hostId}} matches the {{hostId}} of {{new_IP}} > * The endpoint is older than {{leaving_IP}} > ({{{}Gossiper.instance.compareEndpointStartup{}}}) > * The endpoint’s token range (from {{{}endpointStateMap{}}}) intersects with > {{{}new_IP{}}}’s > This modification’s intention is to force nodes to realign on {{old_IP}} > expiration, and expunge it from Gossip so it does not reappear after > {{new_IP}} leaves the ring. > Another approach we have also been considering is expunging {{old_IP}} at the > moment of the StorageService collision resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18319) Cassandra in Kubernetes: IP switch decommission issue
[ https://issues.apache.org/jira/browse/CASSANDRA-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ines Potier updated CASSANDRA-18319: Description: We have recently encountered a recurring old IP reappearance issue while testing decommissions on some of our Kubernetes Cassandra staging clusters. *Issue Description* In Kubernetes, a Cassandra node can change IP at each pod bounce. We have noticed that this behavior, associated with a decommission operation, can get the cluster into an erroneous state. Consider the following situation: a Cassandra node {{node1}} , with {{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP ({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all other nodes’ nodetool status output includes a {{new_IP}} UN entry owning 20.5% of the token ring and no {{old_IP}} entry. Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not have a lot of data, and the decommission operation completes pretty quickly. Logs on other nodes start showing acknowledgment that {{node1}} has left and soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is deleted. After a minute delay, the cluster enters the erroneous state. An {{old_IP}} DN entry reappears in nodetool status, owning 20.5% of the token ring. No node owns this IP anymore and according to logs, {{old_IP}} is still associated with {{{}hostId1{}}}. *Issue Root Cause* By digging through Cassandra logs, and re-testing this scenario over and over again, we have reached the following conclusion: * Other nodes will continue exchanging gossip about {{old_IP}} , even after it becomes a fatClient. * The fatClient timeout and subsequent quarantine does not stop {{old_IP}} from reappearing in a node’s Gossip state, once its quarantine is over. We believe that this is due to a misalignment on all nodes’ {{old_IP}} expiration time. * Once {{new_IP}} has left the cluster, and {{old_IP}} next gossip state message is received by a node, StorageService will no longer face collisions (or will, but with an even older IP) for {{hostId1}} and its corresponding tokens. As a result, {{old_IP}} will regain ownership of 20.5% of the token ring. *Proposed fix* Following the above investigation, we were thinking about implementing the following fix: When a node receives a gossip status change with {{STATE_LEFT}} for a leaving endpoint {{{}new_IP{}}}, before evicting {{{}new_IP from the token ring, purge from Gossip (ie evictFromMembership{}}}) all endpoints that meet the following criteria: * {{endpointStateMap}} contains this endpoint * The endpoint is not currently a token owner ({{{}!tokenMetadata.isMember(endpoint){}}}) * The endpoint’s {{hostId}} matches the {{hostId}} of {{new_IP}} * The endpoint is older than {{leaving_IP}} ({{{}Gossiper.instance.compareEndpointStartup{}}}) * The endpoint’s token range (from {{{}endpointStateMap{}}}) intersects with {{{}new_IP{}}}’s This modification’s intention is to force nodes to realign on {{old_IP}} expiration, and expunge it from Gossip so it does not reappear after {{new_IP}} leaves the ring. Another approach we have also been considering is expunging {{old_IP}} at the moment of the StorageService collision resolution. was: We have recently encountered a recurring old IP reappearance issue while testing decommissions on some of our Kubernetes Cassandra staging clusters. *Issue Description* In Kubernetes, a Cassandra node can change IP at each pod bounce. We have noticed that this behavior, associated with a decommission operation, can get the cluster into an erroneous state. Consider the following situation: a Cassandra node {{node1}} , with {{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP ({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all other nodes’ nodetool status output includes a {{new_IP}} UN entry owning 20.5% of the token ring and no {{old_IP}} entry. Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not have a lot of data, and the decommission operation completes pretty quickly. Logs on other nodes start showing acknowledgment that {{node1}} has left and soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is deleted. After a minute delay, the cluster enters the erroneous state. An {{old_IP}} DN entry reappears in nodetool status, owning 20.5% of the token ring. No node owns this IP anymore and according to logs, {{old_IP}} is still associated with {{{}hostId1{}}}. *Issue Root Cause* By digging through Cassandra logs, and re-testing this scenario over and over again, we have reached the following conclusion: * Other nodes will continue exchanging gossip about {{old_IP}} , even after it becomes a fatClient. * The fatClient timeout and subsequent quarantine does not stop {{old_IP}} from
[cassandra-website] branch asf-staging updated (7b6fcdaa -> 0116387e)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 7b6fcdaa generate docs for c4206294 new 0116387e generate docs for c4206294 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (7b6fcdaa) \ N -- N -- N refs/heads/asf-staging (0116387e) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4796442 -> 4796442 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
Stefan Miklosovic created CASSANDRA-18331: - Summary: Extend implicit allow-filtering to clustering keys as well Key: CASSANDRA-18331 URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 Project: Cassandra Issue Type: Improvement Reporter: Stefan Miklosovic Assignee: Stefan Miklosovic This was overlooked in CASSANDRA-18238. We should be able to do selects on vtables not only when selecting on regular columns but also on clustering ones. Currently we can do that on regulars only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18331: -- Change Category: Operability Complexity: Low Hanging Fruit Component/s: CQL/Semantics Fix Version/s: 5.x Priority: Low (was: Normal) Status: Open (was: Triage Needed) > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18331: -- Test and Documentation Plan: CI Status: Patch Available (was: In Progress) PR: [https://github.com/apache/cassandra/pull/2217/files] j11 [https://app.circleci.com/pipelines/github/instaclustr/cassandra/1990/workflows/140131d1-9f0a-45b5-b8d9-65c2b91c9be4] j8 [https://app.circleci.com/pipelines/github/instaclustr/cassandra/1990/workflows/6742ae5f-adbf-4159-abbc-0e894b015534] > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700342#comment-17700342 ] Stefan Miklosovic commented on CASSANDRA-18331: --- [~aleksey] would you mind to take a look at this, please? It was overlook in CASSANDRA-18238. At least it was not released anywhere yet. > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18331) Extend implicit allow-filtering to clustering keys as well
[ https://issues.apache.org/jira/browse/CASSANDRA-18331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700342#comment-17700342 ] Stefan Miklosovic edited comment on CASSANDRA-18331 at 3/14/23 7:14 PM: [~aleksey] would you mind to take a look at this, please? It was overlooked in CASSANDRA-18238. At least it was not released anywhere yet. was (Author: smiklosovic): [~aleksey] would you mind to take a look at this, please? It was overlook in CASSANDRA-18238. At least it was not released anywhere yet. > Extend implicit allow-filtering to clustering keys as well > -- > > Key: CASSANDRA-18331 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18331 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Semantics >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Low > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > This was overlooked in CASSANDRA-18238. > We should be able to do selects on vtables not only when selecting on regular > columns but also on clustering ones. Currently we can do that on regulars > only. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700354#comment-17700354 ] David Capwell commented on CASSANDRA-18320: --- Circle CI is failing, so holding off until I can get a build working > Incompatible file system thrown while running Simulator > --- > > Key: CASSANDRA-18320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18320 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: David Capwell >Priority: Normal > Fix For: 4.1.x, 5.x > > > {code} > java.io.UncheckedIOException > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831) > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047) > at > org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816) > at > org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370) > at > org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345) > at > org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148) > at > org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33) > Caused by: java.nio.file.DirectoryNotEmptyException: > /cassandra/node1/commitlog > at > com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535) > at > com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517) > at > com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479) > at > com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465) > at > com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261) > at java.nio.file.Files.delete(Files.java:1126) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18320) Incompatible file system thrown while running Simulator
[ https://issues.apache.org/jira/browse/CASSANDRA-18320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700354#comment-17700354 ] David Capwell edited comment on CASSANDRA-18320 at 3/14/23 7:42 PM: Circle CI is failing, so holding off until I can get a build working... Jenkins is chugging alone, so will see what happens was (Author: dcapwell): Circle CI is failing, so holding off until I can get a build working > Incompatible file system thrown while running Simulator > --- > > Key: CASSANDRA-18320 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18320 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: David Capwell >Priority: Normal > Fix For: 4.1.x, 5.x > > > {code} > java.io.UncheckedIOException > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:831) > at > org.apache.cassandra.io.util.PathUtils.propagateUnchecked(PathUtils.java:816) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:257) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:381) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) > at java.util.ArrayList.forEach(ArrayList.java:1259) > at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:395) > at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:483) > at > java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472) > at > java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) > at > java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) > at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) > at > java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485) > at org.apache.cassandra.io.util.PathUtils.forEach(PathUtils.java:155) > at > org.apache.cassandra.io.util.PathUtils.deleteRecursive(PathUtils.java:378) > at > org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1047) > at > org.apache.cassandra.simulator.ClusterSimulation.close(ClusterSimulation.java:816) > at > org.apache.cassandra.simulator.SimulationRunner$Run.run(SimulationRunner.java:370) > at > org.apache.cassandra.simulator.SimulationRunner$BasicCommand.run(SimulationRunner.java:345) > at > org.apache.cassandra.simulator.paxos.PaxosSimulationRunner.main(PaxosSimulationRunner.java:148) > at > org.apache.cassandra.simulator.test.ShortPaxosSimulationTest.simulationTest(ShortPaxosSimulationTest.java:33) > Caused by: java.nio.file.DirectoryNotEmptyException: > /cassandra/node1/commitlog > at > com.google.common.jimfs.FileSystemView.checkEmpty(FileSystemView.java:535) > at > com.google.common.jimfs.FileSystemView.checkDeletable(FileSystemView.java:517) > at > com.google.common.jimfs.FileSystemView.delete(FileSystemView.java:479) > at > com.google.common.jimfs.FileSystemView.deleteFile(FileSystemView.java:465) > at > com.google.common.jimfs.JimfsFileSystemProvider.delete(JimfsFileSystemProvider.java:261) > at java.nio.file.Files.delete(Files.java:1126) > at org.apache.cassandra.io.util.PathUtils.delete(PathUtils.java:252) > {code} -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18319) Cassandra in Kubernetes: IP switch decommission issue
[ https://issues.apache.org/jira/browse/CASSANDRA-18319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17700355#comment-17700355 ] Brandon Williams commented on CASSANDRA-18319: -- bq. We believe that this is due to a misalignment on all nodes’ old_IP expiration time. Do you have any logs to support this theory? > Cassandra in Kubernetes: IP switch decommission issue > - > > Key: CASSANDRA-18319 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18319 > Project: Cassandra > Issue Type: Bug >Reporter: Ines Potier >Priority: Normal > > We have recently encountered a recurring old IP reappearance issue while > testing decommissions on some of our Kubernetes Cassandra staging clusters. > *Issue Description* > In Kubernetes, a Cassandra node can change IP at each pod bounce. We have > noticed that this behavior, associated with a decommission operation, can get > the cluster into an erroneous state. > Consider the following situation: a Cassandra node {{node1}} , with > {{{}hostId1{}}}, owning 20.5% of the token ring, bounces and switches IP > ({{{}old_IP{}}} → {{{}new_IP{}}}). After a couple gossip iterations, all > other nodes’ nodetool status output includes a {{new_IP}} UN entry owning > 20.5% of the token ring and no {{old_IP}} entry. > Shortly after the bounce, {{node1}} gets decommissioned. Our cluster does not > have a lot of data, and the decommission operation completes pretty quickly. > Logs on other nodes start showing acknowledgment that {{node1}} has left and > soon, nodetool status’ {{new_IP}} UL entry disappears. {{node1}} ‘s pod is > deleted. > After a minute delay, the cluster enters the erroneous state. An {{old_IP}} > DN entry reappears in nodetool status, owning 20.5% of the token ring. No > node owns this IP anymore and according to logs, {{old_IP}} is still > associated with {{{}hostId1{}}}. > *Issue Root Cause* > By digging through Cassandra logs, and re-testing this scenario over and over > again, we have reached the following conclusion: > * Other nodes will continue exchanging gossip about {{old_IP}} , even after > it becomes a fatClient. > * The fatClient timeout and subsequent quarantine does not stop {{old_IP}} > from reappearing in a node’s Gossip state, once its quarantine is over. We > believe that this is due to a misalignment on all nodes’ {{old_IP}} > expiration time. > * Once {{new_IP}} has left the cluster, and {{old_IP}} next gossip state > message is received by a node, StorageService will no longer face collisions > (or will, but with an even older IP) for {{hostId1}} and its corresponding > tokens. As a result, {{old_IP}} will regain ownership of 20.5% of the token > ring. > *Proposed fix* > Following the above investigation, we were thinking about implementing the > following fix: > When a node receives a gossip status change with {{STATE_LEFT}} for a leaving > endpoint {{{}new_IP{}}}, before evicting {{{}new_IP from the token ring, > purge from Gossip (ie evictFromMembership{}}}) all endpoints that meet the > following criteria: > * {{endpointStateMap}} contains this endpoint > * The endpoint is not currently a token owner > ({{{}!tokenMetadata.isMember(endpoint){}}}) > * The endpoint’s {{hostId}} matches the {{hostId}} of {{new_IP}} > * The endpoint is older than {{leaving_IP}} > ({{{}Gossiper.instance.compareEndpointStartup{}}}) > * The endpoint’s token range (from {{{}endpointStateMap{}}}) intersects with > {{{}new_IP{}}}’s > This modification’s intention is to force nodes to realign on {{old_IP}} > expiration, and expunge it from Gossip so it does not reappear after > {{new_IP}} leaves the ring. > Another approach we have also been considering is expunging {{old_IP}} at the > moment of the StorageService collision resolution. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17611) Fix replace_address_test.TestReplaceAddress.test_fail_when_seed
[ https://issues.apache.org/jira/browse/CASSANDRA-17611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-17611: - Resolution: Fixed Status: Resolved (was: Open) Unrelated environmental timeout. > Fix replace_address_test.TestReplaceAddress.test_fail_when_seed > --- > > Key: CASSANDRA-17611 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17611 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Ekaterina Dimitrova >Assignee: Brandon Williams >Priority: Normal > Fix For: 3.0.27, 3.11.13 > > > Seen > [with|https://ci-cassandra.apache.org/job/Cassandra-3.11/349/testReport/dtest.replace_address_test/TestReplaceAddress/test_fail_when_seed/] > and [without > vnode|https://ci-cassandra.apache.org/job/Cassandra-3.11/348/testReport/dtest-novnode.replace_address_test/TestReplaceAddress/test_fail_when_seed/]: > {code:java} > Error Message > test teardown failure > Stacktrace > Unexpected error found in node logs (see stdout for full details). Errors: > [ERROR [MessagingService-Incoming-/127.0.0.1] 2022-04-28 22:04:36,765 > CassandraDaemon.java:244 - Exception in thread > Thread[MessagingService-Incoming-/127.0.0.1,5,main] > java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut > down at > org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:58) > at > java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) > at > java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) > at > org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:148) > at > org.apache.cassandra.net.MessagingService.receive(MessagingService.java:1035) > at > org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:215) > at > org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:195) > at > org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:98), > ERROR [MessagingService-Incoming-/127.0.0.1] 2022-04-28 22:04:36,765 > CassandraDaemon.java:244 - Exception in thread > Thread[MessagingService-Incoming-/127.0.0.1,5,main] > java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut > down at > org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:58) > at > java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) > at > java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) > at > org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:148) > at > org.apache.cassandra.net.MessagingService.receive(MessagingService.java:1035) > at > org.apache.cassandra.net.IncomingTcpConnection.receiveMessage(IncomingTcpConnection.java:215) > at > org.apache.cassandra.net.IncomingTcpConnection.receiveMessages(IncomingTcpConnection.java:195) > at > org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:98)] > {code} > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-harry] 05/09: Make it possible to run multiple Harry runners concurrently against the same keyspace
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit c0e749ee4f00d20df44cc9a243dac3e747bdd601 Author: Alex Petrov AuthorDate: Thu Mar 9 17:28:44 2023 +0100 Make it possible to run multiple Harry runners concurrently against the same keyspace Patch by Alex Petrov; reviewed by Abe Ratnofsky for CASSANDRA-18315 --- harry-core/src/harry/core/Configuration.java | 51 +++--- harry-core/src/harry/generators/RngUtils.java | 12 + harry-core/src/harry/model/OpSelectors.java| 19 +-- harry-core/src/harry/runner/Runner.java| 5 ++ .../{RecentValidator.java => RandomValidator.java} | 62 +++--- harry-core/src/harry/visitors/RecentValidator.java | 26 ++--- .../test/harry/model/IntegrationTestBase.java | 2 +- .../model/QuiescentCheckerIntegrationTest.java | 4 +- 8 files changed, 102 insertions(+), 79 deletions(-) diff --git a/harry-core/src/harry/core/Configuration.java b/harry-core/src/harry/core/Configuration.java index 88474b9..0c0e3f5 100644 --- a/harry-core/src/harry/core/Configuration.java +++ b/harry-core/src/harry/core/Configuration.java @@ -61,8 +61,9 @@ import harry.visitors.LoggingVisitor; import harry.visitors.MutatingVisitor; import harry.visitors.MutatingRowVisitor; import harry.visitors.OperationExecutor; -import harry.visitors.Visitor; import harry.visitors.RecentValidator; +import harry.visitors.Visitor; +import harry.visitors.RandomValidator; import harry.runner.Runner; import harry.util.BitSet; import org.reflections.Reflections; @@ -291,7 +292,7 @@ public class Configuration DataTrackerConfiguration data_tracker = new DefaultDataTrackerConfiguration(); RunnerConfiguration runner; SutConfiguration system_under_test; -PDSelectorConfiguration partition_descriptor_selector = new Configuration.DefaultPDSelectorConfiguration(10, 100); +PDSelectorConfiguration partition_descriptor_selector = new Configuration.DefaultPDSelectorConfiguration(10, 100, 0); CDSelectorConfiguration clustering_descriptor_selector; // TODO: sensible default value public ConfigurationBuilder setSeed(long seed) @@ -719,18 +720,30 @@ public class Configuration { public final int window_size; public final int slide_after_repeats; +public final long position_offset; + +@Deprecated +public DefaultPDSelectorConfiguration(int window_size, + int slide_after_repeats) +{ +this.window_size = window_size; +this.slide_after_repeats = slide_after_repeats; +this.position_offset = 0L; +} @JsonCreator public DefaultPDSelectorConfiguration(@JsonProperty(value = "window_size", defaultValue = "10") int window_size, - @JsonProperty(value = "slide_after_repeats", defaultValue = "100") int slide_after_repeats) + @JsonProperty(value = "slide_after_repeats", defaultValue = "100") int slide_after_repeats, + @JsonProperty(value = "position_offset", defaultValue = "0") long position_offset) { this.window_size = window_size; this.slide_after_repeats = slide_after_repeats; +this.position_offset = position_offset; } public OpSelectors.PdSelector make(OpSelectors.Rng rng) { -return new OpSelectors.DefaultPdSelector(rng, window_size, slide_after_repeats); +return new OpSelectors.DefaultPdSelector(rng, window_size, slide_after_repeats, position_offset); } } @@ -1083,27 +1096,49 @@ public class Configuration public static class RecentPartitionsValidatorConfiguration implements VisitorConfiguration { public final int partition_count; -public final int trigger_after; public final int queries; public final Configuration.ModelConfiguration modelConfiguration; // TODO: make query selector configurable @JsonCreator public RecentPartitionsValidatorConfiguration(@JsonProperty("partition_count") int partition_count, - @JsonProperty("trigger_after") int trigger_after, @JsonProperty("queries_per_partition") int queries, @JsonProperty("model") Configuration.ModelConfiguration model) { this.partition_count = partition_count; this.queries = queries; -this.trigger_after = trigger_after; this.modelConfiguration = model; } @Override pub
[cassandra-harry] 06/09: Allow selecting subsets of columns and wilcard queries. Open API for hand-crafting both mutation and read queries Improve errors Fix a problem with corruptor sorting
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit a8b9869972ddd7adf68e12568d70b2c382ae1ad9 Author: Alex Petrov AuthorDate: Thu Mar 9 17:28:44 2023 +0100 Allow selecting subsets of columns and wilcard queries. Open API for hand-crafting both mutation and read queries Improve errors Fix a problem with corruptor sorting Patch by Alex Petrov for CASSANDRA-17603. --- .../src/harry/corruptor/ChangeValueCorruptor.java | 2 +- .../harry/corruptor/QueryResponseCorruptor.java| 16 +- harry-core/src/harry/data/ResultSetRow.java| 7 + harry-core/src/harry/ddl/SchemaSpec.java | 8 +- .../src/harry/generators/DataGenerators.java | 2 + harry-core/src/harry/model/OpSelectors.java| 2 +- harry-core/src/harry/model/QuiescentChecker.java | 96 -- harry-core/src/harry/model/SelectHelper.java | 142 +++-- harry-core/src/harry/operations/Query.java | 233 ++- .../src/harry/operations/QueryGenerator.java | 241 +++ .../src/harry/reconciler/PartitionState.java | 266 + harry-core/src/harry/reconciler/Reconciler.java| 234 +-- .../generators/DataGeneratorsIntegrationTest.java | 2 +- .../ConcurrentQuiescentCheckerIntegrationTest.java | 2 +- .../harry/model/HistoryBuilderIntegrationTest.java | 2 +- .../test/harry/model/ModelTestBase.java| 2 +- .../harry/model/QuerySelectorNegativeTest.java | 2 - .../model/QuiescentCheckerIntegrationTest.java | 5 +- .../QuiescentLocalStateCheckerIntegrationTest.java | 2 +- .../harry/reconciler/SimpleReconcilerTest.java | 332 + 20 files changed, 1107 insertions(+), 491 deletions(-) diff --git a/harry-core/src/harry/corruptor/ChangeValueCorruptor.java b/harry-core/src/harry/corruptor/ChangeValueCorruptor.java index 5f23a06..3074679 100644 --- a/harry-core/src/harry/corruptor/ChangeValueCorruptor.java +++ b/harry-core/src/harry/corruptor/ChangeValueCorruptor.java @@ -75,7 +75,7 @@ public class ChangeValueCorruptor implements RowCorruptor final long oldV = row.vds[idx]; do { -corruptedVds[idx] =+ rng.next(); +corruptedVds[idx] = schema.regularColumns.get(idx).type.generator().adjustEntropyDomain(rng.next()); } // we need to find a value that sorts strictly greater than the current one while (schema.regularColumns.get(idx).type.compareLexicographically(corruptedVds[idx], oldV) <= 0); diff --git a/harry-core/src/harry/corruptor/QueryResponseCorruptor.java b/harry-core/src/harry/corruptor/QueryResponseCorruptor.java index 62bf589..bf29754 100644 --- a/harry-core/src/harry/corruptor/QueryResponseCorruptor.java +++ b/harry-core/src/harry/corruptor/QueryResponseCorruptor.java @@ -84,11 +84,25 @@ public interface QueryResponseCorruptor mismatch = true; } } -assert mismatch || before.length != after.length; +assert mismatch || before.length != after.length : String.format("Could not corrupt.\n" + + "Before\n%s\n" + + "After\n%s\nkma", + toString(before), + toString(after)); return true; } } return false; } + +private static String toString(Object[][] obj) +{ +StringBuilder sb = new StringBuilder(); +for (Object[] objects : obj) +{ +sb.append(Arrays.toString(objects)).append("\n"); +} +return sb.toString(); +} } } diff --git a/harry-core/src/harry/data/ResultSetRow.java b/harry-core/src/harry/data/ResultSetRow.java index d6ff77b..8f37c82 100644 --- a/harry-core/src/harry/data/ResultSetRow.java +++ b/harry-core/src/harry/data/ResultSetRow.java @@ -48,6 +48,13 @@ public class ResultSetRow this.slts = slts; } +public ResultSetRow clone() +{ +return new ResultSetRow(pd, cd, +Arrays.copyOf(sds, sds.length), Arrays.copyOf(slts, slts.length), +Arrays.copyOf(vds, vds.length), Arrays.copyOf(lts, lts.length)); +} + public String toString() { return "resultSetRow(" diff --git a/harry-core/src/harry/ddl/SchemaSpec.java b/harry-core/src/harry/ddl/SchemaSpec.java index 8d8a62f..4a8a2de 100644 --- a/harry-core/src/harry/ddl/SchemaSp
[cassandra-harry] 01/09: Improvements:
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit d4ecaf80e69ea3bd3d8a9538ae3f70851997b6cf Author: Abe Ratnofsky AuthorDate: Mon Oct 17 12:54:44 2022 +0200 Improvements: * formatting * thread shutdown on failed run creation * idempotent query execution * retry delay Patch by Abe Ratnofsky; reviewed by Alex Petrov for CASSANDRA-18315. --- harry-core/src/harry/core/Configuration.java | 56 ++ harry-core/src/harry/ddl/ColumnSpec.java | 16 +++ harry-core/src/harry/ddl/SchemaGenerators.java | 41 ++-- harry-core/src/harry/generators/Bijections.java| 35 +- harry-core/src/harry/generators/PCGFastPure.java | 2 +- harry-core/src/harry/model/SelectHelper.java | 2 +- .../src/harry/model/sut/SystemUnderTest.java | 12 - harry-core/src/harry/runner/HarryRunner.java | 6 +++ harry-core/src/harry/runner/Runner.java| 9 +++- harry-core/src/harry/visitors/MutatingVisitor.java | 6 ++- .../harry/visitors/ParallelRecentValidator.java| 4 +- harry-core/src/harry/visitors/RecentValidator.java | 2 +- 12 files changed, 145 insertions(+), 46 deletions(-) diff --git a/harry-core/src/harry/core/Configuration.java b/harry-core/src/harry/core/Configuration.java index 3545e18..7cd8f72 100644 --- a/harry-core/src/harry/core/Configuration.java +++ b/harry-core/src/harry/core/Configuration.java @@ -65,6 +65,8 @@ public class Configuration { private static final ObjectMapper mapper; +private static final String DEFAULT_KEYSPACE = "harry"; + static { mapper = new ObjectMapper(new YAMLFactory() @@ -220,34 +222,46 @@ public class Configuration public static Run createRun(Configuration snapshot) { -validate(snapshot); +SystemUnderTest sut = null; +try +{ +validate(snapshot); -long seed = snapshot.seed; +long seed = snapshot.seed; -DataTracker tracker = snapshot.data_tracker == null ? new DefaultDataTrackerConfiguration().make() : snapshot.data_tracker.make(); -OpSelectors.Rng rng = new OpSelectors.PCGFast(seed); +DataTracker tracker = snapshot.data_tracker == null ? new DefaultDataTrackerConfiguration().make() : snapshot.data_tracker.make(); +OpSelectors.Rng rng = new OpSelectors.PCGFast(seed); -OpSelectors.MonotonicClock clock = snapshot.clock.make(); +OpSelectors.MonotonicClock clock = snapshot.clock.make(); -MetricReporter metricReporter = snapshot.metric_reporter.make(); +MetricReporter metricReporter = snapshot.metric_reporter.make(); -// TODO: validate that operation kind is compatible with schema, due to statics etc -SystemUnderTest sut = snapshot.system_under_test.make(); +// TODO: validate that operation kind is compatible with schema, due to statics etc +sut = snapshot.system_under_test.make(); -SchemaSpec schemaSpec = snapshot.schema_provider.make(seed, sut); -schemaSpec.validate(); +SchemaSpec schemaSpec = snapshot.schema_provider.make(seed, sut); +schemaSpec.validate(); -OpSelectors.PdSelector pdSelector = snapshot.partition_descriptor_selector.make(rng); -OpSelectors.DescriptorSelector descriptorSelector = snapshot.clustering_descriptor_selector.make(rng, schemaSpec); +OpSelectors.PdSelector pdSelector = snapshot.partition_descriptor_selector.make(rng); +OpSelectors.DescriptorSelector descriptorSelector = snapshot.clustering_descriptor_selector.make(rng, schemaSpec); -return new Run(rng, - clock, - pdSelector, - descriptorSelector, - schemaSpec, - tracker, - sut, - metricReporter); +return new Run(rng, + clock, + pdSelector, + descriptorSelector, + schemaSpec, + tracker, + sut, + metricReporter); +} +catch (Throwable t) +{ +// Make sure to shut down all SUT threads if it has been started +if (sut != null) { +sut.shutdown(); +} +throw t; +} } public static Runner createRunner(Configuration config) @@ -1058,7 +1072,7 @@ public class Configuration { public SchemaSpec make(long seed, SystemUnderTest sut) { -return SchemaGenerators.defaultSchemaSpecGen("harry", "table0") +return SchemaGenerators.defaultS
[cassandra-harry] 03/09: Pull in token util from Cassandra to avoid circular dependency
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit 6e0b450d9df3596d2eb79bfca92edd434cca7035 Author: Alex Petrov AuthorDate: Thu Mar 9 17:17:41 2023 +0100 Pull in token util from Cassandra to avoid circular dependency Patch by Alex Petrov; reviewed by Caleb Rackliffe and Abe Ratnofsky for CASSANDRA-18315 --- harry-core/src/harry/util/TokenUtil.java | 150 +++ 1 file changed, 150 insertions(+) diff --git a/harry-core/src/harry/util/TokenUtil.java b/harry-core/src/harry/util/TokenUtil.java new file mode 100644 index 000..db210b6 --- /dev/null +++ b/harry-core/src/harry/util/TokenUtil.java @@ -0,0 +1,150 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package harry.util; + +import java.nio.ByteBuffer; + +/** + * Copied verbatim from Apache Cassandra codebase to avoid having a dependency on Cassandra. + */ +public class TokenUtil +{ +public static long token(ByteBuffer key) +{ +if (key.remaining() == 0) +return Long.MIN_VALUE; + +return normalize(getHash(key)[0]); +} + +private static long normalize(long v) +{ +// We exclude the MINIMUM value; see getToken() +return v == Long.MIN_VALUE ? Long.MAX_VALUE : v; +} + +private static long[] getHash(ByteBuffer key) +{ +long[] hash = new long[2]; +hash3_x64_128(key, key.position(), key.remaining(), 0, hash); +return hash; +} + +public static void hash3_x64_128(ByteBuffer key, int offset, int length, long seed, long[] result) +{ +final int nblocks = length >> 4; // Process as 128-bit blocks. + +long h1 = seed; +long h2 = seed; + +long c1 = 0x87c37b91114253d5L; +long c2 = 0x4cf5ad432745937fL; + +//-- +// body + +for(int i = 0; i < nblocks; i++) +{ +long k1 = getBlock(key, offset, i * 2 + 0); +long k2 = getBlock(key, offset, i * 2 + 1); + +k1 *= c1; k1 = rotl64(k1,31); k1 *= c2; h1 ^= k1; + +h1 = rotl64(h1,27); h1 += h2; h1 = h1*5+0x52dce729; + +k2 *= c2; k2 = rotl64(k2,33); k2 *= c1; h2 ^= k2; + +h2 = rotl64(h2,31); h2 += h1; h2 = h2*5+0x38495ab5; +} + +//-- +// tail + +// Advance offset to the unprocessed tail of the data. +offset += nblocks * 16; + +long k1 = 0; +long k2 = 0; + +switch(length & 15) +{ +case 15: k2 ^= ((long) key.get(offset+14)) << 48; +case 14: k2 ^= ((long) key.get(offset+13)) << 40; +case 13: k2 ^= ((long) key.get(offset+12)) << 32; +case 12: k2 ^= ((long) key.get(offset+11)) << 24; +case 11: k2 ^= ((long) key.get(offset+10)) << 16; +case 10: k2 ^= ((long) key.get(offset+9)) << 8; +case 9: k2 ^= ((long) key.get(offset+8)) << 0; +k2 *= c2; k2 = rotl64(k2,33); k2 *= c1; h2 ^= k2; + +case 8: k1 ^= ((long) key.get(offset+7)) << 56; +case 7: k1 ^= ((long) key.get(offset+6)) << 48; +case 6: k1 ^= ((long) key.get(offset+5)) << 40; +case 5: k1 ^= ((long) key.get(offset+4)) << 32; +case 4: k1 ^= ((long) key.get(offset+3)) << 24; +case 3: k1 ^= ((long) key.get(offset+2)) << 16; +case 2: k1 ^= ((long) key.get(offset+1)) << 8; +case 1: k1 ^= ((long) key.get(offset)); +k1 *= c1; k1 = rotl64(k1,31); k1 *= c2; h1 ^= k1; +}; + +//-- +// finalization + +h1 ^= length; h2 ^= length; + +h1 += h2; +h2 += h1; + +h1 = fmix(h1); +h2 = fmix(h2); + +h1 += h2; +h2 += h1; + +result[0] = h1; +result[1] = h2; +} + +protected static long getBlock(ByteBuffer key, int offset, int index) +{ +int i_8 = index << 3; +int blockOffset = offset + i_8; +return ((long) key.get(blockOffset + 0) & 0xff) + (((long) key.get(blockOffs
[cassandra-harry] 07/09: Minor API improvements
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-harry.git commit 8a3748c29048b8005dab098ad031b8f75efd942e Author: Alex Petrov AuthorDate: Tue Mar 14 13:08:04 2023 +0100 Minor API improvements Patch by Alex Petrov; reviewed by Abe Ratnofsky for CASSANDRA-18315 --- conf/default.yaml | 3 - conf/external.yaml | 3 - harry-core/src/harry/core/Configuration.java | 79 -- .../harry/model/RepairingLocalStateValidator.java | 43 +- harry-core/src/harry/runner/Runner.java| 4 - .../src/harry/visitors/AllPartitionsValidator.java | 44 +- .../src/harry/visitors/CorruptingVisitor.java | 3 +- .../harry/visitors/ParallelRecentValidator.java| 55 +++- .../src/harry/visitors/ParallelValidator.java | 24 ++ harry-core/src/harry/visitors/QueryLogger.java | 97 ++ .../harry/visitors/RandomPartitionValidator.java | 10 +-- harry-core/src/harry/visitors/RandomValidator.java | 71 +--- harry-core/src/harry/visitors/RecentValidator.java | 48 ++- harry-core/src/harry/visitors/Sampler.java | 56 + .../src/harry/runner/external/MiniStress.java | 5 +- .../ConcurrentQuiescentCheckerIntegrationTest.java | 3 +- .../harry/model/InJVMTokenAwareExecutorTest.java | 2 +- .../model/QuiescentCheckerIntegrationTest.java | 7 +- .../QuiescentLocalStateCheckerIntegrationTest.java | 2 +- 19 files changed, 273 insertions(+), 286 deletions(-) diff --git a/conf/default.yaml b/conf/default.yaml index a004747..4a20782 100644 --- a/conf/default.yaml +++ b/conf/default.yaml @@ -87,18 +87,15 @@ runner: row_visitor: mutating: {} - sampler: - trigger_after: 1000 sample_partitions: 10 - parallel_validate_recent_partitions: partition_count: 100 queries_per_partition: 2 concurrency: 20 - trigger_after: 1 model: quiescent_checker: {} - validate_all_partitions: concurrency: 20 - trigger_after: 10 model: quiescent_checker: {} diff --git a/conf/external.yaml b/conf/external.yaml index b75547e..ae5fda2 100644 --- a/conf/external.yaml +++ b/conf/external.yaml @@ -109,18 +109,15 @@ runner: row_visitor: mutating: {} - sampler: - trigger_after: 1000 sample_partitions: 10 - parallel_validate_recent_partitions: partition_count: 100 queries_per_partition: 2 concurrency: 20 - trigger_after: 1 model: quiescent_checker: {} - validate_all_partitions: concurrency: 20 - trigger_after: 10 model: quiescent_checker: {} diff --git a/harry-core/src/harry/core/Configuration.java b/harry-core/src/harry/core/Configuration.java index 0c0e3f5..9427c89 100644 --- a/harry-core/src/harry/core/Configuration.java +++ b/harry-core/src/harry/core/Configuration.java @@ -34,7 +34,6 @@ import java.util.Objects; import java.util.Set; import java.util.concurrent.TimeUnit; import java.util.function.Consumer; -import java.util.stream.Collectors; import com.fasterxml.jackson.annotation.JsonCreator; import com.fasterxml.jackson.annotation.JsonProperty; @@ -53,17 +52,9 @@ import harry.model.QuiescentChecker; import harry.model.clock.ApproximateMonotonicClock; import harry.model.sut.SystemUnderTest; import harry.runner.LockingDataTracker; -import harry.visitors.AllPartitionsValidator; -import harry.visitors.CorruptingVisitor; +import harry.visitors.*; import harry.runner.DataTracker; import harry.runner.DefaultDataTracker; -import harry.visitors.LoggingVisitor; -import harry.visitors.MutatingVisitor; -import harry.visitors.MutatingRowVisitor; -import harry.visitors.OperationExecutor; -import harry.visitors.RecentValidator; -import harry.visitors.Visitor; -import harry.visitors.RandomValidator; import harry.runner.Runner; import harry.util.BitSet; import org.reflections.Reflections; @@ -500,8 +491,6 @@ public class Configuration } } - - @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.WRAPPER_OBJECT) public interface ClockConfiguration extends OpSelectors.MonotonicClockFactory { @@ -1054,24 +1043,22 @@ public class Configuration public static class AllPartitionsValidatorConfiguration implements VisitorConfiguration { public final int concurrency; -public final int trigger_after; - -@JsonProperty("model") -public final Configuration.ModelConfiguration modelConfiguration; +public final QueryLoggerConfiguration query_logger; +public final Configuration