[cassandra] branch trunk updated: Fix potentially flaky ImportTest
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new e61c09e Fix potentially flaky ImportTest e61c09e is described below commit e61c09eaf5e0b3f5d374032a690ea4bf17b36e2f Author: Marcus Eriksson AuthorDate: Wed Jan 22 19:11:55 2020 +0100 Fix potentially flaky ImportTest Patch by marcuse; reviewed by David Capwell for CASSANDRA-15517 --- test/unit/org/apache/cassandra/db/ImportTest.java | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/test/unit/org/apache/cassandra/db/ImportTest.java b/test/unit/org/apache/cassandra/db/ImportTest.java index 5ceb233..4094aa4 100644 --- a/test/unit/org/apache/cassandra/db/ImportTest.java +++ b/test/unit/org/apache/cassandra/db/ImportTest.java @@ -308,7 +308,7 @@ public class ImportTest extends CQLTester getCurrentColumnFamilyStore().clearUnsafe(); File backupdirCorrect = moveToBackupDir(correctSSTables); -File [] beforeImport = backupdir.listFiles(); +Set beforeImport = Sets.newHashSet(backupdir.listFiles()); // first we moved out 2 sstables, one correct and one corrupt in to a single directory (backupdir) // then we moved out 1 sstable, a correct one (in backupdirCorrect). // now import should fail import on backupdir, but import the one in backupdirCorrect. @@ -323,7 +323,7 @@ public class ImportTest extends CQLTester assertTrue("pk = "+pk, pk >= 100 && pk < 130); } assertEquals("Data dir should contain one file", 1, countFiles(getCurrentColumnFamilyStore().getDirectories().getDirectoryForNewSSTables())); -assertArrayEquals("backupdir contained 2 files before import, should still contain 2 after failing to import it", beforeImport, backupdir.listFiles()); +assertEquals("backupdir contained 2 files before import, should still contain 2 after failing to import it", beforeImport, Sets.newHashSet(backupdir.listFiles())); assertEquals("backupdirCorrect contained 1 file before import, should be empty after import", 0, countFiles(backupdirCorrect)); } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15517) Fix potentially flaky ImportTest#testImportCorruptWithoutValidation
[ https://issues.apache.org/jira/browse/CASSANDRA-15517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15517: Fix Version/s: 4.0-alpha Since Version: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/e61c09eaf5e0b3f5d374032a690ea4bf17b36e2f Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks > Fix potentially flaky ImportTest#testImportCorruptWithoutValidation > --- > > Key: CASSANDRA-15517 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15517 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > The assert relies on the order of files in {{File.listFiles}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15508: Status: Patch Available (was: Ready to Commit) realised we want to commit this to all the branches (2.2, 3.0, 3.11, trunk) - could you provide patches for them as well [~dcapwell]? > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 10m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Exclude purgeable tombstones from repaired data digest
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 9a280516ca8b9e730ae0648e5e29ee6280605132 Author: Sam Tunnicliffe AuthorDate: Wed Dec 18 18:31:36 2019 + Exclude purgeable tombstones from repaired data digest Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-15462 --- CHANGES.txt| 1 + .../cassandra/db/PartitionRangeReadCommand.java| 2 +- src/java/org/apache/cassandra/db/ReadCommand.java | 160 ++-- .../cassandra/db/SinglePartitionReadCommand.java | 2 +- .../cassandra/db/partitions/PurgeFunction.java | 7 +- .../distributed/test/RepairDigestTrackingTest.java | 54 .../org/apache/cassandra/db/ReadCommandTest.java | 272 - 7 files changed, 473 insertions(+), 25 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 3d5217a..43126ed 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha3 + * Exclude purgeable tombstones from repaired data tracking (CASSANDRA-15462) * Exclude legacy counter shards from repaired data tracking (CASSANDRA-15461) * Make it easier to add trace headers to messages (CASSANDRA-15499) * Fix and optimise partial compressed sstable streaming (CASSANDRA-13938) diff --git a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java index 2145389..cb68950 100644 --- a/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java +++ b/src/java/org/apache/cassandra/db/PartitionRangeReadCommand.java @@ -282,7 +282,7 @@ public class PartitionRangeReadCommand extends ReadCommand implements PartitionR if (inputCollector.isEmpty()) return EmptyIterators.unfilteredPartition(metadata()); -return checkCacheFilter(UnfilteredPartitionIterators.mergeLazily(inputCollector.finalizeIterators()), cfs); +return checkCacheFilter(UnfilteredPartitionIterators.mergeLazily(inputCollector.finalizeIterators(cfs, nowInSec(), oldestUnrepairedTombstone)), cfs); } catch (RuntimeException | Error e) { diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java b/src/java/org/apache/cassandra/db/ReadCommand.java index 9485abc..4f8ea3e 100644 --- a/src/java/org/apache/cassandra/db/ReadCommand.java +++ b/src/java/org/apache/cassandra/db/ReadCommand.java @@ -90,7 +90,7 @@ public abstract class ReadCommand extends AbstractReadQuery // for data queries, coordinators may request information on the repaired data used in constructing the response private boolean trackRepairedStatus = false; -// tracker for repaired data, initialized to singelton null object +// tracker for repaired data, initialized to singleton null object private static final RepairedDataInfo NULL_REPAIRED_DATA_INFO = new RepairedDataInfo() { void trackPartitionKey(DecoratedKey key){} @@ -724,7 +724,7 @@ public abstract class ReadCommand extends AbstractReadQuery } private static UnfilteredPartitionIterator withRepairedDataInfo(final UnfilteredPartitionIterator iterator, - final RepairedDataInfo repairedDataInfo) +final RepairedDataInfo repairedDataInfo) { class WithRepairedDataTracking extends Transformation { @@ -744,6 +744,7 @@ public abstract class ReadCommand extends AbstractReadQuery { protected DecoratedKey applyToPartitionKey(DecoratedKey key) { +repairedDataInfo.onNewPartition(iterator); repairedDataInfo.trackPartitionKey(key); return key; } @@ -762,7 +763,7 @@ public abstract class ReadCommand extends AbstractReadQuery protected Row applyToStatic(Row row) { -repairedDataInfo.trackRow(row); +repairedDataInfo.trackStaticRow(row); return row; } @@ -771,21 +772,48 @@ public abstract class ReadCommand extends AbstractReadQuery repairedDataInfo.trackRow(row); return row; } -} +protected void onPartitionClose() +{ +repairedDataInfo.onPartitionClose(); +} +} return Transformation.apply(iterator, new WithTracking()); } private static class RepairedDataInfo { -private Digest hasher; +// Keeps a digest of the partition currently being processed. Since we won't know +// whether a partition will be fully purged from a read result until it's been +// consumed, we buffer this per-partition digest and add it to the final digest +// when the par
[cassandra] branch trunk updated (e61c09e -> 9a28051)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from e61c09e Fix potentially flaky ImportTest new 39eb7db Exclude legacy counters from repaired data digest new 9a28051 Exclude purgeable tombstones from repaired data digest The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 2 + .../cassandra/db/AbstractClusteringPrefix.java | 10 +- .../org/apache/cassandra/db/ClusteringPrefix.java | 8 +- src/java/org/apache/cassandra/db/Columns.java | 6 +- src/java/org/apache/cassandra/db/DeletionTime.java | 6 +- src/java/org/apache/cassandra/db/Digest.java | 200 src/java/org/apache/cassandra/db/LivenessInfo.java | 17 +- .../cassandra/db/PartitionRangeReadCommand.java| 2 +- src/java/org/apache/cassandra/db/ReadCommand.java | 164 -- src/java/org/apache/cassandra/db/ReadResponse.java | 8 +- .../cassandra/db/SinglePartitionReadCommand.java | 2 +- .../cassandra/db/context/CounterContext.java | 20 +- .../db/partitions/PartitionIterators.java | 1 - .../cassandra/db/partitions/PurgeFunction.java | 7 +- .../partitions/UnfilteredPartitionIterators.java | 8 +- .../org/apache/cassandra/db/rows/AbstractCell.java | 24 +- .../org/apache/cassandra/db/rows/AbstractRow.java | 15 +- .../org/apache/cassandra/db/rows/CellPath.java | 8 +- .../org/apache/cassandra/db/rows/ColumnData.java | 7 +- .../cassandra/db/rows/ComplexColumnData.java | 8 +- .../db/rows/RangeTombstoneBoundMarker.java | 8 +- .../db/rows/RangeTombstoneBoundaryMarker.java | 10 +- src/java/org/apache/cassandra/db/rows/Row.java | 8 +- .../org/apache/cassandra/db/rows/RowIterators.java | 17 +- .../org/apache/cassandra/db/rows/Unfiltered.java | 9 +- .../cassandra/db/rows/UnfilteredRowIterators.java | 20 +- .../apache/cassandra/dht/RandomPartitioner.java| 3 +- .../org/apache/cassandra/repair/Validator.java | 165 +- .../apache/cassandra/schema/SchemaConstants.java | 4 +- .../apache/cassandra/schema/SchemaKeyspace.java| 9 +- .../org/apache/cassandra/utils/FBUtilities.java| 14 + .../org/apache/cassandra/utils/HashingUtils.java | 109 --- src/java/org/apache/cassandra/utils/MD5Digest.java | 2 +- src/java/org/apache/cassandra/utils/UUIDGen.java | 24 +- .../distributed/test/RepairDigestTrackingTest.java | 54 .../apache/cassandra/cache/CacheProviderTest.java | 15 +- .../org/apache/cassandra/db/CounterCellTest.java | 17 +- test/unit/org/apache/cassandra/db/DigestTest.java | 122 +++ .../org/apache/cassandra/db/PartitionTest.java | 37 ++- .../org/apache/cassandra/db/ReadCommandTest.java | 354 - .../org/apache/cassandra/db/RowUpdateBuilder.java | 10 + .../org/apache/cassandra/repair/ValidatorTest.java | 47 +-- .../repair/asymmetric/DifferenceHolderTest.java| 6 +- .../apache/cassandra/utils/HashingUtilsTest.java | 92 -- .../org/apache/cassandra/utils/MerkleTreeTest.java | 7 +- .../apache/cassandra/utils/MerkleTreesTest.java| 6 +- 46 files changed, 1063 insertions(+), 629 deletions(-) create mode 100644 src/java/org/apache/cassandra/db/Digest.java delete mode 100644 src/java/org/apache/cassandra/utils/HashingUtils.java create mode 100644 test/unit/org/apache/cassandra/db/DigestTest.java delete mode 100644 test/unit/org/apache/cassandra/utils/HashingUtilsTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: Exclude legacy counters from repaired data digest
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 39eb7db65fd45653fdece1087ba75c3356a10c97 Author: Sam Tunnicliffe AuthorDate: Tue Dec 17 17:56:18 2019 + Exclude legacy counters from repaired data digest Patch by Sam Tunnicliffe; reviewed by Marcus Eriksson for CASSANDRA-15461 --- CHANGES.txt| 1 + .../cassandra/db/AbstractClusteringPrefix.java | 10 +- .../org/apache/cassandra/db/ClusteringPrefix.java | 8 +- src/java/org/apache/cassandra/db/Columns.java | 6 +- src/java/org/apache/cassandra/db/DeletionTime.java | 6 +- src/java/org/apache/cassandra/db/Digest.java | 200 + src/java/org/apache/cassandra/db/LivenessInfo.java | 17 +- src/java/org/apache/cassandra/db/ReadCommand.java | 14 +- src/java/org/apache/cassandra/db/ReadResponse.java | 8 +- .../cassandra/db/context/CounterContext.java | 20 +-- .../db/partitions/PartitionIterators.java | 1 - .../partitions/UnfilteredPartitionIterators.java | 8 +- .../org/apache/cassandra/db/rows/AbstractCell.java | 24 +-- .../org/apache/cassandra/db/rows/AbstractRow.java | 15 +- .../org/apache/cassandra/db/rows/CellPath.java | 8 +- .../org/apache/cassandra/db/rows/ColumnData.java | 7 +- .../cassandra/db/rows/ComplexColumnData.java | 8 +- .../db/rows/RangeTombstoneBoundMarker.java | 8 +- .../db/rows/RangeTombstoneBoundaryMarker.java | 10 +- src/java/org/apache/cassandra/db/rows/Row.java | 8 +- .../org/apache/cassandra/db/rows/RowIterators.java | 17 +- .../org/apache/cassandra/db/rows/Unfiltered.java | 9 +- .../cassandra/db/rows/UnfilteredRowIterators.java | 20 +-- .../apache/cassandra/dht/RandomPartitioner.java| 3 +- .../org/apache/cassandra/repair/Validator.java | 165 + .../apache/cassandra/schema/SchemaConstants.java | 4 +- .../apache/cassandra/schema/SchemaKeyspace.java| 9 +- .../org/apache/cassandra/utils/FBUtilities.java| 14 ++ .../org/apache/cassandra/utils/HashingUtils.java | 109 --- src/java/org/apache/cassandra/utils/MD5Digest.java | 2 +- src/java/org/apache/cassandra/utils/UUIDGen.java | 24 ++- .../apache/cassandra/cache/CacheProviderTest.java | 15 +- .../org/apache/cassandra/db/CounterCellTest.java | 17 +- test/unit/org/apache/cassandra/db/DigestTest.java | 122 + .../org/apache/cassandra/db/PartitionTest.java | 37 ++-- .../org/apache/cassandra/db/ReadCommandTest.java | 88 - .../org/apache/cassandra/db/RowUpdateBuilder.java | 10 ++ .../org/apache/cassandra/repair/ValidatorTest.java | 47 + .../repair/asymmetric/DifferenceHolderTest.java| 6 +- .../apache/cassandra/utils/HashingUtilsTest.java | 92 -- .../org/apache/cassandra/utils/MerkleTreeTest.java | 7 +- .../apache/cassandra/utils/MerkleTreesTest.java| 6 +- 42 files changed, 598 insertions(+), 612 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 85ddd53..3d5217a 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha3 + * Exclude legacy counter shards from repaired data tracking (CASSANDRA-15461) * Make it easier to add trace headers to messages (CASSANDRA-15499) * Fix and optimise partial compressed sstable streaming (CASSANDRA-13938) * Improve error when JVM 11 can't access required modules (CASSANDRA-15468) diff --git a/src/java/org/apache/cassandra/db/AbstractClusteringPrefix.java b/src/java/org/apache/cassandra/db/AbstractClusteringPrefix.java index 884a091..8714936 100644 --- a/src/java/org/apache/cassandra/db/AbstractClusteringPrefix.java +++ b/src/java/org/apache/cassandra/db/AbstractClusteringPrefix.java @@ -20,10 +20,6 @@ package org.apache.cassandra.db; import java.nio.ByteBuffer; import java.util.Objects; -import com.google.common.hash.Hasher; - -import org.apache.cassandra.utils.HashingUtils; - public abstract class AbstractClusteringPrefix implements ClusteringPrefix { public ClusteringPrefix clustering() @@ -42,15 +38,15 @@ public abstract class AbstractClusteringPrefix implements ClusteringPrefix return size; } -public void digest(Hasher hasher) +public void digest(Digest digest) { for (int i = 0; i < size(); i++) { ByteBuffer bb = get(i); if (bb != null) -HashingUtils.updateBytes(hasher, bb.duplicate()); +digest.update(bb); } -HashingUtils.updateWithByte(hasher, kind().ordinal()); +digest.updateWithByte(kind().ordinal()); } @Override diff --git a/src/java/org/apache/cassandra/db/ClusteringPrefix.java b/src/java/org/apache/cassandra/db/ClusteringPrefix.java index 2d9198b..357d746 100644 --- a/src/java/org/apache/cassandra/db/ClusteringPrefix.java +++ b/
[jira] [Updated] (CASSANDRA-15462) Purgable tombstones can cause false positives in repaired data tracking
[ https://issues.apache.org/jira/browse/CASSANDRA-15462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-15462: Fix Version/s: (was: 4.0-beta) 4.0-alpha Since Version: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/9a280516ca8b9e730ae0648e5e29ee6280605132 Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks, committed with nits addressed. > Purgable tombstones can cause false positives in repaired data tracking > --- > > Key: CASSANDRA-15462 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15462 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0-alpha > > > Calculating the repaired data digest on the read path (for the purposes of > detecting mismatches in the repaired data across replicas) is done before > purging any tombstones due to gcgs or last repaired time. This causes false > positives when repaired sstables include GC-able tombstones on some replicas > but not others. > Validation compactions do purge tombstones so it's perfectly possible for > sstables to mismatch in this way without causing any streaming during repair. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15461) Legacy counter shards can cause false positives in repaired data tracking
[ https://issues.apache.org/jira/browse/CASSANDRA-15461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-15461: Fix Version/s: (was: 4.0-beta) 4.0-alpha Since Version: 4.0-alpha Source Control Link: https://github.com/apache/cassandra/commit/39eb7db65fd45653fdece1087ba75c3356a10c97 Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks, committed with nits addressed. > Legacy counter shards can cause false positives in repaired data tracking > - > > Key: CASSANDRA-15461 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15461 > Project: Cassandra > Issue Type: Bug > Components: Observability/Metrics >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0-alpha > > > It is expected that the serialization of legacy (pre-2.1) counter cells may > differ across replicas due to the remote vs local designation of the shards. > This will cause the repaired data digests calculated at read time to differ > where certain legacy shards are encountered. This does not, however, indicate > corruption of the repaired dataset and there isn't any action that operators > can take in this scenario. Excluding counter cells which contain legacy > shards from the digest calculation will avoid false positives. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15518) build artifact download failed due to http
[ https://issues.apache.org/jira/browse/CASSANDRA-15518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-15518: Resolution: Invalid Status: Resolved (was: Triage Needed) As noted, this is fixed in 3.11.5 (it can also be worked around using a {{build.properties}} file if you have to build from earlier source versions). Backporting to 3.11.4 is not a thing. > build artifact download failed due to http > -- > > Key: CASSANDRA-15518 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15518 > Project: Cassandra > Issue Type: Bug > Components: Build >Reporter: Dev Leishangthem >Priority: Normal > > Please feel free to close if not apt. > The build for cassandra 3.11.4 is failing due to `http` usage in > build,properties.default and build.xml. > Maven2 Repository Locations (you can override these in "build.properties" to > point to a local proxy, e.g. Nexus) > artifact.remoteRepository.central:http://repo1.maven.org/maven2 > artifact.remoteRepository.apache: http://repo.maven.apache.org/maven2 > Changing it to `https` resolves the issue. Also seen that 3.11.5 tag has > https. > Can this be backported to 3.11.4 ? -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15392) Pool Merge Iterators
[ https://issues.apache.org/jira/browse/CASSANDRA-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022051#comment-17022051 ] Jordan West commented on CASSANDRA-15392: - {quote} As this relates to SASI, this wouldn’t be a regression right? I'd guess we're already wasting TrivialOneToOne iterators? {quote} Correct. This wouldn't break SASI but it would waste TrivialOneToOne's most of the time and rarely it would waste a ManyToOne that is oversized for its needs. I haven't looked at FQL but for SASI I think its easy enough to just skip using MergeIterators when they aren't needed / skip the pool when they are – happy to submit a patch if its helpful. > Pool Merge Iterators > > > Key: CASSANDRA-15392 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15392 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0 > > > By pooling merge iterators, instead of creating new ones each time we need > them, we can reduce garbage on the compaction and read paths under relevant > workloads by ~4% in many cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022172#comment-17022172 ] Alex Petrov commented on CASSANDRA-15505: - [~yifanc] [~dcapwell] thank you for feedback. Sorry for reworking the patch yet again; having two reviewers often brings many new ideas to the table, and this time it was particularly useful, so I had to implement the changes you two have suggested. I've implemented a new version, which incorporates most of your comments (with an exception of David's suggestion to use Instance instead of Int). A short list of changes: * 1 filter per matching condition * allow executing a part of matcher on the instance. This means that you'll be able to reconstruct `Message` on the instance and perform any necessary checks. Since anything that gets into this matcher would have to be serializable, you won't be able to use things like accumulators, countdown latches, atomic longs, etc, since they'll simply be copied. So for the matchers you execute on the node, you have to use two matchers: {{Matcher onInstance, Function onMain}}. This means you'll run the first part of the matcher on the instance, and the second one, with results of execution on the instance transferred to host class loader. * switch from Verb to int (once again, in a longer-term effort to make APIs not depending on Cassandra internals) * rolled back changes to {{matches/permit}} after David's suggestion to avoid confusion * added some more tests (including negative ones, and ones for {{off}}) * added {{pass}} alongside with {{drop}}, to allow passthrough filters that can be used for intercept purposes withouot filtering messages out. > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
Jordan West created CASSANDRA-15519: --- Summary: Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime Key: CASSANDRA-15519 URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 Project: Cassandra Issue Type: Improvement Components: Local/Config, Messaging/Client Reporter: Jordan West Assignee: Jordan West CASSANDRA-15013 added configurable global and per endpoint limits on the number of in flight requests, measured in bytes. During times when the cluster is overloaded, it can be useful to change this setting without having to restart the Cassandra process. Changing the limits should affect all existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15519: Change Category: Operability Complexity: Normal Fix Version/s: 4.0-beta Status: Open (was: Triage Needed) > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta > > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-15519: --- Labels: pull-request-available (was: ) > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta > > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15519: Test and Documentation Plan: Added unit test and manually tested JMX modifications via jconsole Status: Patch Available (was: Open) A couple notes for reviewers: * I used the BiConsumer (remover) pattern in EndpointPayloadTracker to make the code more testable but if there is concern about the memory overhead I can change it * Two concurrent calls to setEndpointLimit could leave the node in a state where some endpoints have one limit and some have the other. This would require human’s to make JMX calls concurrently so I didn’t see much value in adding more concurrency control (especially since the fix is to just set the limit one more time w/ human coordination). Open to revisiting though. > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022232#comment-17022232 ] Jordan West commented on CASSANDRA-15519: - Also looks like I broke a few unit tests w/ the MBean registration. Will fix. > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12995) update hppc dependency to 0.7
[ https://issues.apache.org/jira/browse/CASSANDRA-12995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022309#comment-17022309 ] Jeremiah Jordan commented on CASSANDRA-12995: - I don’t know that we want to be changing this is any version besides 4.0, unless there is a big security flaw or something being addressed. We usually do not change dependencies in released versions unless there is a bug, that we are vulnerable to/affected by, being addressed by the upgrade. > update hppc dependency to 0.7 > - > > Key: CASSANDRA-12995 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12995 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies, Packaging >Reporter: Tomas Repik >Priority: Normal > Labels: easyfix > Fix For: 4.0 > > Attachments: cassandra-3.11.0-hppc.patch > > > Cassandra 3.11.0 is about to be included in Fedora. There are some tweaks to > the sources we need to do in order to successfully build it. Cassandra > depends on hppc 0.5.4, but In Fedora we have the newer version 0.7.1 Upstream > released even newer version 0.7.2. I attached a patch updating cassandra > sources to depend on the 0.7.1 hppc sources. It should be also compatible > with the newest upstream version. The only actual changes are the removal of > Open infix in class names. The issue was discussed in here: > https://bugzilla.redhat.com/show_bug.cgi?id=1340876 Please consider updating. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022406#comment-17022406 ] Chris Lohfink commented on CASSANDRA-15519: --- minor thing, but could you move the setter into StorageProxyMBean with a more meaningful name? like {{setNativeTransportMaxConcurrentRequestBytes}} maybe next to{{ setNativeTransportMaxConcurrentConnections}} > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022406#comment-17022406 ] Chris Lohfink edited comment on CASSANDRA-15519 at 1/23/20 7:26 PM: minor thing, but could you move the setter into StorageProxyMBean with a more meaningful name? like {{setNativeTransportMaxConcurrentRequestBytes}} maybe next to {{setNativeTransportMaxConcurrentConnections}} was (Author: clohfink): minor thing, but could you move the setter into StorageProxyMBean with a more meaningful name? like {{setNativeTransportMaxConcurrentRequestBytes}} maybe next to{{ setNativeTransportMaxConcurrentConnections}} > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12995) update hppc dependency to 0.7
[ https://issues.apache.org/jira/browse/CASSANDRA-12995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022429#comment-17022429 ] Tomo Suzuki commented on CASSANDRA-12995: - This ticket is asking for the upgrade for the sake of compatibility with other libraries, not security or vulnerability. As Cassandra 3.X keeps using classes and methods that are not available in newer versions of popular libraries (such as hpcc and Guava), Cassandra 3.X users have hard time to run it with other libraries and frameworks, as well as the developers who would like to integrate their products with Cassandra. > update hppc dependency to 0.7 > - > > Key: CASSANDRA-12995 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12995 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies, Packaging >Reporter: Tomas Repik >Priority: Normal > Labels: easyfix > Fix For: 4.0 > > Attachments: cassandra-3.11.0-hppc.patch > > > Cassandra 3.11.0 is about to be included in Fedora. There are some tweaks to > the sources we need to do in order to successfully build it. Cassandra > depends on hppc 0.5.4, but In Fedora we have the newer version 0.7.1 Upstream > released even newer version 0.7.2. I attached a patch updating cassandra > sources to depend on the 0.7.1 hppc sources. It should be also compatible > with the newest upstream version. The only actual changes are the removal of > Open infix in class names. The issue was discussed in here: > https://bugzilla.redhat.com/show_bug.cgi?id=1340876 Please consider updating. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Remove joda time dependency.
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 2dbae87 Remove joda time dependency. 2dbae87 is described below commit 2dbae87c781b994e0c03ad6aa9e54118ddeddfde Author: Boris Tsirkin AuthorDate: Sat Aug 3 23:39:45 2019 +0200 Remove joda time dependency. Patch by Boris Tsikin, reviewed by brandonwilliams for CASSANDRA-15257 --- CHANGES.txt| 1 + build.xml | 3 - ide/nbproject/project.xml | 2 +- lib/joda-time-2.4.jar | Bin 586201 -> 0 bytes .../serializers/SimpleDateSerializer.java | 76 --- .../cassandra/tools/SSTableMetadataViewer.java | 6 +- .../cassandra/cql3/functions/CastFctsTest.java | 29 +++--- .../cassandra/cql3/functions/TimeFctsTest.java | 106 +++-- .../serializers/SimpleDateSerializerTest.java | 5 +- 9 files changed, 100 insertions(+), 128 deletions(-) diff --git a/CHANGES.txt b/CHANGES.txt index 43126ed..ec056e2 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha3 + * Remove joda time dependency (CASSANDRA-15257) * Exclude purgeable tombstones from repaired data tracking (CASSANDRA-15462) * Exclude legacy counter shards from repaired data tracking (CASSANDRA-15461) * Make it easier to add trace headers to messages (CASSANDRA-15499) diff --git a/build.xml b/build.xml index 4de9285..0c95687 100644 --- a/build.xml +++ b/build.xml @@ -576,7 +576,6 @@ - @@ -709,7 +708,6 @@ - @@ -775,7 +773,6 @@ - diff --git a/ide/nbproject/project.xml b/ide/nbproject/project.xml index ece0d01..6b8a31e 100644 --- a/ide/nbproject/project.xml +++ b/ide/nbproject/project.xml @@ -7,7 +7,7 @@ .. -${project.dir}/lib/HdrHistogram-2.1.9.jar:${project.dir}/lib/ST4-4.0.8.jar:${project.dir}/lib/airline-0.8.jar:${project.dir}/lib/antlr-runtime-3.5.2.jar:${project.dir}/lib/asm-7.1.jar:${project.dir}/lib/caffeine-2.3.5.jar:${project.dir}/lib/cassandra-driver-core-3.6.0-shaded.jar:${project.dir}/lib/chronicle-bytes-1.16.3.jar:${project.dir}/lib/chronicle-core-1.16.4.jar:${project.dir}/lib/chronicle-queue-4.16.3.jar:${project.dir}/li [...] +${project.dir}/lib/HdrHistogram-2.1.9.jar:${project.dir}/lib/ST4-4.0.8.jar:${project.dir}/lib/airline-0.8.jar:${project.dir}/lib/antlr-runtime-3.5.2.jar:${project.dir}/lib/asm-7.1.jar:${project.dir}/lib/caffeine-2.3.5.jar:${project.dir}/lib/cassandra-driver-core-3.6.0-shaded.jar:${project.dir}/lib/chronicle-bytes-1.16.3.jar:${project.dir}/lib/chronicle-core-1.16.4.jar:${project.dir}/lib/chronicle-queue-4.16.3.jar:${project.dir}/li [...] diff --git a/lib/joda-time-2.4.jar b/lib/joda-time-2.4.jar deleted file mode 100644 index ace67d7..000 Binary files a/lib/joda-time-2.4.jar and /dev/null differ diff --git a/src/java/org/apache/cassandra/serializers/SimpleDateSerializer.java b/src/java/org/apache/cassandra/serializers/SimpleDateSerializer.java index 075094c..fbb5087 100644 --- a/src/java/org/apache/cassandra/serializers/SimpleDateSerializer.java +++ b/src/java/org/apache/cassandra/serializers/SimpleDateSerializer.java @@ -18,22 +18,26 @@ package org.apache.cassandra.serializers; import java.nio.ByteBuffer; +import java.time.Duration; +import java.time.Instant; +import java.time.LocalDate; +import java.time.ZonedDateTime; +import java.time.format.DateTimeFormatter; +import java.time.format.DateTimeParseException; import java.util.concurrent.TimeUnit; import java.util.regex.Pattern; -import org.joda.time.DateTime; -import org.joda.time.DateTimeZone; -import org.joda.time.LocalDate; -import org.joda.time.format.DateTimeFormat; -import org.joda.time.format.DateTimeFormatter; - import org.apache.cassandra.utils.ByteBufferUtil; +import static java.time.ZoneOffset.UTC; +import static java.time.format.ResolverStyle.STRICT; + // For byte-order comparability, we shift by Integer.MIN_VALUE and treat the data as an unsigned integer ranging from // min date to max date w/epoch sitting in the center @ 2^31 public class SimpleDateSerializer implements TypeSerializer { -private static final DateTimeFormatter formatter = DateTimeFormat.forPattern("-MM-dd").withZone(DateTimeZone.UTC); +private static final DateTimeFormatter formatter = + DateTimeFormatter.ISO_LOCAL_DATE.withZone(UTC).withResolverStyle(STRICT); private static final long minSupportedDateMillis = TimeUnit.DAYS.toMillis(Integer.MIN_VALUE
[jira] [Updated] (CASSANDRA-15257) Remove joda time from dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15257: - Reviewers: Brandon Williams, Brandon Williams (was: Brandon Williams) Brandon Williams, Brandon Williams Status: Review In Progress (was: Patch Available) > Remove joda time from dependencies > -- > > Key: CASSANDRA-15257 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15257 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Boris Tsirkin >Assignee: Boris Tsirkin >Priority: Low > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Joda time is not supported anymore > {quote} > Note that Joda-Time is considered to be a largely “finished” project. No > major enhancements are planned. If using Java SE 8, please migrate to > {{java.time}} (JSR-310). > {quote} > See: https://www.joda.org/joda-time/ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15257) Remove joda time from dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15257: - Status: Ready to Commit (was: Review In Progress) > Remove joda time from dependencies > -- > > Key: CASSANDRA-15257 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15257 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Boris Tsirkin >Assignee: Boris Tsirkin >Priority: Low > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Joda time is not supported anymore > {quote} > Note that Joda-Time is considered to be a largely “finished” project. No > major enhancements are planned. If using Java SE 8, please migrate to > {{java.time}} (JSR-310). > {quote} > See: https://www.joda.org/joda-time/ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15257) Remove joda time from dependencies
[ https://issues.apache.org/jira/browse/CASSANDRA-15257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Williams updated CASSANDRA-15257: - Source Control Link: https://github.com/apache/cassandra/commit/2dbae87c781b994e0c03ad6aa9e54118ddeddfde Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed w/minor additional minor changes (removal from build/ide deps) > Remove joda time from dependencies > -- > > Key: CASSANDRA-15257 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15257 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Boris Tsirkin >Assignee: Boris Tsirkin >Priority: Low > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Joda time is not supported anymore > {quote} > Note that Joda-Time is considered to be a largely “finished” project. No > major enhancements are planned. If using Java SE 8, please migrate to > {{java.time}} (JSR-310). > {quote} > See: https://www.joda.org/joda-time/ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15379) Make it possible to flush with a different compression strategy than we compact with
[ https://issues.apache.org/jira/browse/CASSANDRA-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15379: - Status: Changes Suggested (was: Review In Progress) Hi [~jolynch], thanks for the patch. I went over it and it looks generally good. On a high level the only concern I have is introducing a {{NoOpCompressor}} may lead to some performance issues compared to our current state. This is mainly due to Java JIT's inability to optimize megamorphic call sites. However, I think this is just a theory and we should try and validate it using an actual performance test. IMHO, the advantages that you have laid out would outweight a bit of performance penalty. Other than that, I had some code related feedback. It fixes the {{DatabaseDescriptorRefTest}} and also makes minor structural modifications for safety and clarity. I have illustrated in my branch [here|https://github.com/apache/cassandra/compare/trunk...dineshjoshi:CASSANDRA-15379-review?expand=1]. Please feel free to cherry pick the commits in your branch. > Make it possible to flush with a different compression strategy than we > compact with > > > Key: CASSANDRA-15379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction, Local/Config, Local/Memtable >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0-alpha > > > [~josnyder] and I have been testing out CASSANDRA-14482 (Zstd compression) on > some of our most dense clusters and have been observing close to 50% > reduction in footprint with Zstd on some of our workloads! Unfortunately > though we have been running into an issue where the flush might take so long > (Zstd is slower to compress than LZ4) that we can actually block the next > flush and cause instability. > Internally we are working around this with a very simple patch which flushes > SSTables as the default compression strategy (LZ4) regardless of the table > params. This is a simple solution but I think the ideal solution though might > be for the flush compression strategy to be configurable separately from the > table compression strategy (while defaulting to the same thing). Instead of > adding yet another compression option to the yaml (like hints and commitlog) > I was thinking of just adding it to the table parameters and then adding a > {{default_table_parameters}} yaml option like: > {noformat} > # Default table properties to apply on freshly created tables. The currently > supported defaults are: > # * compression : How are SSTables compressed in general (flush, > compaction, etc ...) > # * flush_compression : How are SSTables compressed as they flush > # supported > default_table_parameters: > compression: > class_name: 'LZ4Compressor' > parameters: > chunk_length_in_kb: 16 > flush_compression: > class_name: 'LZ4Compressor' > parameters: > chunk_length_in_kb: 4 > {noformat} > This would have the nice effect as well of giving our configuration a path > forward to providing user specified defaults for table creation (so e.g. if a > particular user wanted to use a different default chunk_length_in_kb they can > do that). > So the proposed (~mandatory) scope is: > * Flush with a faster compression strategy > I'd like to implement the following at the same time: > * Per table flush compression configuration > * Ability to default the table flush and compaction compression in the yaml. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15379) Make it possible to flush with a different compression strategy than we compact with
[ https://issues.apache.org/jira/browse/CASSANDRA-15379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15379: - Reviewers: Dinesh Joshi, Dinesh Joshi (was: Dinesh Joshi) Dinesh Joshi, Dinesh Joshi (was: Dinesh Joshi) Status: Review In Progress (was: Patch Available) > Make it possible to flush with a different compression strategy than we > compact with > > > Key: CASSANDRA-15379 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15379 > Project: Cassandra > Issue Type: Improvement > Components: Local/Compaction, Local/Config, Local/Memtable >Reporter: Joey Lynch >Assignee: Joey Lynch >Priority: Normal > Fix For: 4.0-alpha > > > [~josnyder] and I have been testing out CASSANDRA-14482 (Zstd compression) on > some of our most dense clusters and have been observing close to 50% > reduction in footprint with Zstd on some of our workloads! Unfortunately > though we have been running into an issue where the flush might take so long > (Zstd is slower to compress than LZ4) that we can actually block the next > flush and cause instability. > Internally we are working around this with a very simple patch which flushes > SSTables as the default compression strategy (LZ4) regardless of the table > params. This is a simple solution but I think the ideal solution though might > be for the flush compression strategy to be configurable separately from the > table compression strategy (while defaulting to the same thing). Instead of > adding yet another compression option to the yaml (like hints and commitlog) > I was thinking of just adding it to the table parameters and then adding a > {{default_table_parameters}} yaml option like: > {noformat} > # Default table properties to apply on freshly created tables. The currently > supported defaults are: > # * compression : How are SSTables compressed in general (flush, > compaction, etc ...) > # * flush_compression : How are SSTables compressed as they flush > # supported > default_table_parameters: > compression: > class_name: 'LZ4Compressor' > parameters: > chunk_length_in_kb: 16 > flush_compression: > class_name: 'LZ4Compressor' > parameters: > chunk_length_in_kb: 4 > {noformat} > This would have the nice effect as well of giving our configuration a path > forward to providing user specified defaults for table creation (so e.g. if a > particular user wanted to use a different default chunk_length_in_kb they can > do that). > So the proposed (~mandatory) scope is: > * Flush with a faster compression strategy > I'd like to implement the following at the same time: > * Per table flush compression configuration > * Ability to default the table flush and compaction compression in the yaml. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15392) Pool Merge Iterators
[ https://issues.apache.org/jira/browse/CASSANDRA-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022492#comment-17022492 ] Blake Eggleston commented on CASSANDRA-15392: - [~jrwest] sure, that would be great. Tag me for review if you'd like. > Pool Merge Iterators > > > Key: CASSANDRA-15392 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15392 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0 > > > By pooling merge iterators, instead of creating new ones each time we need > them, we can reduce garbage on the compaction and read paths under relevant > workloads by ~4% in many cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15475) Full Query Logging - New Feature
[ https://issues.apache.org/jira/browse/CASSANDRA-15475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15475: - Reviewers: Dinesh Joshi, Yifan Cai (was: Dinesh Joshi) > Full Query Logging - New Feature > > > Key: CASSANDRA-15475 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15475 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on Full Query logging, a new feature. > https://github.com/apache/cassandra/pull/404 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15478) Improved Messaging
[ https://issues.apache.org/jira/browse/CASSANDRA-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15478: - Reviewers: Dinesh Joshi, Joey Lynch (was: Dinesh Joshi) > Improved Messaging > --- > > Key: CASSANDRA-15478 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15478 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on improved messaging. > https://github.com/apache/cassandra/pull/407 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15478) Improved Messaging
[ https://issues.apache.org/jira/browse/CASSANDRA-15478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15478: - Reviewers: Joey Lynch (was: Dinesh Joshi, Joey Lynch) > Improved Messaging > --- > > Key: CASSANDRA-15478 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15478 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on improved messaging. > https://github.com/apache/cassandra/pull/407 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15484) Read Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15484: - Reviewers: Blake Eggleston, Joey Lynch (was: Dinesh Joshi) > Read Repair > --- > > Key: CASSANDRA-15484 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15484 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Fix For: 4.0 > > > Added page on Read Repair. > https://github.com/apache/cassandra/pull/413 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15484) Read Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022566#comment-17022566 ] Dinesh Joshi commented on CASSANDRA-15484: -- [~jolynch] [~bdeggleston] could you please help review this? > Read Repair > --- > > Key: CASSANDRA-15484 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15484 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Fix For: 4.0 > > > Added page on Read Repair. > https://github.com/apache/cassandra/pull/413 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15486) Dynamo
[ https://issues.apache.org/jira/browse/CASSANDRA-15486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15486: - Reviewers: Joey Lynch (was: Dinesh Joshi) > Dynamo > -- > > Key: CASSANDRA-15486 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15486 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added new sections to Dynamo page. > https://github.com/apache/cassandra/pull/415 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15488) Configuration file
[ https://issues.apache.org/jira/browse/CASSANDRA-15488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15488: - Reviewers: Joey Lynch (was: Dinesh Joshi) > Configuration file > -- > > Key: CASSANDRA-15488 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15488 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Fix For: 4.0 > > > Added configuration file page. > https://github.com/apache/cassandra/pull/417 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15492) DDL update
[ https://issues.apache.org/jira/browse/CASSANDRA-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15492: - Reviewers: Joey Lynch (was: Dinesh Joshi) > DDL update > -- > > Key: CASSANDRA-15492 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15492 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > - Added more detail and examples to some sections such as Speculative retry > - Added table option 'cdc' > https://github.com/apache/cassandra/pull/420 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15482) Guarantees
[ https://issues.apache.org/jira/browse/CASSANDRA-15482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15482: - Reviewers: Vinay Chella (was: Dinesh Joshi) > Guarantees > -- > > Key: CASSANDRA-15482 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15482 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > Fix For: 4.0 > > > Added a page on Guarantees. > https://github.com/apache/cassandra/pull/411 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[ https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-15470: - Status: Ready to Commit (was: Changes Suggested) +1 > Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB > & Bytes > - > > Key: CASSANDRA-15470 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15470 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Jordan West >Assignee: Mallika Kulkarni >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-rc > > Time Spent: 10m > Remaining Estimate: 0h > > {{DatabaseDescriptor}} has several functions that convert between user > supplied sizes in KB/MB and bytes. These are implemented without much > consistency and, while unlikely, several have the potential to overflow since > validation on the input is missing. Meanwhile, some widen the number to a > long correctly. Options include: widening in all places or simply doing > better validation on start up — currently only the lower bound of the valid > range is checked for many of these fields. > List of Affected {{DatabaseDescriptor}} Methods: > * {{getColumnIndexSize}} > * {{getColumnIndexCacheSize}} > * {{getBatchSizeWarnThreshold}} > * {{getNativeTransportFrameBlockSize}} > * {{getRepairSessionSpaceInMegabytes}} > * {{getNativeTransportMaxFrameSize}} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022582#comment-17022582 ] David Capwell commented on CASSANDRA-15508: --- Sure, I'll provide a set of patches. I have a feeling 3.0 will start to fail... lets see! > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022591#comment-17022591 ] David Capwell commented on CASSANDRA-15505: --- Ok, [~ifesdjeen] and I talked about this... so putting it here for history. There exists a desire to accumulate messages and process them later in the test {code} ThreadSafeList accum cluster.filters().from(1, 2).to(3).filter((_, _, msg) -> { accum.add(msg.payload.decoratedKey) return true; }) {code} Here msg is net.Message, but this doesn't work because of mixing class loaders. {code} ThreadSafeList accum cluster.filters().from(1, 2).to(3).filter((_, _, msg) -> { accum.add(deserialize(msg).payload.decoratedKey) return true; }) {code} In this example msg is dtest Message and the filter is still transferred. This also fails because mixing class loaders (DecoratedKey exists in test class loader, and again for each instance) {code} cluster.filters().from(1, 2).to(3).filter((_, _, msg) -> { Assert.assertTrue(, deserialize(msg).payload.decoratedKey) return true; }).pass() {code} There was also a desire to assert a specific state when a message happens. This doesn't work since the call site is MessagingService, so the assert will cause a exception thrown in the instance and not the test method. The above needs to be rewrite to the following to work {code} AtomicReference epicFail cluster.filters.allVerbs().from(1).to(2).filter(msg -> { epicFail.set(deserialize(msg).header.contains("totally doesn't compile but you get the point")) }) ... epicFail == null // didn't see epicFail == true // assert {code} After all this, we finalized that the below is desirable {code} CountDownLatch latch = new CountDownLatch(1); cluster.filters.allVerbs().from(1).to(2).filter(msg -> { // THIS IS EXECUTED ON INSTANCE latch.countDown(); return true; }) {code} This works only if the filter is not "transferred" to the instance and stays in the test class loader (transfer clones where as lambda "captures"; aka uses references rather than copies) If you need to transfer, you can selectively do it {code} IMessageFilters.Filter filter = cluster.filters() .allVerbs() .from(1) .to(2) .messagesMatching((from, to, msg) -> { cluster.get(from).runOnInstance(() -> { try { DataInputPlus.DataInputStreamPlus stream = new DataInputPlus.DataInputStreamPlus(new ByteArrayInputStream(msg.bytes())); Message instance = (Message) Verb.values()[msg.id()].serializer().deserialize(stream, to); System.out.println(instance.header.createdAtNanos); } catch (Exception e) { throw new RuntimeException(e); } }); return true; }).pass(); {code} Given all of the above, we settled on the following * drop the 2 function method (on instance, off instance) * matcher should have the signature (int, int, IMessage) -> Boolean (this is optional) * don't transfer the matcher > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022599#comment-17022599 ] David Capwell commented on CASSANDRA-15508: --- Ok, working on 2.2 and noticed that generate.sh causes a lot of changes... looking at the history it seems that not everything was controlled by .circleci/config-2_1.yml ill see about making this clear in the history. > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022599#comment-17022599 ] David Capwell edited comment on CASSANDRA-15508 at 1/24/20 12:14 AM: - Ok, working on 2.2 and noticed that generate.sh causes a lot of changes... looking at the history it seems that not everything was controlled by .circleci/config-2_1.yml ill see about making this clear in the history. smithed to 2.2 without my change, ran generate.sh and get the following {code} $ git diff | wc -l 1072 {code} this is 0 on trunk. was (Author: dcapwell): Ok, working on 2.2 and noticed that generate.sh causes a lot of changes... looking at the history it seems that not everything was controlled by .circleci/config-2_1.yml ill see about making this clear in the history. > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: fix SimpleDateTypeTest
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 2f2b2ae fix SimpleDateTypeTest 2f2b2ae is described below commit 2f2b2ae607b2531836c805da913ca2e545162275 Author: Brandon Williams AuthorDate: Thu Jan 23 18:19:12 2020 -0600 fix SimpleDateTypeTest --- .../apache/cassandra/db/marshal/SimpleDateTypeTest.java| 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/test/unit/org/apache/cassandra/db/marshal/SimpleDateTypeTest.java b/test/unit/org/apache/cassandra/db/marshal/SimpleDateTypeTest.java index 5c9ed4e..f45c55f 100644 --- a/test/unit/org/apache/cassandra/db/marshal/SimpleDateTypeTest.java +++ b/test/unit/org/apache/cassandra/db/marshal/SimpleDateTypeTest.java @@ -95,29 +95,29 @@ public class SimpleDateTypeTest SimpleDateSerializer.instance.deserialize(d1), SimpleDateSerializer.instance.deserialize(d2)); -d1 = SimpleDateType.instance.fromString("-1-10-10"); -d2 = SimpleDateType.instance.fromString("1-10-10"); +d1 = SimpleDateType.instance.fromString("-2200-10-10"); +d2 = SimpleDateType.instance.fromString("2200-10-10"); assert SimpleDateType.instance.compare(d1, d2) < 0 : String.format("Failed neg/pos string comparison with %s and %s", SimpleDateSerializer.instance.deserialize(d1), SimpleDateSerializer.instance.deserialize(d2)); d1 = SimpleDateType.instance.fromString("1969-12-31"); -d2 = SimpleDateType.instance.fromString("1970-1-1"); +d2 = SimpleDateType.instance.fromString("1970-01-01"); assert SimpleDateType.instance.compare(d1, d2) < 0 : String.format("Failed pre/post epoch comparison with %s and %s", SimpleDateSerializer.instance.deserialize(d1), SimpleDateSerializer.instance.deserialize(d2)); -d1 = SimpleDateType.instance.fromString("1970-1-1"); -d2 = SimpleDateType.instance.fromString("1970-1-1"); +d1 = SimpleDateType.instance.fromString("1970-01-01"); +d2 = SimpleDateType.instance.fromString("1970-01-01"); assert SimpleDateType.instance.compare(d1, d2) == 0 : String.format("Failed == date from string comparison with %s and %s", SimpleDateSerializer.instance.deserialize(d1), SimpleDateSerializer.instance.deserialize(d2)); -d1 = SimpleDateType.instance.fromString("1970-1-1"); -d2 = SimpleDateType.instance.fromString("1970-1-2"); +d1 = SimpleDateType.instance.fromString("1970-01-01"); +d2 = SimpleDateType.instance.fromString("1970-01-02"); assert SimpleDateType.instance.compare(d1, d2) < 0 : String.format("Failed post epoch string comparison with %s and %s", SimpleDateSerializer.instance.deserialize(d1), - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022618#comment-17022618 ] Dinesh Joshi commented on CASSANDRA-10190: -- Thanks [~andrew.tolbert]. I still see a [few failures|https://circleci.com/workflow-run/34346de3-a7fd-47de-8c15-ce17c0c105ad] but they're looking better than before. I will try to debug them. In the meanwhile if you have any fixes in the pipeline, please post them here. > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-10190: - Status: Review In Progress (was: Changes Suggested) > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022623#comment-17022623 ] David Capwell edited comment on CASSANDRA-15508 at 1/24/20 1:02 AM: 2.2 src: https://github.com/apache/cassandra/pull/430 - 3 commits so its clear what changed what. tests: https://circleci.com/gh/dcapwell/cassandra/tree/cleanHistory-2.2 was (Author: dcapwell): https://github.com/apache/cassandra/pull/430 is 3 commits so its clear what changed what. > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 40m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022623#comment-17022623 ] David Capwell commented on CASSANDRA-15508: --- https://github.com/apache/cassandra/pull/430 is 3 commits so its clear what changed what. > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 40m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15520) split circle ci commands into reusable scripts which can be used outside of circle ci
[ https://issues.apache.org/jira/browse/CASSANDRA-15520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-15520: -- Change Category: Code Clarity Complexity: Low Hanging Fruit Status: Open (was: Triage Needed) > split circle ci commands into reusable scripts which can be used outside of > circle ci > - > > Key: CASSANDRA-15520 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15520 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > CircleCI is one of the main tools we use for build and test, but there is > also ASF Jenkins and many people run builds in their own companies as well. > It would be nice to refactor the existing CircleCI yaml to delegate to a set > of scripts which could be reused by other build systems. > I feel that we could do the following directory layout > {code} > ci - top level directory containing all scripts > - - directory containing the different build > steps > - - a single build with > the required steps to run it > - split.sh - script which takes in a output file to write to and > dumps out all test cases (not partitioned) > - run_partition.sh - script which takes a partitioned list of tests > and executes the build (does not move around artifacts) > {code} > This would allow CircleCI and Jenkins to run the same way, but also acts as > documentation for how to run some of the tests (jvm-dtest and python dtest > upgrade tests take more time to figure out how to run). > CircleCI would also be simpler as it would mostly be the circle ci specific > logic (partition tests, move code/test results around, etc.) calling these > scripts. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15520) split circle ci commands into reusable scripts which can be used outside of circle ci
David Capwell created CASSANDRA-15520: - Summary: split circle ci commands into reusable scripts which can be used outside of circle ci Key: CASSANDRA-15520 URL: https://issues.apache.org/jira/browse/CASSANDRA-15520 Project: Cassandra Issue Type: Improvement Components: Build Reporter: David Capwell Assignee: David Capwell CircleCI is one of the main tools we use for build and test, but there is also ASF Jenkins and many people run builds in their own companies as well. It would be nice to refactor the existing CircleCI yaml to delegate to a set of scripts which could be reused by other build systems. I feel that we could do the following directory layout {code} ci - top level directory containing all scripts - - directory containing the different build steps - - a single build with the required steps to run it - split.sh - script which takes in a output file to write to and dumps out all test cases (not partitioned) - run_partition.sh - script which takes a partitioned list of tests and executes the build (does not move around artifacts) {code} This would allow CircleCI and Jenkins to run the same way, but also acts as documentation for how to run some of the tests (jvm-dtest and python dtest upgrade tests take more time to figure out how to run). CircleCI would also be simpler as it would mostly be the circle ci specific logic (partition tests, move code/test results around, etc.) calling these scripts. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022633#comment-17022633 ] David Capwell commented on CASSANDRA-15508: --- 3.0 src: https://github.com/apache/cassandra/pull/431 - 2 commits, same reason tests: https://circleci.com/gh/dcapwell/cassandra/tree/cleanHistory-3.0 > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 50m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022638#comment-17022638 ] David Capwell commented on CASSANDRA-15508: --- 3.11 is missing things... {code} $ git diff trunk config-2_1.yml | wc -l 396 {code} Ill just rewrite for this branch since patch doesn't work =( > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 50m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022637#comment-17022637 ] Jordan West commented on CASSANDRA-15519: - [~clohfink] thanks for the input. Could you help me understand a bit better the value in moving the methods (I can certainly make them more meaningful in either place)? The reason I ask is because from my (limited) perspective its pretty much the same difficultly to change the values regardless of the MBean they live in and discovery is a matter of documentation (which I can add as part of this patch). I'm open to making the change if your experience tells you it will be much more valuable – my hesitation is that it exposes either Server to StorageProxy or visa-versa when currently there is a pretty clean delineation of layers there. > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15519) Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at runtime
[ https://issues.apache.org/jira/browse/CASSANDRA-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022647#comment-17022647 ] Jordan West commented on CASSANDRA-15519: - Fixed the [tests|https://circleci.com/gh/jrwest/cassandra/tree/jwest%2F15519] > Make native_transport_max_concurrent_requests_in_bytes(_per_ip) changeable at > runtime > - > > Key: CASSANDRA-15519 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15519 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config, Messaging/Client >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-beta > > Time Spent: 10m > Remaining Estimate: 0h > > CASSANDRA-15013 added configurable global and per endpoint limits on the > number of in flight requests, measured in bytes. During times when the > cluster is overloaded, it can be useful to change this setting without having > to restart the Cassandra process. Changing the limits should affect all > existing and new connections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12995) update hppc dependency to 0.7
[ https://issues.apache.org/jira/browse/CASSANDRA-12995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022665#comment-17022665 ] Ekaterina Dimitrova commented on CASSANDRA-12995: - [~suztomo] unfortunately, 3.11 is only bug fixing and 4.0 is in freeze so the only thing I can do in the current situation will be to ensure we have the new versions in v.4 as [~jjordan] suggested already. I will work on this tomorrow/Monday. For reference, please, check "Choosing the Right Branches to Work On" [here|http://cassandra.apache.org/doc/latest/development/patches.html?highlight=contribution] > update hppc dependency to 0.7 > - > > Key: CASSANDRA-12995 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12995 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies, Packaging >Reporter: Tomas Repik >Priority: Normal > Labels: easyfix > Fix For: 4.0 > > Attachments: cassandra-3.11.0-hppc.patch > > > Cassandra 3.11.0 is about to be included in Fedora. There are some tweaks to > the sources we need to do in order to successfully build it. Cassandra > depends on hppc 0.5.4, but In Fedora we have the newer version 0.7.1 Upstream > released even newer version 0.7.2. I attached a patch updating cassandra > sources to depend on the 0.7.1 hppc sources. It should be also compatible > with the newest upstream version. The only actual changes are the removal of > Open infix in class names. The issue was discussed in here: > https://bugzilla.redhat.com/show_bug.cgi?id=1340876 Please consider updating. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022666#comment-17022666 ] David Capwell commented on CASSANDRA-15508: --- ok, I have something which compiles for 3.11 3.11 src: https://github.com/dcapwell/cassandra/pull/new/cleanHistory-3.11 tests: https://circleci.com/gh/dcapwell/cassandra/tree/cleanHistory-3.11 > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 50m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15209) nodetool tablestats indexes name is written twice
[ https://issues.apache.org/jira/browse/CASSANDRA-15209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022669#comment-17022669 ] Ekaterina Dimitrova commented on CASSANDRA-15209: - [~l...@lapo.it], can you, please, provide some more information? Running a couple of simple tests didn't reproduce the issue. Please specify Cassandra version, maybe schema? Indexes? Thanks in advance! > nodetool tablestats indexes name is written twice > - > > Key: CASSANDRA-15209 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15209 > Project: Cassandra > Issue Type: Bug > Components: Tool/nodetool >Reporter: Lapo Luchini >Assignee: Ekaterina Dimitrova >Priority: Normal > > Tables names are printed OK, but index names always seems to be printed twice > in a row: > {code} > # nodetool tablestats -H | fgrep '(index)' > Table (index): fs_content.fsbyhashfs_content.fsbyhash > Table (index): documents.document_statusdocuments.document_status > Table (index): documents.document_statusdocuments.document_status > Table (index): certificates.certificates_cfcertificates.certificates_cf > Table (index): documents.document_id_dossierdocuments.document_id_dossier > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022670#comment-17022670 ] David Capwell commented on CASSANDRA-15508: --- hold off, the builds (though green) look off. ill say when done. > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 50m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15521) Update default for num_tokens from 256 to something more reasonable
Jeremy Hanna created CASSANDRA-15521: Summary: Update default for num_tokens from 256 to something more reasonable Key: CASSANDRA-15521 URL: https://issues.apache.org/jira/browse/CASSANDRA-15521 Project: Cassandra Issue Type: Improvement Components: Feature/Virtual Nodes Reporter: Jeremy Hanna Assignee: Jeremy Hanna The default for num_tokens or the number of token ranges assigned to a node using virtual nodes is way too high. 256 token ranges makes repair painful. Since it's a default, someone new to Cassandra won't know better and will have to live with it or perform a migration to a new datacenter with a lower number. At the same time, going too low with the default allocation algorithm can hotspot nodes to have more tokens assigned than others. There is a new token allocation algorithm introduced but it's not default. The proposal of this ticket is to set the default to something more reasonable to align with best practices without using the new token algorithm or giving it specific token values as some do. 32 is a good compromise and is what the project uses in a lot of the tests that are done. So generally it would be good to move to a more sane value and to align with testing so users are more confident that the defaults have a lot of testing behind them. As discussed on the dev mailing list, we want to make sure this change to the default doesn't come as an unpleasant surprise to cluster operators. For num_tokens specifically, if you were to upgrade to a version with the new default and the user didn't change it to the existing value, the node would not start, saying you can't change the num_tokens on an existing node. So we will want to put a release note to indicate that when upgrading, make a note of the num_tokens change when looking at the new configuration. Along with not being able to start nodes, which is fail-fast, there is the matter of adding new nodes to the cluster. You can certainly add a new node to a cluster or datacenter with a different number of token ranges assigned. It will give that node a different amount of data to be responsible for. For example, if the nodes in a datacenter all have num_tokens=256 (current default) and you add a node to that datacenter with num_tokens=32 (new default), it will only claim 1/8th of the token ranges and data as the other nodes in that datacenter. Fortunately, this is a property that is explicitly defined rather than implicit like some of the table settings. Also most if not all operators will upgrade the existing nodes to that new version before trying to add a node with that new version. So if there is a different number for num_tokens on the existing nodes, they'll be aware of it immediately. In any case, this is a long proposal for what will be a small change in the cassandra.yaml and something in the release notes, that is, changing the default num_tokens value from 256 to 32. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15521) Update default for num_tokens from 256 to something more reasonable
[ https://issues.apache.org/jira/browse/CASSANDRA-15521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-15521: - Description: The default for num_tokens or the number of token ranges assigned to a node using virtual nodes is way too high. 256 token ranges makes repair painful. Since it's a default, someone new to Cassandra won't know better and if left unchanged, they will have to live with it or perform a migration to a new datacenter with a lower number. At the same time, going too low with the default allocation algorithm can hotspot nodes to have more tokens assigned than others. There is a new token allocation algorithm introduced but it's not default. The proposal of this ticket is to set the default to something more reasonable to align with best practices without using the new token algorithm or giving it specific token values as some do. 32 is a good compromise and is what the project uses in a lot of the tests that are done. So generally it would be good to move to a more sane value and to align with testing so users are more confident that the defaults have a lot of testing behind them. As discussed on the dev mailing list, we want to make sure this change to the default doesn't come as an unpleasant surprise to cluster operators. For num_tokens specifically, if you were to upgrade to a version with the new default and the user didn't change it to the existing value, the node would not start, saying you can't change the num_tokens on an existing node. So we will want to put a release note to indicate that when upgrading, make a note of the num_tokens change when looking at the new configuration. Along with not being able to start nodes, which is fail-fast, there is the matter of adding new nodes to the cluster. You can certainly add a new node to a cluster or datacenter with a different number of token ranges assigned. It will give that node a different amount of data to be responsible for. For example, if the nodes in a datacenter all have num_tokens=256 (current default) and you add a node to that datacenter with num_tokens=32 (new default), it will only claim 1/8th of the token ranges and data as the other nodes in that datacenter. Fortunately, this is a property that is explicitly defined rather than implicit like some of the table settings. Also most if not all operators will upgrade the existing nodes to that new version before trying to add a node with that new version. So if there is a different number for num_tokens on the existing nodes, they'll be aware of it immediately. In any case, this is a long proposal for what will be a small change in the cassandra.yaml and something in the release notes, that is, changing the default num_tokens value from 256 to 32. was: The default for num_tokens or the number of token ranges assigned to a node using virtual nodes is way too high. 256 token ranges makes repair painful. Since it's a default, someone new to Cassandra won't know better and will have to live with it or perform a migration to a new datacenter with a lower number. At the same time, going too low with the default allocation algorithm can hotspot nodes to have more tokens assigned than others. There is a new token allocation algorithm introduced but it's not default. The proposal of this ticket is to set the default to something more reasonable to align with best practices without using the new token algorithm or giving it specific token values as some do. 32 is a good compromise and is what the project uses in a lot of the tests that are done. So generally it would be good to move to a more sane value and to align with testing so users are more confident that the defaults have a lot of testing behind them. As discussed on the dev mailing list, we want to make sure this change to the default doesn't come as an unpleasant surprise to cluster operators. For num_tokens specifically, if you were to upgrade to a version with the new default and the user didn't change it to the existing value, the node would not start, saying you can't change the num_tokens on an existing node. So we will want to put a release note to indicate that when upgrading, make a note of the num_tokens change when looking at the new configuration. Along with not being able to start nodes, which is fail-fast, there is the matter of adding new nodes to the cluster. You can certainly add a new node to a cluster or datacenter with a different number of token ranges assigned. It will give that node a different amount of data to be responsible for. For example, if the nodes in a datacenter all have num_tokens=256 (current default) and you add a node to that datacenter with num_tokens=32 (new default), it will only claim 1/8th of the token ranges and data as the other nodes in that datacenter. Fortunately, this is a property that is
[jira] [Created] (CASSANDRA-15522) Update defaults for the validity timeframe of roles, permissions, and credentials
Jeremy Hanna created CASSANDRA-15522: Summary: Update defaults for the validity timeframe of roles, permissions, and credentials Key: CASSANDRA-15522 URL: https://issues.apache.org/jira/browse/CASSANDRA-15522 Project: Cassandra Issue Type: Improvement Components: Feature/Authorization Reporter: Jeremy Hanna It's been found that the defaults for \{roles_validity_in_ms}, \{permissions_validity_in_ms}, and \{credentials_validity_in_ms} have been too low at 2000 ms or 2 seconds each. As [~alexott] put it in the dev list discussion about defaults: {quote} I have seen multiple times when authentication was failing under the heavy load because queries to system tables were timing out - with these defaults people may still have the possibility to get updates to roles/credentials faster when specifying _update_interval_ variants of these configurations. {quote} The suggestion is to set it to 6 (1 minute) or 12 (2 minutes). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15522) Update defaults for the validity timeframe of roles, permissions, and credentials
[ https://issues.apache.org/jira/browse/CASSANDRA-15522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-15522: - Description: It's been found that the defaults for \{{roles_validity_in_ms}}, \{{permissions_validity_in_ms}}, and \{{credentials_validity_in_ms}} have been too low at 2000 ms or 2 seconds each. As [~alexott] put it in the dev list discussion about defaults: {quote}I have seen multiple times when authentication was failing under the heavy load because queries to system tables were timing out - with these defaults people may still have the possibility to get updates to roles/credentials faster when specifying _update_interval_ variants of these configurations. {quote} The suggestion is to set it to 6 (1 minute) or 12 (2 minutes). was: It's been found that the defaults for \{roles_validity_in_ms}, \{permissions_validity_in_ms}, and \{credentials_validity_in_ms} have been too low at 2000 ms or 2 seconds each. As [~alexott] put it in the dev list discussion about defaults: {quote} I have seen multiple times when authentication was failing under the heavy load because queries to system tables were timing out - with these defaults people may still have the possibility to get updates to roles/credentials faster when specifying _update_interval_ variants of these configurations. {quote} The suggestion is to set it to 6 (1 minute) or 12 (2 minutes). > Update defaults for the validity timeframe of roles, permissions, and > credentials > - > > Key: CASSANDRA-15522 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15522 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Jeremy Hanna >Priority: Normal > > It's been found that the defaults for \{{roles_validity_in_ms}}, > \{{permissions_validity_in_ms}}, and \{{credentials_validity_in_ms}} have > been too low at 2000 ms or 2 seconds each. As [~alexott] put it in the dev > list discussion about defaults: > {quote}I have seen multiple times when authentication was failing under the > heavy load because queries to system tables were timing out - with these > defaults people may still have the possibility to get updates to > roles/credentials faster when specifying _update_interval_ variants of these > configurations. > {quote} > The suggestion is to set it to 6 (1 minute) or 12 (2 minutes). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[ https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022672#comment-17022672 ] David Capwell commented on CASSANDRA-15508: --- ok, so for some reason circle ci put all my jobs in "not running" state even though the pipeline is running (and nothing else is running); given this can't validate in the UI. Looking at the YAML, all the branches look ok. [~marcuse] should be good. > Failing jvm dtest: FailingRepairTest.testFailingMessage > --- > > Key: CASSANDRA-15508 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15508 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: Marcus Eriksson >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png > > Time Spent: 50m > Remaining Estimate: 0h > > It seems we can't run parameterized unit tests with {{ant testsome}}: > {code} > $ ant testsome > -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest > -Dtest.methods=testFailingMessage > > [junit-timeout] Testcase: > initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR > [junit-timeout] No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] java.lang.Exception: No tests found matching Method > testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest) > from org.junit.internal.requests.ClassRequest@4d95d2a2 > [junit-timeout] at > java.lang.reflect.Constructor.newInstance(Constructor.java:423) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15523) Update default snitch from SimpleSnitch to GossipingPropertyFileSnitch
Jeremy Hanna created CASSANDRA-15523: Summary: Update default snitch from SimpleSnitch to GossipingPropertyFileSnitch Key: CASSANDRA-15523 URL: https://issues.apache.org/jira/browse/CASSANDRA-15523 Project: Cassandra Issue Type: Improvement Components: Local/Config Reporter: Jeremy Hanna Traditionally the project has had {{SimpleSnitch}} as the default primarily because it makes it easy for a user to download the software and start Cassandra without changing any defaults and it will just work. However, {{SimpleSnitch}} is not datacenter aware. {{GossipingPropertyFileSnitch}} could also be used as the default and would make the default snitch be datacenter aware. The user simply needs to update the local datacenter and rack names on each node and when it joins the cluster, it will propagate that topology information around the ring. This ticket would be to update the default from {{SimpleSnitch}} to {{GossipingPropertyFileSnitch}} to make the onboarding experience better for those who don't know to change this default if they intend to expand to multiple datacenters without too much hassle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15523) Update default snitch from SimpleSnitch to GossipingPropertyFileSnitch
[ https://issues.apache.org/jira/browse/CASSANDRA-15523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Hanna updated CASSANDRA-15523: - Description: Traditionally the project has had {{SimpleSnitch}} as the default primarily because it makes it easy for a user to download the software and start Cassandra without changing any defaults and it will just work. However, {{SimpleSnitch}} is not datacenter aware. {{GossipingPropertyFileSnitch}} could also be used as the default and would make the default snitch be datacenter aware. Out of the box it will work with the default configuration. This ticket would be to update the default from {{SimpleSnitch}} to {{GossipingPropertyFileSnitch}} to make the onboarding experience better for those who don't know to change this default if they intend to expand to multiple datacenters without too much hassle. was: Traditionally the project has had {{SimpleSnitch}} as the default primarily because it makes it easy for a user to download the software and start Cassandra without changing any defaults and it will just work. However, {{SimpleSnitch}} is not datacenter aware. {{GossipingPropertyFileSnitch}} could also be used as the default and would make the default snitch be datacenter aware. The user simply needs to update the local datacenter and rack names on each node and when it joins the cluster, it will propagate that topology information around the ring. This ticket would be to update the default from {{SimpleSnitch}} to {{GossipingPropertyFileSnitch}} to make the onboarding experience better for those who don't know to change this default if they intend to expand to multiple datacenters without too much hassle. > Update default snitch from SimpleSnitch to GossipingPropertyFileSnitch > -- > > Key: CASSANDRA-15523 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15523 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Jeremy Hanna >Priority: Normal > > Traditionally the project has had {{SimpleSnitch}} as the default primarily > because it makes it easy for a user to download the software and start > Cassandra without changing any defaults and it will just work. However, > {{SimpleSnitch}} is not datacenter aware. {{GossipingPropertyFileSnitch}} > could also be used as the default and would make the default snitch be > datacenter aware. Out of the box it will work with the default configuration. > This ticket would be to update the default from {{SimpleSnitch}} to > {{GossipingPropertyFileSnitch}} to make the onboarding experience better for > those who don't know to change this default if they intend to expand to > multiple datacenters without too much hassle. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Joshi updated CASSANDRA-10190: - Status: Changes Suggested (was: Review In Progress) > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022679#comment-17022679 ] Dinesh Joshi commented on CASSANDRA-10190: -- I have rebased and squashed, [~ptbannister]'s original branch into my repo [here|https://github.com/dineshjoshi/cassandra/tree/10190-trunk-review] and run [circleci|https://circleci.com/workflow-run/53b5f86f-2a97-4ce8-8269-9fab522f9406] jobs. There are about 5 tests that are failing right now. The following tests pass locally but fail on CircleCI (let's debug these later) – - test_writing_with_token_boundaries - test_datetime_values - test_remote_query The following tests fail locally and on CircleCI so I think these are Genuine failures (let's focus on them right now) – - test_complex_schema - test_complex_data_types > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-10190) Python 3 support for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-10190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022679#comment-17022679 ] Dinesh Joshi edited comment on CASSANDRA-10190 at 1/24/20 4:36 AM: --- I have rebased and squashed, [~ptbannister]'s original branch into my repo [here|https://github.com/dineshjoshi/cassandra/tree/10190-trunk-review]. Let's use this branch going forward. I also ran [circleci|https://circleci.com/workflow-run/53b5f86f-2a97-4ce8-8269-9fab522f9406] jobs. There are about 5 tests that are failing right now. The following tests pass locally but fail on CircleCI (let's debug these later) – - test_writing_with_token_boundaries - test_datetime_values - test_remote_query The following tests fail locally and on CircleCI so I think these are Genuine failures (let's focus on them right now) – - test_complex_schema - test_complex_data_types was (Author: djoshi3): I have rebased and squashed, [~ptbannister]'s original branch into my repo [here|https://github.com/dineshjoshi/cassandra/tree/10190-trunk-review] and run [circleci|https://circleci.com/workflow-run/53b5f86f-2a97-4ce8-8269-9fab522f9406] jobs. There are about 5 tests that are failing right now. The following tests pass locally but fail on CircleCI (let's debug these later) – - test_writing_with_token_boundaries - test_datetime_values - test_remote_query The following tests fail locally and on CircleCI so I think these are Genuine failures (let's focus on them right now) – - test_complex_schema - test_complex_data_types > Python 3 support for cqlsh > -- > > Key: CASSANDRA-10190 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10190 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Andrew Pennebaker >Assignee: Patrick Bannister >Priority: Normal > Labels: cqlsh > Fix For: 4.0, 4.0-alpha > > Attachments: > 0001-Fix-issues-from-version-specific-logic-commit.patch, > 0001-Update-six-to-1.12.0.patch, > 0002-Simplify-version-specific-logic-by-using-six.moves-a.patch, > coverage_notes.txt > > > Users who operate in a Python 3 environment may have trouble launching cqlsh. > Could we please update cqlsh's syntax to run in Python 3? > As a workaround, users can setup pyenv, and cd to a directory with a > .python-version containing "2.7". But it would be nice if cqlsh supported > modern Python versions out of the box. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15358) Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue
[ https://issues.apache.org/jira/browse/CASSANDRA-15358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021835#comment-17021835 ] Gaurav Ashok edited comment on CASSANDRA-15358 at 1/24/20 5:16 AM: --- Hi, I also experienced this error while testing out 4.alpha2 version. I have been testing the incremental repair. In one of the test a node was missing 30GB of data. (~ 100GB total data, #nodes = 3, rf = 3). Soon after starting the repair, I get this exception and the repair process stops making any progress. I do get "Maximum memory usage reached (2.000GiB), cannot allocate chunk of 8.000MiB" before the exception. I am attaching the trace of a repair process and the debug logs. [^repair_1_trace.txt] [^debug_logs_during_repair.txt] Any way I can test out this patch? was (Author: g-ashok): Hi, I also experienced this error while testing out 4.alpha2 version. I have been testing the incremental repair. In one of the test a node was missing 30GB of data. (~ 100GB total data). Soon after starting the repair, I get this exception and the repair process stops making any progress. I do get "Maximum memory usage reached (2.000GiB), cannot allocate chunk of 8.000MiB" before the exception. I am attaching the trace of a repair process and the debug logs. [^repair_1_trace.txt] [^debug_logs_during_repair.txt] Any way I can test out this patch? > Cassandra alpha 4 testing - Nodes crashing due to bufferpool allocator issue > > > Key: CASSANDRA-15358 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15358 > Project: Cassandra > Issue Type: Bug > Components: Test/benchmark >Reporter: Santhosh Kumar Ramalingam >Assignee: Benedict Elliott Smith >Priority: Normal > Labels: 4.0, alpha > Attachments: all_errors.txt, debug_logs_during_repair.txt, > repair_1_trace.txt, verbose_logs.diff, verbose_logs.txt > > > Hitting a bug with cassandra 4 alpha version. The same bug is repeated with > difefrent version of Java(8,11 &12) [~benedict] > > Stack trace: > {code:java} > INFO [main] 2019-10-11 16:07:12,024 Server.java:164 - Starting listening for > CQL clients on /1.3.0.6:9042 (unencrypted)... > WARN [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:13,961 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > WARN [Messaging-EventLoop-3-2] 2019-10-11 16:07:22,038 NoSpamLogger.java:94 - > 10.3x.4x.5x:7000->1.3.0.5:7000-LARGE_MESSAGES-[no-channel] dropping message > of type PING_REQ whose timeout expired before reaching the network > WARN [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:343 > - CassandraRoleManager skipped default role setup: some nodes were not ready > INFO [OptionalTasks:1] 2019-10-11 16:07:23,963 CassandraRoleManager.java:369 > - Setup task failed with error, rescheduling > INFO [Messaging-EventLoop-3-6] 2019-10-11 16:07:32,759 NoSpamLogger.java:91 - > 10.3x.4x.5x:7000->1.3.0.2:7000-URGENT_MESSAGES-[no-channel] failed to connect > io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) > failed: Connection refused: /1.3.0.2:7000 > Caused by: java.net.ConnectException: finishConnect(..) failed: Connection > refused > at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) > at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:667) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:644) > at > io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:524) > at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:414) > at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:326) > at > io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:918) > at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) > at > io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) > at java.base/java.lang.Thread.run(Thread.java:834) > WARN [Messaging-EventLoop-3-3] 2019-10-11 16:11:32,639 NoSpamLogger.java:94 - > 1.3.4.6:7000->1.3.4.5:7000-URGENT_MESSAGES-[no-channel] dropping message of > type GOSSIP_DIGEST_SYN whose timeout expired before reaching the network > INFO [Messaging-EventLoop-3-18] 2019-10-11 16:11:33,077 NoSpamLogger.java:91 > - 1.3.4.5:7000->1.3.4.4:7000-URGENT_MESSAGES-[no-channel] failed to connect > >
[jira] [Commented] (CASSANDRA-15469) IndexInfo/ColumnIndex Refactors & Cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-15469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022698#comment-17022698 ] Jordan West commented on CASSANDRA-15469: - Good catch [~djoshi]. I think these could be addressed storing the cache size in a field and updating it on every call to {{reset()}}. The buffer allocation issue could be addressed by reallocating when a difference in the size of the buffer and the expected size given the current field value is detected. WDTY? One issue I see tangentially related to this is that updating the value doesn't affect the existing cached indexes. I think its safe to assume that an operator can drop the cache if needed but I'd like to check w/ [~clohfink] here if something more heavy handed is desired re: existing cached items when updating the cache size property. > IndexInfo/ColumnIndex Refactors & Cleanup > - > > Key: CASSANDRA-15469 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15469 > Project: Cassandra > Issue Type: Improvement > Components: Local/SSTable >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta > > > Changes determined by audit of IndexInfo and ColumnIndex changes for > Cassandra 4.0 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15520) split circle ci commands into reusable scripts which can be used outside of circle ci
[ https://issues.apache.org/jira/browse/CASSANDRA-15520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022710#comment-17022710 ] Michael Semb Wever commented on CASSANDRA-15520: Jenkins uses build scripts already, those in https://github.com/apache/cassandra-builds/tree/master/build-scripts AFAIK these were added before CircleCI, and because CircleCI had it's own challenges and advantages they weren't re-used. [~spod] knows more on this. It would be nice to a) move these scripts in-tree (eg into a {{`.ci`}} folder), and b) bring them up to scratch for CircleCI. I suspect doing this will also give us a chunk of the work required to parallelise the dtests in Jenkins :) > split circle ci commands into reusable scripts which can be used outside of > circle ci > - > > Key: CASSANDRA-15520 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15520 > Project: Cassandra > Issue Type: Improvement > Components: Build >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > CircleCI is one of the main tools we use for build and test, but there is > also ASF Jenkins and many people run builds in their own companies as well. > It would be nice to refactor the existing CircleCI yaml to delegate to a set > of scripts which could be reused by other build systems. > I feel that we could do the following directory layout > {code} > ci - top level directory containing all scripts > - - directory containing the different build > steps > - - a single build with > the required steps to run it > - split.sh - script which takes in a output file to write to and > dumps out all test cases (not partitioned) > - run_partition.sh - script which takes a partitioned list of tests > and executes the build (does not move around artifacts) > {code} > This would allow CircleCI and Jenkins to run the same way, but also acts as > documentation for how to run some of the tests (jvm-dtest and python dtest > upgrade tests take more time to figure out how to run). > CircleCI would also be simpler as it would mostly be the circle ci specific > logic (partition tests, move code/test results around, etc.) calling these > scripts. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[ https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022712#comment-17022712 ] Thomas Steinmaurer commented on CASSANDRA-15430: [~benedict], in the previously provided OneDrive Link, I have put another JFR where the entire cluster was on 3.0.18, thus any {{LegacyLayout}} related signs in the stack traces should be gone. I see no reason to open another ticket for that, cause it does not change the situation, that the write path churns a lot (compared to 2.1). Thanks! > Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations > compared to 2.1.18 > > > Key: CASSANDRA-15430 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15430 > Project: Cassandra > Issue Type: Bug >Reporter: Thomas Steinmaurer >Priority: Normal > Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png, > screenshot-1.png, screenshot-2.png, screenshot-3.png, screenshot-4.png > > > In a 6 node loadtest cluster, we have been running with 2.1.18 a certain > production-like workload constantly and sufficiently. After upgrading one > node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of > regression described below), 3.0.18 is showing increased CPU usage, increase > GC, high mutation stage pending tasks, dropped mutation messages ... > Some spec. All 6 nodes equally sized: > * Bare metal, 32 physical cores, 512G RAM > * Xmx31G, G1, max pause millis = 2000ms > * cassandra.yaml basically unchanged, thus same settings in regard to number > of threads, compaction throttling etc. > Following dashboard shows highlighted areas (CPU, suspension) with metrics > for all 6 nodes and the one outlier being the node upgraded to Cassandra > 3.0.18. > !dashboard.png|width=1280! > Additionally we see a large increase on pending tasks in the mutation stage > after the upgrade: > !mutation_stage.png! > And dropped mutation messages, also confirmed in the Cassandra log: > {noformat} > INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 - > MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout > and 0 for cross node timeout > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool > NameActive Pending Completed Blocked All Time > Blocked > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 - > MutationStage 256 81824 3360532756 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 - > ViewMutationStage 0 0 0 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 - > ReadStage 0 0 62862266 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 - > RequestResponseStage 0 0 2176659856 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 - > ReadRepairStage 0 0 0 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 - > CounterMutationStage 0 0 0 0 > 0 > ... > {noformat} > Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different > node, high-level, it looks like the code path underneath > {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in > 3.0.18 compared to 2.1.18. > !jfr_allocations.png! > Left => 3.0.18 > Right => 2.1.18 > JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I > can upload them, if there is another destination available. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[ https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17022712#comment-17022712 ] Thomas Steinmaurer edited comment on CASSANDRA-15430 at 1/24/20 6:07 AM: - [~benedict], in the previously provided OneDrive Link, I have put another JFR (sub-directory {{full_on_cas_3.0.18}}) where the entire cluster was on 3.0.18, thus any {{LegacyLayout}} related signs in the stack traces should be gone. I see no reason to open another ticket for that, cause it does not change the situation, that the write path churns a lot (compared to 2.1). Thanks! was (Author: tsteinmaurer): [~benedict], in the previously provided OneDrive Link, I have put another JFR where the entire cluster was on 3.0.18, thus any {{LegacyLayout}} related signs in the stack traces should be gone. I see no reason to open another ticket for that, cause it does not change the situation, that the write path churns a lot (compared to 2.1). Thanks! > Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations > compared to 2.1.18 > > > Key: CASSANDRA-15430 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15430 > Project: Cassandra > Issue Type: Bug >Reporter: Thomas Steinmaurer >Priority: Normal > Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png, > screenshot-1.png, screenshot-2.png, screenshot-3.png, screenshot-4.png > > > In a 6 node loadtest cluster, we have been running with 2.1.18 a certain > production-like workload constantly and sufficiently. After upgrading one > node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of > regression described below), 3.0.18 is showing increased CPU usage, increase > GC, high mutation stage pending tasks, dropped mutation messages ... > Some spec. All 6 nodes equally sized: > * Bare metal, 32 physical cores, 512G RAM > * Xmx31G, G1, max pause millis = 2000ms > * cassandra.yaml basically unchanged, thus same settings in regard to number > of threads, compaction throttling etc. > Following dashboard shows highlighted areas (CPU, suspension) with metrics > for all 6 nodes and the one outlier being the node upgraded to Cassandra > 3.0.18. > !dashboard.png|width=1280! > Additionally we see a large increase on pending tasks in the mutation stage > after the upgrade: > !mutation_stage.png! > And dropped mutation messages, also confirmed in the Cassandra log: > {noformat} > INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 - > MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout > and 0 for cross node timeout > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool > NameActive Pending Completed Blocked All Time > Blocked > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 - > MutationStage 256 81824 3360532756 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 - > ViewMutationStage 0 0 0 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 - > ReadStage 0 0 62862266 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 - > RequestResponseStage 0 0 2176659856 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 - > ReadRepairStage 0 0 0 0 > 0 > INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 - > CounterMutationStage 0 0 0 0 > 0 > ... > {noformat} > Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different > node, high-level, it looks like the code path underneath > {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in > 3.0.18 compared to 2.1.18. > !jfr_allocations.png! > Left => 3.0.18 > Right => 2.1.18 > JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I > can upload them, if there is another destination available. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org