[jira] [Commented] (CASSANDRA-16465) Increased Read Latency With Cassandra >= 3.11.7

[Stefan Miklosovic (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332982#comment-17332982
 ] 

Stefan Miklosovic commented on CASSANDRA-16465:
---

Hey guys, is this acutally an issue for 4.0 as well? I am just wondering if we 
see this in 3, why could not it be in 4 as well?

> Increased Read Latency With Cassandra >= 3.11.7
> ---
>
> Key: CASSANDRA-16465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16465
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Ahmed ELJAMI
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
> Attachments: Screenshot from 2021-04-24 20-50-47.png
>
>
> After upgrading Cassandra from 3.11.3 to 3.11.9, Cassandra read latency 99% 
> increased significantly. Getting back to 3.11.3 immediately fixed the issue.
> I have observed "SStable reads" increases after upgrading to 3.11.9.
> The same behavior was observed by some other users: 
> [https://www.mail-archive.com/user@cassandra.apache.org/msg61247.html]
> According to Paulo Motta's comment, this behavior may be caused by 
> https://issues.apache.org/jira/browse/CASSANDRA-15690 which was introduced on 
> 3.11.7 and removed an optimization that may cause a correctness issue when 
> there are partition deletions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Marcus Eriksson (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332966#comment-17332966
 ] 

Marcus Eriksson commented on CASSANDRA-16634:
-

bq. The only drawback is if the size of the output files is significantly 
smaller than the source, in which case the source level would be under-sized.
we have CASSANDRA-7414 for this case

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16626) Improve how-to commit docs

[Berenguer Blasi (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16626:

Source Control Link: 
https://github.com/apache/cassandra/commit/e74d89114d7d24a17a51bda60a3ac8ad24627058
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> Improve how-to commit docs
> --
>
> Key: CASSANDRA-16626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16626
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: index.jpg
>
>
> The how to commit page looks outdated where 3.3 is being merged into trunk 
> whereas it should be 3.11 instead. Also adding a comment on folding patches 
> to the forward merge.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Fix How-to Commit docs

[bereng]

This is an automated email from the ASF dual-hosted git repository.

bereng pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new e74d891  Fix How-to Commit docs
e74d891 is described below

commit e74d89114d7d24a17a51bda60a3ac8ad24627058
Author: Bereng 
AuthorDate: Thu Apr 22 09:38:38 2021 +0200

Fix How-to Commit docs

patch by Berenguer Blasi; reviewed by Mick Semb Wever, Ekaterina Dimitrova 
for CASSANDRA-16626
---
 doc/source/development/how_to_commit.rst | 49 +---
 1 file changed, 32 insertions(+), 17 deletions(-)

diff --git a/doc/source/development/how_to_commit.rst 
b/doc/source/development/how_to_commit.rst
index 528ac40..148f3f0 100644
--- a/doc/source/development/how_to_commit.rst
+++ b/doc/source/development/how_to_commit.rst
@@ -23,26 +23,26 @@ If you are a committer, feel free to pick any process that 
works for you - so lo
 
 Here is how committing and merging will usually look for merging and pushing 
for tickets that follow the convention (if patch-based):
 
-Hypothetical CASSANDRA-12345 ticket is a cassandra-3.0 based bug fix that 
requires different code for cassandra-3.3, and trunk. Contributor Jackie 
supplied a patch for the root branch (12345-3.0.patch), and patches for the 
remaining branches (12345-3.3.patch, 12345-trunk.patch).
+Hypothetical CASSANDRA-12345 ticket is a cassandra-3.0 based bug fix that 
requires different code for cassandra-3.11, and trunk. Contributor Jackie 
supplied a patch for the root branch (12345-3.0.patch), and patches for the 
remaining branches (12345-3.11.patch, 12345-trunk.patch).
 
 On cassandra-3.0:
#. ``git am -3 12345-3.0.patch`` (if we have a problem b/c of CHANGES.txt 
not merging anymore, we fix  it ourselves, in place)
#. ``ant realclean && ant jar build-test`` (rebuild to make sure code 
compiles)
 
-On cassandra-3.3:
+On cassandra-3.11:
#. ``git merge cassandra-3.0 -s ours``
-   #. ``git apply -3 12345-3.3.patch`` (likely to have an issue with 
CHANGES.txt here: fix it ourselves, then git add CHANGES.txt)
+   #. ``git apply -3 12345-3.11.patch`` (likely to have an issue with 
CHANGES.txt here: fix it ourselves, then git add CHANGES.txt)
#. ``ant realclean && ant jar build-test`` (rebuild to make sure code 
compiles)
-   #. ``git commit --amend``
+   #. ``git commit --amend`` (Notice this will squash the 3.11 applied patch 
into the forward merge commit)
 
 On trunk:
-   #. ``git merge cassandra-3.3 -s ours``
+   #. ``git merge cassandra-3.11 -s ours``
#. ``git apply -3 12345-trunk.patch`` (likely to have an issue with 
CHANGES.txt here: fix it ourselves, then git add CHANGES.txt)
#. ``ant realclean && ant jar build-test`` (rebuild to make sure code 
compiles)
-   #. ``git commit --amend``
+   #. ``git commit --amend`` (Notice this will squash the trunk applied patch 
into the forward merge commit)
 
 On any branch:
-   #. ``git push origin cassandra-3.0 cassandra-3.3 trunk --atomic``
+   #. ``git push origin cassandra-3.0 cassandra-3.11 trunk --atomic``
 
 Same scenario, but a branch-based contribution:
 
@@ -50,22 +50,22 @@ On cassandra-3.0:
#. ``git cherry-pick `` (if we have a problem b/c of 
CHANGES.txt not merging anymore, we fix it ourselves, in place)
#. ``ant realclean && ant jar build-test`` (rebuild to make sure code 
compiles)
 
-On cassandra-3.3:
+On cassandra-3.11:
#. ``git merge cassandra-3.0 -s ours``
-   #. ``git format-patch -1 ``
-   #. ``git apply -3 .patch`` (likely to have an issue with 
CHANGES.txt here: fix it ourselves, then git add CHANGES.txt)
+   #. ``git format-patch -1 ``
+   #. ``git apply -3 .patch`` (likely to have an issue 
with CHANGES.txt here: fix it ourselves, then git add CHANGES.txt)
#. ``ant realclean && ant jar build-test`` (rebuild to make sure code 
compiles)
-   #. ``git commit --amend``
+   #. ``git commit --amend`` (Notice this will squash the 3.11 applied patch 
into the forward merge commit)
 
 On trunk:
-   #. ``git merge cassandra-3.3 -s ours``
+   #. ``git merge cassandra-3.11 -s ours``
#. ``git format-patch -1 ``
#. ``git apply -3 .patch`` (likely to have an issue 
with CHANGES.txt here: fix it ourselves, then git add CHANGES.txt)
#. ``ant realclean && ant jar build-test`` (rebuild to make sure code 
compiles)
-   #. ``git commit --amend``
+   #. ``git commit --amend`` (Notice this will squash the trunk applied patch 
into the forward merge commit)
 
 On any branch:
-   #. ``git push origin cassandra-3.0 cassandra-3.3 trunk --atomic``
+   #. ``git push origin cassandra-3.0 cassandra-3.11 trunk --atomic``
 
 If the patch is for an older branch, and doesn't impact later branches (such 
as trunk), we still need to merge up.
 
@@ -73,16 +73,31 @@ On cassandra-3.0:
#. ``git cherry-pick `` (if we have a problem b/c of 
CHANGES.txt not merging anymore, we fix it ourselves, in place)
  

[jira] [Updated] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-16634:

Status: Ready to Commit  (was: Review In Progress)

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Marcus Eriksson (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332946#comment-17332946
 ] 

Marcus Eriksson commented on CASSANDRA-16634:
-

+1, running tests here before committing: 
https://app.circleci.com/pipelines/github/krummas/cassandra?branch=bug%2FCASSANDRA-16634

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16626) Improve how-to commit docs

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332943#comment-17332943
 ] 

Berenguer Blasi commented on CASSANDRA-16626:
-

Final version of the page 
[here|https://issues.apache.org/jira/secure/attachment/13024637/index.jpg]

> Improve how-to commit docs
> --
>
> Key: CASSANDRA-16626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16626
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: index.jpg
>
>
> The how to commit page looks outdated where 3.3 is being merged into trunk 
> whereas it should be 3.11 instead. Also adding a comment on folding patches 
> to the forward merge.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16626) Improve how-to commit docs

[Berenguer Blasi (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16626:

Status: Ready to Commit  (was: Review In Progress)

> Improve how-to commit docs
> --
>
> Key: CASSANDRA-16626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16626
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: index.jpg
>
>
> The how to commit page looks outdated where 3.3 is being merged into trunk 
> whereas it should be 3.11 instead. Also adding a comment on folding patches 
> to the forward merge.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-16634:

Reviewers: Marcus Eriksson, Marcus Eriksson  (was: Marcus Eriksson)
   Marcus Eriksson, Marcus Eriksson  (was: Marcus Eriksson)
   Status: Review In Progress  (was: Patch Available)

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16626) Improve how-to commit docs

[Berenguer Blasi (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16626:

Attachment: (was: index.jpg)

> Improve how-to commit docs
> --
>
> Key: CASSANDRA-16626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16626
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: index.jpg
>
>
> The how to commit page looks outdated where 3.3 is being merged into trunk 
> whereas it should be 3.11 instead. Also adding a comment on folding patches 
> to the forward merge.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16626) Improve how-to commit docs

[Berenguer Blasi (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi updated CASSANDRA-16626:

Attachment: index.jpg

> Improve how-to commit docs
> --
>
> Key: CASSANDRA-16626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16626
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: index.jpg
>
>
> The how to commit page looks outdated where 3.3 is being merged into trunk 
> whereas it should be 3.11 instead. Also adding a comment on folding patches 
> to the forward merge.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-16634:

Reviewers: Marcus Eriksson

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332814#comment-17332814
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16634:
-

Hey [~scottcarey],

Thank you for the patch. I was trying to find you on Slack to say hi and help 
you to clear some misunderstanding from our docs, but I didn't find you in the 
ASF Slack. (You might want to check the #cassandra-dev channel and the dev 
mailing list - https://cassandra.apache.org/community/).  If you drop a 
question in the #cassandra-dev channel there are high chances of getting a fast 
response as we have contributors to the project all over the world and people 
are willing to help when they have a bit of time. 

Both patch and pull request submission work but you are right that pull 
requests are easier to handle and almost everyone use them on the project 
nowadays. We don't open pull requests to the Apache/cassandra repo though. Most 
people just have their fork where they work on things. I would suggest you to 
check this page (if you haven't) 
-https://cassandra.apache.org/doc/latest/development/patches.html?highlight=contributing.
  

Also, we normally try to run some preliminary tests at least. More information 
on how to run tests locally or setup CI can be found here - 
[https://cassandra.apache.org/doc/latest/development/testing.html?highlight=testing]

Unfortunately, at this point the official Jenkins CI is available only for 
committers and PMC members but anyone of us will be happy to submit a CI run 
for you. Circle CI free tier is also an option, just bear in mind that you will 
see Python dtests failing  due to not enough resources available, that is 
expected. 

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple l

[jira] [Updated] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16634:
-
Test and Documentation Plan: test included
 Status: Patch Available  (was: Open)

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332805#comment-17332805
 ] 

Brandon Williams commented on CASSANDRA-16634:
--

[~marcuse] can you take a look?

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16634:
-
 Bug Category: Parent values: Correctness(12982)
   Complexity: Normal
  Component/s: Local/Compaction
Discovered By: User Report
Fix Version/s: 4.0.x
   3.11.x
 Severity: Normal
   Status: Open  (was: Triage Needed)

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Scott Carey (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332803#comment-17332803
 ] 

Scott Carey commented on CASSANDRA-16634:
-

[https://github.com/apache/cassandra/pull/986]

 

Its based on the 3.11 branch, but applies cleanly to trunk as well.

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16238) Fix flaky jvm-dtests that fail with Unable to contact any seeds

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332802#comment-17332802
 ] 

David Capwell commented on CASSANDRA-16238:
---

Review

* 
https://github.com/apache/cassandra/compare/trunk...driftx:CASSANDRA-16238#diff-99267a2170b04fd7dd24d6c6bf2ba1fc26d6dc896cd74f8c5bd56c476e2540e4R580
 - nit: you can call isEmpty rather than size

For the host replacement tests, I set that field low to help find issues, so if 
this case happens more frequently because of that I am cool removing it; but we 
do see this outside of these classes as well.

bq. .1 is detected for the first time via gossip, and as it is going through 
StorageService but before it is added to TokenMetatadata, the gossiper's status 
check has begun running

If I understand you, the call to StorageService.onChange (which calls 
handleStateNormal) happens-after gossip status check, so removes the state? 
GossipDigestAck2 should be handled in the gossip stage and eventually call 
applyNewStates to apply the state and trigger notifications, but doStatusCheck 
is called in the GossipTasks thread pool, which checks isGossipOnlyMember which 
returns true in this case (as state isn't fully settled yet), at which point we 
schedule a task in the gossip stage to remove (but at this point the 
isGossipOnlyMember(endpoint) == false).  

If I understand you correctly, this feels like a race condition where we read 
data not fully committed, which feels like a bug (which is why it was set to 0 
in the first place).  Do I understand you [~brandon.williams]?

> Fix flaky jvm-dtests that fail with Unable to contact any seeds
> ---
>
> Key: CASSANDRA-16238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: 16238-archived-failures.txt
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/745/workflows/1c7e589e-b5af-4a56-b40a-43da424602c7/jobs/4231
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795), 
> ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795)]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332796#comment-17332796
 ] 

Brandon Williams commented on CASSANDRA-16634:
--

Opening a PR is fine and you can base your work on any branch but I would 
recommend trunk and then backporting can be figured out later if necessary.

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Scott Carey (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332793#comment-17332793
 ] 

Scott Carey commented on CASSANDRA-16634:
-

I need some minor guidance on the process.  Documentation is all over the place 
– some says that we must not use pull requests and must attach a patch, but it 
looks like PRs are utilized now?    I also don't know what the earliest branch 
that this should apply to is.

 

I am an Apache member.  My github account is linked to the apache org.

For now I will open a PR that can be viewed with my suggested fix.  We can 
correct whatever errors I've made process wise after that.

> Garbagecollect should not output all tables to L0 with 
> LeveledCompactionStrategy
> 
>
> Key: CASSANDRA-16634
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Scott Carey
>Assignee: Scott Carey
>Priority: Normal
>
> nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.
> This is awful.  On a large LCS table, this means that at the end of the 
> garbagecollect process, all data is in L0.
>  
> This results in an awful sequence of useless temporary space usage and write 
> amplification:
>  # L0 is repeatedly size-tiered compacted until it doesn't have too many 
> SSTables.  If the original LCS table had 2000 tables... this takes a long time
>  # L0 is compacted to L1 in one to a couple very very large compactions
>  # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore
> Due to the above, 'nodetool garbagecollect' is close to worthless for large 
> LCS tables.  A full compaction is always less write amplification and similar 
> temp disk space required.  The only exception is if you can use 'nodetool 
> garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 
> is too large.  In this case if you are lucky, and the order that it chose to 
> process SSTables coincides with tables that have the most  disk space to 
> clear, you might free up enough disk space to succeed in your original goal.
>  
> However, from what I can tell, there is no good reason to move the output to 
> L0.  Leaving the output table in the same SSTableLevel as the source table 
> does not violate any of the LeveledCompactionStrategy placement rules, as the 
> output by definition has a token range equal to or smaller than the source.
> The only drawback is if the size of the output files is significantly smaller 
> than the source, in which case the source level would be under-sized.   But 
> that seems like a problem that LCS has to handle, not garbagecollect.
> LCS could have a "pull up" operation where it does something like the 
> following.   Assume a table has L4 as the max level, and L3 and L4 are both 
> 'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
> overlap with the token ranges of the L3 tables.  After that, it can choose to 
> do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
> significantly under-sized.
> From what I can tell, garbagecollect should just re-write tables in place, 
> and leave the compaction strategy to deal with any consequences.
> Moving to L0 is a bad idea.  In addition to the extra write amplification and 
> extreme increase in temporary disk space required, I observed the following:
> A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
> We stopped it about 20% through the process, and it managed to compact down 
> the top couple levels.  So we tried to run 'garbagecollect' again, but the 
> first tables it chose to operate on were in L1, not the 'leafs' in L5!   This 
> was because the order of SSTables chosen currently does not consider the 
> level, and instead looks purely at the max timestamp in the  file.  But 
> because we moved _very old_ data from L5 into L0 as a result of the prior 
> gabagecollect, manytables in L1 and L2 now had very wide ranges between their 
> min and max timestamps – essentially some of the oldest and newest data all 
> in one table.    This breaks the usual structure of an LCS table where the 
> oldest data is at the high levels.
>  
> I hope that others agree that this is a bug, and deserving of a fix.
> I have a very simple patch for this that I will be creating a PR for soon.  3 
> lines for the code change, 70 lines for a new unit test.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16238) Fix flaky jvm-dtests that fail with Unable to contact any seeds

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16238:
--
Reviewers: David Capwell, Ekaterina Dimitrova, David Capwell  (was: David 
Capwell, Ekaterina Dimitrova)
   David Capwell, Ekaterina Dimitrova, David Capwell  (was: 
Ekaterina Dimitrova)
   Status: Review In Progress  (was: Patch Available)

> Fix flaky jvm-dtests that fail with Unable to contact any seeds
> ---
>
> Key: CASSANDRA-16238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: 16238-archived-failures.txt
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/745/workflows/1c7e589e-b5af-4a56-b40a-43da424602c7/jobs/4231
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795), 
> ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795)]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16238) Fix flaky jvm-dtests that fail with Unable to contact any seeds

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332790#comment-17332790
 ] 

David Capwell commented on CASSANDRA-16238:
---

The host replacement tests set it low, but what about the other classes which 
hit this (such as the original test 
https://app.circleci.com/pipelines/github/dcapwell/cassandra/745/workflows/1c7e589e-b5af-4a56-b40a-43da424602c7/jobs/4231)?
  We have seen this with 
nodeDownDuringMove(org.apache.cassandra.distributed.test.GossipTest) as well.

> Fix flaky jvm-dtests that fail with Unable to contact any seeds
> ---
>
> Key: CASSANDRA-16238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: 16238-archived-failures.txt
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/745/workflows/1c7e589e-b5af-4a56-b40a-43da424602c7/jobs/4231
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795), 
> ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795)]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16634) Garbagecollect should not output all tables to L0 with LeveledCompactionStrategy

[Scott Carey (Jira)]

Scott Carey created CASSANDRA-16634:
---

 Summary: Garbagecollect should not output all tables to L0 with 
LeveledCompactionStrategy
 Key: CASSANDRA-16634
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16634
 Project: Cassandra
  Issue Type: Bug
Reporter: Scott Carey
Assignee: Scott Carey


nodetool garbagecollect always outputs to L0 with LeveledCompactionStrategy.

This is awful.  On a large LCS table, this means that at the end of the 
garbagecollect process, all data is in L0.

 

This results in an awful sequence of useless temporary space usage and write 
amplification:
 # L0 is repeatedly size-tiered compacted until it doesn't have too many 
SSTables.  If the original LCS table had 2000 tables... this takes a long time
 # L0 is compacted to L1 in one to a couple very very large compactions
 # L1 is compacted to L2, L3 to L4, etc.  Write amplification galore

Due to the above, 'nodetool garbagecollect' is close to worthless for large LCS 
tables.  A full compaction is always less write amplification and similar temp 
disk space required.  The only exception is if you can use 'nodetool 
garbagecolect' part-way, and then use 'nodetool stop' to cancel it before L0 is 
too large.  In this case if you are lucky, and the order that it chose to 
process SSTables coincides with tables that have the most  disk space to clear, 
you might free up enough disk space to succeed in your original goal.

 

However, from what I can tell, there is no good reason to move the output to 
L0.  Leaving the output table in the same SSTableLevel as the source table does 
not violate any of the LeveledCompactionStrategy placement rules, as the output 
by definition has a token range equal to or smaller than the source.

The only drawback is if the size of the output files is significantly smaller 
than the source, in which case the source level would be under-sized.   But 
that seems like a problem that LCS has to handle, not garbagecollect.

LCS could have a "pull up" operation where it does something like the 
following.   Assume a table has L4 as the max level, and L3 and L4 are both 
'under-sized'.  L3 can attempt to 'pull up' any tables from L4 that do not 
overlap with the token ranges of the L3 tables.  After that, it can choose to 
do some compactions that mix L3 and L4 to pull up data into L3 if it is still 
significantly under-sized.

>From what I can tell, garbagecollect should just re-write tables in place, and 
>leave the compaction strategy to deal with any consequences.

Moving to L0 is a bad idea.  In addition to the extra write amplification and 
extreme increase in temporary disk space required, I observed the following:

A 'nodetool garbagecollect' was placing a lot of pressure on a L0 of a node.  
We stopped it about 20% through the process, and it managed to compact down the 
top couple levels.  So we tried to run 'garbagecollect' again, but the first 
tables it chose to operate on were in L1, not the 'leafs' in L5!   This was 
because the order of SSTables chosen currently does not consider the level, and 
instead looks purely at the max timestamp in the  file.  But because we moved 
_very old_ data from L5 into L0 as a result of the prior gabagecollect, 
manytables in L1 and L2 now had very wide ranges between their min and max 
timestamps – essentially some of the oldest and newest data all in one table.   
 This breaks the usual structure of an LCS table where the oldest data is at 
the high levels.

 

I hope that others agree that this is a bug, and deserving of a fix.

I have a very simple patch for this that I will be creating a PR for soon.  3 
lines for the code change, 70 lines for a new unit test.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-website] branch asf-site updated (83ba244 -> 1e3b00f)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git.


from 83ba244  hack in plausible tracking with
 add 1e3b00f  Fix world-party link and page. Fix plausible tracking on new 
design pages.

No new revisions were added by this update.

Summary of changes:
 .../Apache-Cassandra-Changelog-1-October-2020.html |  2 +-
 ...Apache-Cassandra-Changelog-2-December-2020.html |  2 +-
 .../Apache-Cassandra-Changelog-3-January-2021.html |  2 +-
 ...Apache-Cassandra-Changelog-4-February-2021.html |  2 +-
 .../Apache-Cassandra-Changelog-5-March-2021.html   |  2 +-
 .../Apache-Cassandra-Changelog-6-April-2021.html   |  2 +-
 .../blog/Apache-Cassandra-Usage-Report-2020.html   |  2 +-
 .../blog/Audit-Logging-in-Apache-Cassandra-4.html  |  2 +-
 ...andra-and-Kubernetes-SIG-Update-and-Survey.html |  2 +-
 ...ty-with-5x-Faster-Streaming-in-Cassandra-4.html |  2 +-
 ...ra's-Internals-with-Property-based-Testing.html |  2 +-
 ...-Zero-Copy-Streaming-in-Apache-Cassandra-4.html |  2 +-
 ...che-Cassandras-Front-Door-and-Backpressure.html |  2 +-
 ...assandra-4-Beta-Battle-Tested-From-Day-One.html |  2 +-
 .../blog/Introducing-Transient-Replication.html|  2 +-
 content/blog/Join-Casssandra-GSoC-2021.html|  2 +-
 ...ced-for-April-28-Cassandra-4.0-World-Party.html |  2 +-
 content/blog/Testing-Apache-Cassandra-4.html   |  2 +-
 content/blog/World-Party.html  |  2 +-
 content/blog/index.html|  2 +-
 content/blog/template.html |  2 +-
 content/case-studies/index.html|  2 +-
 content/cassandra-basics/index.html|  2 +-
 content/community/index.html   |  2 +-
 content/download/index.html|  2 +-
 content/ecosystem/index.html   |  2 +-
 content/index.html |  6 +++---
 content/quickstart/index.html  |  2 +-
 content/resources/index.html   |  2 +-
 content/world-party/index.html | 22 ++
 30 files changed, 33 insertions(+), 51 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16238) Fix flaky jvm-dtests that fail with Unable to contact any seeds

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16238:

Reviewers: Ekaterina Dimitrova

> Fix flaky jvm-dtests that fail with Unable to contact any seeds
> ---
>
> Key: CASSANDRA-16238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: 16238-archived-failures.txt
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/745/workflows/1c7e589e-b5af-4a56-b40a-43da424602c7/jobs/4231
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795), 
> ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795)]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16238) Fix flaky jvm-dtests that fail with Unable to contact any seeds

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16238:
-
Test and Documentation Plan: CI
 Status: Patch Available  (was: In Progress)

> Fix flaky jvm-dtests that fail with Unable to contact any seeds
> ---
>
> Key: CASSANDRA-16238
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16238
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/java
>Reporter: David Capwell
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: 16238-archived-failures.txt
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/745/workflows/1c7e589e-b5af-4a56-b40a-43da424602c7/jobs/4231
> {code}
> test teardown failure
> Unexpected error found in node logs (see stdout for full details). Errors: 
> [ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795), 
> ERROR [main] 2020-10-29 17:38:13,808 CassandraDaemon.java:817 - Exception 
> encountered during startup
> java.lang.IllegalStateException: Unable to contact any seeds!
>   at 
> org.apache.cassandra.service.StorageService.bootstrap(StorageService.java:1601)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:931)
>   at 
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:892)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:699)
>   at 
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:635)
>   at 
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:407)
>   at 
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:671)
>   at 
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:795)]
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16238) Fix flaky jvm-dtests that fail with Unable to contact any seeds

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332737#comment-17332737
 ] 

Brandon Williams commented on CASSANDRA-16238:
--

Branch [here|https://github.com/driftx/cassandra/tree/CASSANDRA-16238].

[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/714/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/714/pipeline]

{noformat}
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:05,381 
Gossiper.java:1296 - Node /127.0.0.1:7012 is now part of the cluster
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:05,381 
StorageService.java:2730 - Node /127.0.0.1:7012 state NORMAL, token 
[-3074457345618258603]
[junit-timeout] INFO  [node4_GossipTasks:1] node4 2021-04-23 16:23:05,393 
Gossiper.java:997 - FatClient /127.0.0.1:7012 has been silent for 0ms, removing 
from gossip
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:05,398 
StorageService.java:2677 - New node /127.0.0.1:7012 at token 
-3074457345618258603
{noformat}

We can see here that .1 is detected for the first time via gossip, and as it is 
going through StorageService but before it is added to TokenMetatadata, the 
gossiper's status check has begun running.  Since the quarantine delay is 
overridden to zero, without a presence in TMD the node is not a member yet and 
thus deemed a fat client, and removed.

{noformat}
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:05,407 
TokenMetadata.java:505 - Updating topology for /127.0.0.1:7012
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:05,414 
TokenMetadata.java:505 - Updating topology for /127.0.0.1:7012
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:05,422 
StorageService.java:2730 - Node /127.0.0.1:7012 state NORMAL, token 
[-3074457345618258603]
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:05,422 
StorageService.java:2733 - Node /127.0.0.1:7012 state jump to NORMAL
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:05,447 
Gossiper.java:1243 - removing expire time for endpoint : /127.0.0.1:7012
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:05,447 
Gossiper.java:1244 - InetAddress /127.0.0.1:7012 is now UP
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:05,447 
Gossiper.java:579 - removed /127.0.0.1:7012 from seeds, updated seeds list = []
[junit-timeout] WARN  16:23:05 Seeds list is now empty!
[junit-timeout] WARN  [node4_GossipStage:1] node4 2021-04-23 16:23:05,447 
Gossiper.java:581 - Seeds list is now empty!
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:05,452 
Gossiper.java:590 - removing endpoint /127.0.0.1:7012
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:05,452 
Gossiper.java:561 - evicting /127.0.0.1:7012 from gossip
[junit-timeout] DEBUG [node4_GossipTasks:1] node4 2021-04-23 16:23:06,453 
Gossiper.java:1025 - 0 elapsed, /127.0.0.1:7012 gossip quarantine over
{noformat}

Crucially, as part of this removal the node is also removed from the seeds 
list, since it is listed there. The warning about the empty seed list is added 
from my branch.

{noformat}
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:07,176 
Gossiper.java:1296 - Node /127.0.0.1:7012 is now part of the cluster
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:07,177 
StorageService.java:2730 - Node /127.0.0.1:7012 state NORMAL, token 
[-3074457345618258603]
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:07,189 
StorageService.java:2677 - New node /127.0.0.1:7012 at token 
-3074457345618258603
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:07,198 
TokenMetadata.java:505 - Updating topology for /127.0.0.1:7012
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:07,201 
TokenMetadata.java:505 - Updating topology for /127.0.0.1:7012
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:07,208 
StorageService.java:2730 - Node /127.0.0.1:7012 state NORMAL, token 
[-3074457345618258603]
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:07,208 
StorageService.java:2733 - Node /127.0.0.1:7012 state jump to NORMAL
[junit-timeout] DEBUG [node4_GossipStage:1] node4 2021-04-23 16:23:07,220 
Gossiper.java:1243 - removing expire time for endpoint : /127.0.0.1:7012
[junit-timeout] INFO  [node4_GossipStage:1] node4 2021-04-23 16:23:07,220 
Gossiper.java:1244 - InetAddress /127.0.0.1:7012 is now UP
[junit-timeout] DEBUG [node4_BatchlogTasks:1] node4 2021-04-23 16:23:07,383 
BatchlogManager.java:246 - Updating batchlog replay throttle to 1024 KB/s, 256 
KB/s per endpoint
[junit-timeout] DEBUG [node4_isolatedExecutor:1] node4 2021-04-23 16:23:12,457 
Gossiper.java:2142 - Gossip looks s

[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches

[Alexey Zotov (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332735#comment-17332735
 ] 

Alexey Zotov commented on CASSANDRA-16404:
--

It turned out that AssertJ is preferable library for test assertions. I've 
converted all assertions in the PR from Hamcrest to AssertJ.

> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Alexey Zotov
>Priority: Normal
> Fix For: 4.x
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like 
> KeyCache, RowCache and CounterCache. 
> Being able to invalidate auth caches as well can come in handy in situations 
> where, critical backend auth changes may need to be in effect right away for 
> all the connections, especially in configurations where cache validity is 
> chosen to be for a longer duration. An example can be that an authenticated 
> user "User1" is no longer authorized to access a table resource "table1" and 
> it is vital that this change is reflected right away, without having to wait 
> for cache expiry/refresh to trigger.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16629) Rafactor tests for NodeTool commands

[Alexey Zotov (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332719#comment-17332719
 ] 

Alexey Zotov commented on CASSANDRA-16629:
--

{quote}I do not know how Hamcrest was introduced.
{quote}
Not sure, but most of the {{NodeTool}} tests were Hamcrest-based. That's why 
originally I converted the rest from AssertJ to Hamcrest. Since now it is clear 
that AssertJ is preferable, I've converted all tests to AssertJ. Looking 
forward for review comments.

Thanks for providing a great support!

> Rafactor tests for NodeTool commands
> 
>
> Key: CASSANDRA-16629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16629
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/unit
>Reporter: Alexey Zotov
>Assignee: Alexey Zotov
>Priority: Low
> Fix For: 4.0.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> h3. Overview
> As a part of CASSANDRA-16404, I've been working on {{NodeTool}} commands for 
> invalidation of the auth caches. While working on that, I've mentioned that 
> the existing NodeTool unit tests are written in a different manner and reside 
> in wrong packages. I've made the following changes to them:
> # moved them to the proper packages (that match original command class)
> # unified naming, formatting, imports
> # migrated from {{org.assertj}} to {{hamcrest}} for assertions (though it is 
> not smth really required)
> # made a general cleanup
> PR: https://github.com/apache/cassandra/pull/982



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16633) Only include versioned files in rat-report

[Benedict Elliott Smith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332695#comment-17332695
 ] 

Benedict Elliott Smith commented on CASSANDRA-16633:


Awesome. +1

> Only include versioned files in rat-report
> --
>
> Key: CASSANDRA-16633
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16633
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-rc2, 2.2.x, 3.0.x, 3.11.x
>
>
> Recent changes to the ant build.xml file to include running the Apache Rat 
> reporting tool break the build when there are unversioned files in the source 
> tree.
> We could modify the build to only include versioned files in the Rat report.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Sam Tunnicliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332691#comment-17332691
 ] 

Sam Tunnicliffe commented on CASSANDRA-16558:
-

I've opened CASSANDRA-16633 to make {{rat-check}} ignore untracked files. This 
is ready for review now.

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16633) Only include versioned files in rat-report

[Sam Tunnicliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332688#comment-17332688
 ] 

Sam Tunnicliffe edited comment on CASSANDRA-16633 at 4/26/21, 7:05 PM:
---

Fixed in trunk, then cherry-picked to earlier branches. Some files weren't 
being included in the rat report due to typos in the previous {{includes}} 
specificiation and a handful of those had non-compliant or missing licence 
headers, so I've added fixes for those where necessary.

|Branch||
|[16633-2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...beobal:16633-2.2]|
|[16633-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...beobal:16633-3.0?expand=1]|
|[16633-3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...beobal:16633-3.11?expand=1]|
|[16633-trunk|https://github.com/apache/cassandra/compare/trunk...beobal:16633-trunk?expand=1]|


was (Author: beobal):
Fixed in trunk, then cherry-picked to earlier branches. Some files weren't 
being included in the rat report due to typos in the previous {includes} 
specificiation and a handful of those had non-compliant or missing licence 
headers, so I've added fixes for those where necessary.

|Branch||
|[16633-2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...beobal:16633-2.2]|
|[16633-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...beobal:16633-3.0?expand=1]|
|[16633-3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...beobal:16633-3.11?expand=1]|
|[16633-trunk|https://github.com/apache/cassandra/compare/trunk...beobal:16633-trunk?expand=1]|

> Only include versioned files in rat-report
> --
>
> Key: CASSANDRA-16633
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16633
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-rc2, 2.2.x, 3.0.x, 3.11.x
>
>
> Recent changes to the ant build.xml file to include running the Apache Rat 
> reporting tool break the build when there are unversioned files in the source 
> tree.
> We could modify the build to only include versioned files in the Rat report.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16633) Only include versioned files in rat-report

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-16633:

Test and Documentation Plan: Manually tested running ant rat-check with 
untracked files in the source tree.
 Status: Patch Available  (was: Open)

Fixed in trunk, then cherry-picked to earlier branches. Some files weren't 
being included in the rat report due to typos in the previous {includes} 
specificiation and a handful of those had non-compliant or missing licence 
headers, so I've added fixes for those where necessary.

|Branch||
|[16633-2.2|https://github.com/apache/cassandra/compare/cassandra-2.2...beobal:16633-2.2]|
|[16633-3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...beobal:16633-3.0?expand=1]|
|[16633-3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...beobal:16633-3.11?expand=1]|
|[16633-trunk|https://github.com/apache/cassandra/compare/trunk...beobal:16633-trunk?expand=1]|

> Only include versioned files in rat-report
> --
>
> Key: CASSANDRA-16633
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16633
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-rc2, 2.2.x, 3.0.x, 3.11.x
>
>
> Recent changes to the ant build.xml file to include running the Apache Rat 
> reporting tool break the build when there are unversioned files in the source 
> tree.
> We could modify the build to only include versioned files in the Rat report.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16633) Only include versioned files in rat-report

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-16633:

 Bug Category: Parent values: Degradation(12984)Level 1 values: Other 
Exception(12998)
   Complexity: Low Hanging Fruit
Discovered By: User Report
Fix Version/s: 3.11.x
   3.0.x
   2.2.x
   4.0-rc2
 Severity: Normal
   Status: Open  (was: Triage Needed)

> Only include versioned files in rat-report
> --
>
> Key: CASSANDRA-16633
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16633
> Project: Cassandra
>  Issue Type: Bug
>  Components: Build
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-rc2, 2.2.x, 3.0.x, 3.11.x
>
>
> Recent changes to the ant build.xml file to include running the Apache Rat 
> reporting tool break the build when there are unversioned files in the source 
> tree.
> We could modify the build to only include versioned files in the Rat report.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16633) Only include versioned files in rat-report

[Sam Tunnicliffe (Jira)]

Sam Tunnicliffe created CASSANDRA-16633:
---

 Summary: Only include versioned files in rat-report
 Key: CASSANDRA-16633
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16633
 Project: Cassandra
  Issue Type: Bug
  Components: Build
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe


Recent changes to the ant build.xml file to include running the Apache Rat 
reporting tool break the build when there are unversioned files in the source 
tree.
We could modify the build to only include versioned files in the Rat report.




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16465) Increased Read Latency With Cassandra >= 3.11.7

[Jai Bheemsen Rao Dhanwada (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332621#comment-17332621
 ] 

Jai Bheemsen Rao Dhanwada commented on CASSANDRA-16465:
---

Thank you [~nvest] [~AhmedElJAMI]

 

I tried upgrading from 3.11.6 to 3.11.9 and didn't see any perf drop with my 
application use-case and also the cassandra-stress too. I use LCS in all my 
tables (system* tables uses default shipped with Cassandra). Do you mind 
sharing your test scenarios where you see latency, so I can make sure I am not 
missing anything. 

 

[~nvest] please share your finding with 3.11.10 as well, so it will be useful 
for me to assess which version to upgrade.

> Increased Read Latency With Cassandra >= 3.11.7
> ---
>
> Key: CASSANDRA-16465
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16465
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/Local Write-Read Paths
>Reporter: Ahmed ELJAMI
>Priority: Normal
> Fix For: 3.11.x, 4.0.x
>
> Attachments: Screenshot from 2021-04-24 20-50-47.png
>
>
> After upgrading Cassandra from 3.11.3 to 3.11.9, Cassandra read latency 99% 
> increased significantly. Getting back to 3.11.3 immediately fixed the issue.
> I have observed "SStable reads" increases after upgrading to 3.11.9.
> The same behavior was observed by some other users: 
> [https://www.mail-archive.com/user@cassandra.apache.org/msg61247.html]
> According to Paulo Motta's comment, this behavior may be caused by 
> https://issues.apache.org/jira/browse/CASSANDRA-15690 which was introduced on 
> 3.11.7 and removed an optimization that may cause a correctness issue when 
> there are partition deletions.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Benedict Elliott Smith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332600#comment-17332600
 ] 

Benedict Elliott Smith commented on CASSANDRA-16558:


bq. it doesn't appear to use java/perl regexes

It looks like you can use JDK regular expressions after all (see 
https://ant.apache.org/manual/Types/regexp.html)

So it should be quite easy to translate the exclusions into a regex to produce 
an inclusion file.

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16558) Fix rat check (April 2021)

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332567#comment-17332567
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16558 at 4/26/21, 4:28 PM:
---

bq. They would not fail due to untracked files on CI machines, though.

Yes, but they will fail in CI because someone didn't fail the build locally, 
unfortunately :(

Just imagine a new contributor who haven't even heard of rat. The more this 
could be automated and made easy for everyone (to be clear, the intention 
wasn't to make people struggle and any feedback/PRs for improvement are more 
than welcome!) experienced or not, the better IMHO

 


was (Author: e.dimitrova):
 

 bq. They would not fail due to untracked files on CI machines, though.

Yes, but they will fail in CI because someone didn't fail the build locally, 
unfortunately :(

Just imagine a new contributor who haven't even heard of rat. The more this 
could be automated and made easy for everyone (to be clear, the intention 
wasn't to make people struggle and any feedback/PRs for improvement are more 
than welcome!) experienced or not, the better IMHO

 

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16558) Fix rat check (April 2021)

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332567#comment-17332567
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16558 at 4/26/21, 4:27 PM:
---

{quote}{quote} They would not fail due to untracked files on CI machines, 
though.
{quote}{quote}
Yes, but they will fail in CI because someone didn't fail the build locally, 
unfortunately :(

Just imagine a new contributor who haven't even heard of rat. The more this 
could be automated and made easy for everyone (to be clear, the intention 
wasn't to make people struggle and any feedback/PRs for improvement are more 
than welcome!) experienced or not, the better IMHO

 


was (Author: e.dimitrova):
{quote}bq. They would not fail due to untracked files on CI machines, though.
{quote}
Yes, but they will fail in CI because someone didn't fail the build locally, 
unfortunately :( 

 

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16558) Fix rat check (April 2021)

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332567#comment-17332567
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16558 at 4/26/21, 4:27 PM:
---

 

 bq. They would not fail due to untracked files on CI machines, though.

Yes, but they will fail in CI because someone didn't fail the build locally, 
unfortunately :(

Just imagine a new contributor who haven't even heard of rat. The more this 
could be automated and made easy for everyone (to be clear, the intention 
wasn't to make people struggle and any feedback/PRs for improvement are more 
than welcome!) experienced or not, the better IMHO

 


was (Author: e.dimitrova):
{quote}{quote} They would not fail due to untracked files on CI machines, 
though.
{quote}{quote}
Yes, but they will fail in CI because someone didn't fail the build locally, 
unfortunately :(

Just imagine a new contributor who haven't even heard of rat. The more this 
could be automated and made easy for everyone (to be clear, the intention 
wasn't to make people struggle and any feedback/PRs for improvement are more 
than welcome!) experienced or not, the better IMHO

 

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332567#comment-17332567
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16558:
-

{quote}bq. They would not fail due to untracked files on CI machines, though.
{quote}
Yes, but they will fail in CI because someone didn't fail the build locally, 
unfortunately :( 

 

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Benedict Elliott Smith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332565#comment-17332565
 ] 

Benedict Elliott Smith commented on CASSANDRA-16558:


So, it's possible to dump the files in tree using the following:

{code}








{code}

It would be possible to dump this to a temporary file, and to then run it 
through some [filters|https://ant.apache.org/manual/Types/filterchain.html]. It 
looks a little painful, and poorly documented - and it doesn't appear to use 
java/perl regexes. But something like this would require no work on the part of 
the user, if we can figure out the precise incantation.

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332564#comment-17332564
 ] 

Michael Semb Wever commented on CASSANDRA-16558:


bq. Ideally, rat-check would only check tracked files. Is there anything we can 
do to narrow its scope?

Using a custom or script selector in the ant fileset. But I'm not keen to be 
adding java classes to customise our ant build. 

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Sam Tunnicliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332561#comment-17332561
 ] 

Sam Tunnicliffe commented on CASSANDRA-16558:
-

bq. Ideally, rat-check would only check tracked files. Is there anything we can 
do to narrow its scope?

There is, I think. I'll post a patch shortly

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332563#comment-17332563
 ] 

Brandon Williams commented on CASSANDRA-16632:
--

[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/710/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/710/pipeline]
[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/711/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/711/pipeline]


> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Assignee: Matt Fleming
>Priority: Normal
> Fix For: 4.0-rc
>
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.
> Patch here: 
> https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332556#comment-17332556
 ] 

Michael Semb Wever commented on CASSANDRA-16558:


Like this: 
https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/rat.skip/trunk
 

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332555#comment-17332555
 ] 

Brandon Williams commented on CASSANDRA-16558:
--

bq. Then I am concerned we will have failing builds more often.

They would not fail due to untracked files on CI machines, though.

bq. The rat check is important if we want to keep our trunk always ready for a 
release to be cut.

This is a good point, and perhaps reducing the execution frequency for 
rat-check is not the right approach, but I think we'll have continual problems 
trying to exclude everything via .gitignore, especially when rat uses ant-style 
globs, meaning we don't have a universal .gitignore format.

Ideally,  rat-check would only check tracked files. Is there anything we can do 
to narrow its scope?


> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16630) Migrate to JUnit5

[John B (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332547#comment-17332547
 ] 

John B commented on CASSANDRA-16630:


Checking in here. I chimed in on the slack regarding the hybrid transition 
using junit5 vintage and jupiter engines side-by-side.

I have experience doing similar conversions (junit4 -> junit5 vintage + jupiter 
-> junit5 jupiter) and am happy to help in this effort. I have not used Ant in 
>10 years, so on that side, I probably can't be much help (I use gradle and 
maven).

 

> Migrate to JUnit5
> -
>
> Key: CASSANDRA-16630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16630
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Alexey Zotov
>Priority: Normal
>
> h3. Overview
> Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
> version 4.13.2 which we could update to. However, JUnit4 is generally 
> considered to be outdated and it is reasonable to migrate to JUnit5.
> Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, 
> ect), there are no blockers that push us to move from JUnit4. The main 
> motivation for this initiative is rule of thumb to use up-to-date versions of 
> the dependencies.
> Obviously this change is not backward compatible with the open PRs and 
> previous C* versions. Therefore, it will require an additional effort for 
> backporting the changes and updating PRs that have tests. However, I believe 
> it should not be a blocker for this initiative.
> h3. Scope (preliminary list)
> # change JUnit4 to JUnit5 dependencies and make necessary changes in ant 
> tasks (https://ant.apache.org/manual/Tasks/junitlauncher.html)
> # update syntax in all tests (imports, Before/After annotations, etc)
> # update parameterized tests
> # create a new version of {{OrderedJUnit4ClassRunner}} and update 
> corresponding tests
> # update tests that use {{BMUnitRunner}} (as per 
> https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)
> # update tests with {{@Rule}}
> # update tests with expected exceptions
> h3. Order of operations
> In order to make the transition more smooth we want to use a phased approach:
> # migrate to JUnit5 with [Vintage 
> Engine|https://junit.org/junit5/docs/current/user-guide/#dependency-metadata-junit-vintage],
>  so all JUnit4 tests work as is
> # update tests in a few bunches (to not have a huge single PR with numerous 
> conflicts)
> # disable (remove dependency) Vintage Engine, so only JUnit5 tests work



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332540#comment-17332540
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16558:
-

{quote}We could. But folk do commit without using CI, and this is one of those 
things that is off-the-radar for most
{quote}
That is also a valid point I share with [~mck]

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16629) Rafactor tests for NodeTool commands

[Benjamin Lerer (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332539#comment-17332539
 ] 

Benjamin Lerer commented on CASSANDRA-16629:


{quote}should I migrate tests to assertj?{quote}

We had a clear agreement on the mailing list on AssertJ. I do not know how 
Hamcrest was introduced.
In my opinion we should standardize on AssertJ. Having to adapt to different 
test frameworks when working on a patch is a bit painful. 


> Rafactor tests for NodeTool commands
> 
>
> Key: CASSANDRA-16629
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16629
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/unit
>Reporter: Alexey Zotov
>Assignee: Alexey Zotov
>Priority: Low
> Fix For: 4.0.x
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> h3. Overview
> As a part of CASSANDRA-16404, I've been working on {{NodeTool}} commands for 
> invalidation of the auth caches. While working on that, I've mentioned that 
> the existing NodeTool unit tests are written in a different manner and reside 
> in wrong packages. I've made the following changes to them:
> # moved them to the proper packages (that match original command class)
> # unified naming, formatting, imports
> # migrated from {{org.assertj}} to {{hamcrest}} for assertions (though it is 
> not smth really required)
> # made a general cleanup
> PR: https://github.com/apache/cassandra/pull/982



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16558) Fix rat check (April 2021)

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332532#comment-17332532
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16558 at 4/26/21, 3:48 PM:
---

{quote}I think we're going to keep having issues like this while rat-check is 
run by default. Would running it only as part of CI not be enough?
{quote}
Then I am concerned we will have failing builds more often. Even if it is just 
on the Jenkins dev... 

What I am thinking is, maybe we can add some parameter (I think that sounds as 
a good idea [~mck] ); make rat check mandatory but further to the report, also 
to have a warning about that parameter that it can be used in cases like those 
already mentioned.

The rat check is important if we want to keep our trunk always ready for a 
release to be cut.


was (Author: e.dimitrova):
{quote}I think we're going to keep having issues like this while rat-check is 
run by default. Would running it only as part of CI not be enough?
{quote}
Then I am concerned we will have failing builds more often. Even if it is just 
on the dev branch... 

What I am thinking is, maybe we can add some parameter (I think that sounds as 
a good idea [~mck] ); make rat check mandatory but further to the report, also 
to have a warning about that parameter that it can be used in cases like those 
already mentioned.

The rat check is important if we want to keep our trunk always ready for a 
release to be cut.

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332532#comment-17332532
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16558:
-

{quote}I think we're going to keep having issues like this while rat-check is 
run by default. Would running it only as part of CI not be enough?
{quote}
Then I am concerned we will have failing builds more often. Even if it is just 
on the dev branch... 

What I am thinking is, maybe we can add some parameter (I think that sounds as 
a good idea [~mck] ); make rat check mandatory but further to the report, also 
to have a warning about that parameter that it can be used in cases like those 
already mentioned.

The rat check is important if we want to keep our trunk always ready for a 
release to be cut.

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332530#comment-17332530
 ] 

Michael Semb Wever commented on CASSANDRA-16558:


bq. Would running it only as part of CI not be enough?

We could. But folk do commit without using CI, and this is one of those things 
that is off-the-radar for most.

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-website] branch asf-site updated (c017c1e -> 83ba244)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git.


 discard c017c1e  hack in plausible tracking with
 discard 486602d  generate docs for 637ee64f
omit 637ee64  Update "Beta" terminology to "RC" for 4.0-rc1  Also add 
src/doc/4.0-beta4
omit 33679b9  Cassandra release 4.0-rc1 
https://lists.apache.org/thread.html/rb814c1227ab460eab47fda386a3f9b55536a676c9a7ee78c0086f317%40%3Cdev.cassandra.apache.org%3E
 add f9df550  generate docs for 2d94e2b9
 add 18322e9  Latest html website design from Joshua, Paul, Melissa and co
 add bb53788  hack new design website with latest 4.0-rc1 release
 new 83ba244  hack in plausible tracking with

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (c017c1e)
\
 N -- N -- N   refs/heads/asf-site (83ba244)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 content/.htaccess  |23 +
 .../Apache-Cassandra-Changelog-1-October-2020.html |   311 +
 ...Apache-Cassandra-Changelog-2-December-2020.html |   309 +
 .../Apache-Cassandra-Changelog-3-January-2021.html |   292 +
 ...Apache-Cassandra-Changelog-4-February-2021.html |   325 +
 .../Apache-Cassandra-Changelog-5-March-2021.html   |   315 +
 .../Apache-Cassandra-Changelog-6-April-2021.html   |   315 +
 .../blog/Apache-Cassandra-Usage-Report-2020.html   |   327 +
 .../blog/Audit-Logging-in-Apache-Cassandra-4.html  |   433 +
 ...andra-and-Kubernetes-SIG-Update-and-Survey.html |   304 +
 ...ty-with-5x-Faster-Streaming-in-Cassandra-4.html |   308 +
 ...ra's-Internals-with-Property-based-Testing.html |   338 +
 ...-Zero-Copy-Streaming-in-Apache-Cassandra-4.html |   301 +
 ...che-Cassandras-Front-Door-and-Backpressure.html |   330 +
 ...assandra-4-Beta-Battle-Tested-From-Day-One.html |   287 +
 .../blog/Introducing-Transient-Replication.html|   363 +
 content/blog/Join-Casssandra-GSoC-2021.html|   254 +
 ...ced-for-April-28-Cassandra-4.0-World-Party.html |   263 +
 content/blog/Testing-Apache-Cassandra-4.html   |   277 +
 content/blog/World-Party.html  |   260 +
 content/blog/index.html|   470 +-
 content/blog/template.html |   228 +
 content/case-studies/index.html|   952 ++
 content/cassandra-basics/index.html|   284 +
 content/community/index.html   |   641 +-
 content/doc/4.0-beta4/_images/Figure_1_backups.jpg |   Bin 38551 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_1_data_model.jpg  |   Bin 17469 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_1_guarantees.jpg  |   Bin 17993 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_1_read_repair.jpg |   Bin 36919 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_2_data_model.jpg  |   Bin 20925 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_2_read_repair.jpg |   Bin 45595 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_3_read_repair.jpg |   Bin 43021 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_4_read_repair.jpg |   Bin 43021 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_5_read_repair.jpg |   Bin 42560 -> 0 bytes
 .../doc/4.0-beta4/_images/Figure_6_read_repair.jpg |   Bin 57489 -> 0 bytes
 .../_images/data_modeling_chebotko_logical.png |   Bin 87366 -> 0 bytes
 .../_images/data_modeling_chebotko_physical.png|   Bin 4553809 -> 0 bytes
 .../_images/data_modeling_hotel_bucketing.png  |   Bin 22009 -> 0 bytes
 .../4.0-beta4/_images/data_modeling_hotel_erd.png  |   Bin 233309 -> 0 bytes
 .../_images/data_modeling_hotel_logical.png|   Bin 116998 -> 0 bytes
 .../_images/data_modeling_hotel_physical.png   |   Bin 119795 -> 0 bytes
 .../_images/data_modeling_hotel_queries.png|   Bin 103940 -> 0 bytes
 .../_images/data_modeling_hotel_relational.png |   Bin 102656 -> 0 bytes
 .../_images/data_modeling_reservation_logical.png  |   Bin 121750 -> 0 bytes
 .../_images/data_modeling_reservation_physical.png |   Bin 142416 -> 0 bytes
 content/doc/4.0-beta4/_images/docs_commit.png  |   Bin 104667 -> 0 bytes
 .../doc/4.0-beta4/_images/docs_create_branch.png   |   Bin 181860 -> 0 bytes
 cont

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332512#comment-17332512
 ] 

Brandon Williams commented on CASSANDRA-16558:
--

bq. Another idea, and i'm brain-storming quickly, is an easy way to skip the 
rat-check. e.g. either file-based or property based.

I think we're going to keep having issues like this while rat-check is run by 
default.  Would running it only as part of CI not be enough?

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16630) Migrate to JUnit5

[Alexey Zotov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zotov updated CASSANDRA-16630:
-
Description: 
h3. Overview
Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
version 4.13.2 which we could update to. However, JUnit4 is generally 
considered to be outdated and it is reasonable to migrate to JUnit5.

Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, ect), 
there are no blockers that push us to move from JUnit4. The main motivation for 
this initiative is rule of thumb to use up-to-date versions of the dependencies.

Obviously this change is not backward compatible with the open PRs and previous 
C* versions. Therefore, it will require an additional effort for backporting 
the changes and updating PRs that have tests. However, I believe it should not 
be a blocker for this initiative.

h3. Scope (preliminary list)
# change JUnit4 to JUnit5 dependencies and make necessary changes in ant tasks 
(https://ant.apache.org/manual/Tasks/junitlauncher.html)
# update syntax in all tests (imports, Before/After annotations, etc)
# update parameterized tests
# create a new version of {{OrderedJUnit4ClassRunner}} and update corresponding 
tests
# update tests that use {{BMUnitRunner}} (as per 
https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)
# update tests with {{@Rule}}
# update tests with expected exceptions

h3. Order of operations
In order to make the transition more smooth we want to use a phased approach:
# migrate to JUnit5 with [Vintage 
Engine|https://junit.org/junit5/docs/current/user-guide/#dependency-metadata-junit-vintage],
 so all JUnit4 tests work as is
# update tests in a few bunches (to not have a huge single PR with numerous 
conflicts)
# disable (remove dependency) Vintage Engine, so only JUnit5 tests work

  was:
h3. Overview
Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
version 4.13.2 which we could update to. However, JUnit4 is generally 
considered to be outdated and it is reasonable to migrate to JUnit5.

Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, ect), 
there are no blockers that push us to move from JUnit4. The main motivation for 
this initiative is rule of thumb to use up-to-date versions of the dependencies.

Obviously this change is not backward compatible with the open PRs and previous 
C* versions. Therefore, it will require an additional effort for backporting 
the changes and updating PRs that have tests. However, I believe it should not 
be a blocker for this initiative.

h3. Scope (preliminary list)
# change JUnit4 to JUnit5 dependencies and make necessary changes in ant tasks 
(https://ant.apache.org/manual/Tasks/junitlauncher.html)
# update syntax in all tests (imports, Before/After annotations, etc)
# update parameterized tests
# create a new version of {{OrderedJUnit4ClassRunner}} and update corresponding 
tests
# update tests that use {{BMUnitRunner}} (as per 
https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)
# update tests with {{@Rule}}
# update tests with expected exceptions

h3. Order of operations
In order to make the transition more smooth we want to use a phased approach:
# migrate to JUnit5 with [Vintage 
Engine|https://junit.org/junit5/docs/current/user-guide/#dependency-metadata-junit-vintage],
 so all JUnit4 tests work as is
# update tests in a few bunches (to not have a huge single PR with a bunch of 
conflicts)
# disable (remove dependency) Vintage Engine, so only JUnit5 tests work


> Migrate to JUnit5
> -
>
> Key: CASSANDRA-16630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16630
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Alexey Zotov
>Priority: Normal
>
> h3. Overview
> Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
> version 4.13.2 which we could update to. However, JUnit4 is generally 
> considered to be outdated and it is reasonable to migrate to JUnit5.
> Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, 
> ect), there are no blockers that push us to move from JUnit4. The main 
> motivation for this initiative is rule of thumb to use up-to-date versions of 
> the dependencies.
> Obviously this change is not backward compatible with the open PRs and 
> previous C* versions. Therefore, it will require an additional effort for 
> backporting the changes and updating PRs that have tests. However, I believe 
> it should not be a blocker for this initiative.
> h3. Scope (preliminary list)
> # change JUnit4 to JUnit5 dependencies and make necessary changes in ant 
> tasks (https://ant.apache.org/manual/Tasks/junitlauncher.html)
> # update syntax in all tests (imports, Before/After annotations, etc)
> #

[jira] [Updated] (CASSANDRA-16605) The dependencies in the Cassandra DEB package prevent installation on Ubuntu 20.04+

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16605:
-
Fix Version/s: (was: 4.0.x)
   3.11.11
   3.0.25

> The dependencies in the Cassandra DEB package prevent installation on Ubuntu 
> 20.04+
> ---
>
> Key: CASSANDRA-16605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16605
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Michael Kelly
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 3.0.25, 3.11.11
>
>
> The DEB package for Cassandra has a dependency on 'python >= 2.7'.
> In Ubuntu 18.04 there is a package called 'python' which installs python2.7.
> This package has been removed in Ubuntu 20.04 so the dependency cannot be 
> satisified.
>  
> Proposed solution:
> Change the dependency 'python >= 2.7' to something like 'python2 >= 2.7 OR 
> python3 >= 3.6'.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16605) The dependencies in the Cassandra DEB package prevent installation on Ubuntu 20.04+

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16605:
-
Test and Documentation Plan: CI
 Status: Patch Available  (was: Open)

> The dependencies in the Cassandra DEB package prevent installation on Ubuntu 
> 20.04+
> ---
>
> Key: CASSANDRA-16605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16605
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Michael Kelly
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> The DEB package for Cassandra has a dependency on 'python >= 2.7'.
> In Ubuntu 18.04 there is a package called 'python' which installs python2.7.
> This package has been removed in Ubuntu 20.04 so the dependency cannot be 
> satisified.
>  
> Proposed solution:
> Change the dependency 'python >= 2.7' to something like 'python2 >= 2.7 OR 
> python3 >= 3.6'.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16605) The dependencies in the Cassandra DEB package prevent installation on Ubuntu 20.04+

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332504#comment-17332504
 ] 

Brandon Williams commented on CASSANDRA-16605:
--


||Patch||CI||
|[3.0|https://github.com/driftx/cassandra/tree/CASSANDRA-16605]|[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/712/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/712/pipeline]|
|[3.11|https://github.com/driftx/cassandra/tree/CASSANDRA-16605-3.11]|[!https://ci-cassandra.apache.org/job/Cassandra-devbranch/713/badge/icon!|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/713/pipeline]|


> The dependencies in the Cassandra DEB package prevent installation on Ubuntu 
> 20.04+
> ---
>
> Key: CASSANDRA-16605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16605
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Michael Kelly
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> The DEB package for Cassandra has a dependency on 'python >= 2.7'.
> In Ubuntu 18.04 there is a package called 'python' which installs python2.7.
> This package has been removed in Ubuntu 20.04 so the dependency cannot be 
> satisified.
>  
> Proposed solution:
> Change the dependency 'python >= 2.7' to something like 'python2 >= 2.7 OR 
> python3 >= 3.6'.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-16605) The dependencies in the Cassandra DEB package prevent installation on Ubuntu 20.04+

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams reassigned CASSANDRA-16605:


Assignee: Brandon Williams  (was: Erick Ramirez)

> The dependencies in the Cassandra DEB package prevent installation on Ubuntu 
> 20.04+
> ---
>
> Key: CASSANDRA-16605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16605
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Michael Kelly
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> The DEB package for Cassandra has a dependency on 'python >= 2.7'.
> In Ubuntu 18.04 there is a package called 'python' which installs python2.7.
> This package has been removed in Ubuntu 20.04 so the dependency cannot be 
> satisified.
>  
> Proposed solution:
> Change the dependency 'python >= 2.7' to something like 'python2 >= 2.7 OR 
> python3 >= 3.6'.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16630) Migrate to JUnit5

[Alexey Zotov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zotov updated CASSANDRA-16630:
-
Description: 
h3. Overview
Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
version 4.13.2 which we could update to. However, JUnit4 is generally 
considered to be outdated and it is reasonable to migrate to JUnit5.

Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, ect), 
there are no blockers that push us to move from JUnit4. The main motivation for 
this initiative is rule of thumb to use up-to-date versions of the dependencies.

Obviously this change is not backward compatible with the open PRs and previous 
C* versions. Therefore, it will require an additional effort for backporting 
the changes and updating PRs that have tests. However, I believe it should not 
be a blocker for this initiative.

h3. Scope (preliminary list)
# change JUnit4 to JUnit5 dependencies and make necessary changes in ant tasks 
(https://ant.apache.org/manual/Tasks/junitlauncher.html)
# update syntax in all tests (imports, Before/After annotations, etc)
# update parameterized tests
# create a new version of {{OrderedJUnit4ClassRunner}} and update corresponding 
tests
# update tests that use {{BMUnitRunner}} (as per 
https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)
# update tests with {{@Rule}}
# update tests with expected exceptions

h3. Order of operations
In order to make the transition more smooth we want to use a phased approach:
# migrate to JUnit5 with [Vintage 
Engine|https://junit.org/junit5/docs/current/user-guide/#dependency-metadata-junit-vintage],
 so all JUnit4 tests work as is
# update tests in a few bunches (to not have a huge single PR with a bunch of 
conflicts)
# disable (remove dependency) Vintage Engine, so only JUnit5 tests work

  was:
h3. Overview
Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
version 4.13.2 which we could update to. However, JUnit4 is generally 
considered to be outdated and it is reasonable to migrate to JUnit5.

Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, ect), 
there are no blockers that push us to move from JUnit4. The main motivation for 
this initiative is rule of thumb to use up-to-date versions of the dependencies.

Obviously this change is not backward compatible with the open PRs and previous 
C* versions. Therefore, it will require an additional effort for backporting 
the changes and updating PRs that have tests. However, I believe it should not 
be a blocker for this initiative.

h3. Scope (preliminary list)
# change JUnit4 to JUnit5 dependencies and make necessary changes in ant tasks 
(https://ant.apache.org/manual/Tasks/junitlauncher.html)
# update syntax in all tests (imports, Before/After annotations, etc)
# update parameterized tests
# create a new version of {{OrderedJUnit4ClassRunner}} and update corresponding 
tests
# update tests that use {{BMUnitRunner}} (as per 
https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)
# update tests with {{@Rule}}
# update tests with expected exceptions

h3. Order of operations
In order to make the transition more smooth we want to use a phased approach:
1. migrate to JUnit5 with [Vintage 
Engine|https://junit.org/junit5/docs/current/user-guide/#dependency-metadata-junit-vintage],
 so all JUnit4 tests work as is
2. update tests in a few bunches (to not have a huge single PR with a bunch of 
conflicts)
3. disable (remove dependency) Vintage Engine, so only JUnit5 tests work


> Migrate to JUnit5
> -
>
> Key: CASSANDRA-16630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16630
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Alexey Zotov
>Priority: Normal
>
> h3. Overview
> Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
> version 4.13.2 which we could update to. However, JUnit4 is generally 
> considered to be outdated and it is reasonable to migrate to JUnit5.
> Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, 
> ect), there are no blockers that push us to move from JUnit4. The main 
> motivation for this initiative is rule of thumb to use up-to-date versions of 
> the dependencies.
> Obviously this change is not backward compatible with the open PRs and 
> previous C* versions. Therefore, it will require an additional effort for 
> backporting the changes and updating PRs that have tests. However, I believe 
> it should not be a blocker for this initiative.
> h3. Scope (preliminary list)
> # change JUnit4 to JUnit5 dependencies and make necessary changes in ant 
> tasks (https://ant.apache.org/manual/Tasks/junitlauncher.html)
> # update syntax in all tests (imports, Before/After annotations, etc

[jira] [Updated] (CASSANDRA-16630) Migrate to JUnit5

[Alexey Zotov (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Zotov updated CASSANDRA-16630:
-
Description: 
h3. Overview
Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
version 4.13.2 which we could update to. However, JUnit4 is generally 
considered to be outdated and it is reasonable to migrate to JUnit5.

Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, ect), 
there are no blockers that push us to move from JUnit4. The main motivation for 
this initiative is rule of thumb to use up-to-date versions of the dependencies.

Obviously this change is not backward compatible with the open PRs and previous 
C* versions. Therefore, it will require an additional effort for backporting 
the changes and updating PRs that have tests. However, I believe it should not 
be a blocker for this initiative.

h3. Scope (preliminary list)
# change JUnit4 to JUnit5 dependencies and make necessary changes in ant tasks 
(https://ant.apache.org/manual/Tasks/junitlauncher.html)
# update syntax in all tests (imports, Before/After annotations, etc)
# update parameterized tests
# create a new version of {{OrderedJUnit4ClassRunner}} and update corresponding 
tests
# update tests that use {{BMUnitRunner}} (as per 
https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)
# update tests with {{@Rule}}
# update tests with expected exceptions

h3. Order of operations
In order to make the transition more smooth we want to use a phased approach:
1. migrate to JUnit5 with [Vintage 
Engine|https://junit.org/junit5/docs/current/user-guide/#dependency-metadata-junit-vintage],
 so all JUnit4 tests work as is
2. update tests in a few bunches (to not have a huge single PR with a bunch of 
conflicts)
3. disable (remove dependency) Vintage Engine, so only JUnit5 tests work

  was:
h3. Overview
Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
version 4.13.2 which we could update to. However, JUnit4 is generally 
considered to be outdated and it is reasonable to migrate to JUnit5.

Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, ect), 
there are no blockers that push us to move from JUnit4. The main motivation for 
this initiative is rule of thumb to use up-to-date versions of the dependencies.

Obviously this change is not backward compatible with the open PRs and previous 
C* versions. Therefore, it will require an additional effort for backporting 
the changes and updating PRs that have tests. However, I believe it should not 
be a blocker for this initiative.

h3. Action items (preliminary list)
# change JUnit4 to JUnit5 dependencies and make necessary changes in ant tasks 
(https://ant.apache.org/manual/Tasks/junitlauncher.html)
# update syntax in all tests (imports, Before/After annotations, etc)
# update parameterized tests
# create a new version of {{OrderedJUnit4ClassRunner}} and update corresponding 
tests
# update tests that use {{BMUnitRunner}} (as per 
https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)



> Migrate to JUnit5
> -
>
> Key: CASSANDRA-16630
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16630
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/unit
>Reporter: Alexey Zotov
>Priority: Normal
>
> h3. Overview
> Currently C* uses JUnit4 (version 4.12) which is obsolete. There is a newer 
> version 4.13.2 which we could update to. However, JUnit4 is generally 
> considered to be outdated and it is reasonable to migrate to JUnit5.
> Despite of having a syntax sugar in JUnit5 (assertThrow, lamda's support, 
> ect), there are no blockers that push us to move from JUnit4. The main 
> motivation for this initiative is rule of thumb to use up-to-date versions of 
> the dependencies.
> Obviously this change is not backward compatible with the open PRs and 
> previous C* versions. Therefore, it will require an additional effort for 
> backporting the changes and updating PRs that have tests. However, I believe 
> it should not be a blocker for this initiative.
> h3. Scope (preliminary list)
> # change JUnit4 to JUnit5 dependencies and make necessary changes in ant 
> tasks (https://ant.apache.org/manual/Tasks/junitlauncher.html)
> # update syntax in all tests (imports, Before/After annotations, etc)
> # update parameterized tests
> # create a new version of {{OrderedJUnit4ClassRunner}} and update 
> corresponding tests
> # update tests that use {{BMUnitRunner}} (as per 
> https://developer.jboss.org/docs/DOC-52953 it supports JUnit5)
> # update tests with {{@Rule}}
> # update tests with expected exceptions
> h3. Order of operations
> In order to make the transition more smooth we want to use a phased approach:
> 1. migrate to JUnit5 with [Vintage 
> Engine|https://junit.org/j

[jira] [Comment Edited] (CASSANDRA-16558) Fix rat check (April 2021)

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332498#comment-17332498
 ] 

Michael Semb Wever edited comment on CASSANDRA-16558 at 4/26/21, 3:27 PM:
--

[~benedict], i tried a number of approaches. The layout of the Cassandra 
directory makes this difficult (there's source files in different locations and 
build/generated files in numerous places. Problem with variations that are 
friendly is that we go back to letting files slip through, getting committed 
without the license header and being the release manager's headache.

Would including a user-based global {{~/.gitignore}} be a way we could get 
around this?
e.g. 
{code}
git config --global core.excludesFile '~/.gitignore'
{code}

Another idea, and i'm brain-storming quickly, is an easy way to skip the 
rat-check. e.g. either file-based or property based.


was (Author: michaelsembwever):
[~benedict], i tried a number of approaches. The layout of the Cassandra 
directory makes this difficult (there's source files in different locations and 
build/generated files in numerous places. Problem with variations that are 
friendly is that we go back to letting files slip through, getting committed 
without the license header and being the release manager's headache.

Would including a user-based global {{~/.gitignore}} be a way we could get 
around this?
e.g. 
{code}
git config --global core.excludesFile '~/.gitignore'
{code}

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16558) Fix rat check (April 2021)

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332498#comment-17332498
 ] 

Michael Semb Wever edited comment on CASSANDRA-16558 at 4/26/21, 3:25 PM:
--

[~benedict], i tried a number of approaches. The layout of the Cassandra 
directory makes this difficult (there's source files in different locations and 
build/generated files in numerous places. Problem with variations that are 
friendly is that we go back to letting files slip through, getting committed 
without the license header and being the release manager's headache.

Would including a user-based global {{~/.gitignore}} be a way we could get 
around this?
e.g. 
{code}
git config --global core.excludesFile '~/.gitignore'
{code}


was (Author: michaelsembwever):
[~benedict], i tried a number of approaches. The layout of the Cassandra 
directory makes this difficult (there's source files in different locations and 
build/generated files in numerous places. Problem with variations that are 
friendly is that we go back to letting files slip through, getting committed 
without the license header and being the release manager's headache.

Is using a user-based global {{~/.gitignore}} a way we could get around this?
e.g.
{code}
git config --global core.excludesFile '~/.gitignore'
{code}

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332498#comment-17332498
 ] 

Michael Semb Wever commented on CASSANDRA-16558:


[~benedict], i tried a number of approaches. The layout of the Cassandra 
directory makes this difficult (there's source files in different locations and 
build/generated files in numerous places. Problem with variations that are 
friendly is that we go back to letting files slip through, getting committed 
without the license header and being the release manager's headache.

Is using a user-based global {{~/.gitignore}} a way we could get around this?
e.g.
{code}
git config --global core.excludesFile '~/.gitignore'
{code}

> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16619) Loss of commit log data possible after sstable ingest

[Benjamin Lerer (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332479#comment-17332479
 ] 

Benjamin Lerer edited comment on CASSANDRA-16619 at 4/26/21, 3:07 PM:
--

As {{4.0-rc1}} has been released we need to provide forward compatibility for 
data. I do not think it is the case with the current patch. Users using 
{{4.0-rc1}} will have an {{na}} format *without* hostID whereas when they will 
deploy {{4.0-rc2}} they will have an {{na}} format *with* hostID which will 
fail to read the {{na}} SSTables from the {{4.0-rc1}}. To fix that I think that 
we need to introduce a new {{nb}} version.

Regarding the patches, we should refactor the 
{{MetadataSerializerTest.testXReadY}} tests to ensure that they test all the 
possible combinations not only some of them.


was (Author: blerer):
As {{4.0-rc1}} has been released we need to provide forward compatibility for 
data. I do not think it is the case with the current patch. Users using 
{{4.0-rc1}} will have an {{na}} format *without* hostID whereas when they will 
deploy {{4.0-rc2}} they will have an {{na}} format *with* hostID which will 
fail to read the {{na}} SSTables from the {{4.0-rc1}}. To fix that I think that 
we need to introduce a new {{nb}} version.

Regarding the patches should refactor the {{MetadataSerializerTest.testXReadY}} 
tests to ensure that they test all the possible combinations not only some of 
them.

> Loss of commit log data possible after sstable ingest
> -
>
> Key: CASSANDRA-16619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16619
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Commit Log
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> SSTable metadata contains commit log positions of the sstable. These 
> positions are used to filter out mutations from the commit log on restart and 
> only make sense for the node on which the data was flushed.
> If an SSTable is moved between nodes they may cover regions that the 
> receiving node has not yet flushed, and result in valid data being lost 
> should these sections of the commit log need to be replayed.
> Solution:
> The chosen solution introduces a new sstable metadata (StatsMetadata) - 
> originatingHostId (UUID), which is the local host id of the node on which the 
> sstable was created, or null if not known. Commit log intervals from an 
> sstable are taken into account during Commit Log replay only when the 
> originatingHostId of the sstable matches the local node's hostId.
> For new sstables the originatingHostId is set according to StorageService's 
> local hostId.
> For compacted sstables the originatingHostId set according to 
> StorageService's local hostId, and only commit log intervals from local 
> sstables is preserved in the resulting sstable.
> discovered by [~jakubzytka]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16619) Loss of commit log data possible after sstable ingest

[Benjamin Lerer (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332479#comment-17332479
 ] 

Benjamin Lerer edited comment on CASSANDRA-16619 at 4/26/21, 3:06 PM:
--

As {{4.0-rc1}} has been released by consequence we need to provide forward 
compatibility for data. I do not think it is the case with the current patch. 
Users using {{4.0-rc1}} will have an {{na}} format *without* hostID whereas 
when they will deploy {{4.0-rc2}} they will have an {{na}} format *with* hostID 
which will fail to read the {{na}} SSTables from the {{4.0-rc1}}. To fix that I 
think that we need to introduce a new {{nb}} version.

Regarding the patches should refactor the {{MetadataSerializerTest.testXReadY}} 
tests to ensure that they test all the possible combinations not only some of 
them.


was (Author: blerer):
As {{4.0-rc1}} has been released by consequence we need to provide forward 
compatibility for data. I do not think it is the case with the current patch. 
Users using {{4.0-rc1}} will have an {{na}} format *without* hostID whereas 
when they will deploy {{4.0-rc2}} they will have an {{na}} format *with* hostID 
which will fail to read the {{na}} SSTables from the {{4.0-rc1}}. To fix that I 
think that we need to introduce a new {{nb}} version.

Regarding the patches should refactor the testMcReadMc

> Loss of commit log data possible after sstable ingest
> -
>
> Key: CASSANDRA-16619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16619
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Commit Log
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> SSTable metadata contains commit log positions of the sstable. These 
> positions are used to filter out mutations from the commit log on restart and 
> only make sense for the node on which the data was flushed.
> If an SSTable is moved between nodes they may cover regions that the 
> receiving node has not yet flushed, and result in valid data being lost 
> should these sections of the commit log need to be replayed.
> Solution:
> The chosen solution introduces a new sstable metadata (StatsMetadata) - 
> originatingHostId (UUID), which is the local host id of the node on which the 
> sstable was created, or null if not known. Commit log intervals from an 
> sstable are taken into account during Commit Log replay only when the 
> originatingHostId of the sstable matches the local node's hostId.
> For new sstables the originatingHostId is set according to StorageService's 
> local hostId.
> For compacted sstables the originatingHostId set according to 
> StorageService's local hostId, and only commit log intervals from local 
> sstables is preserved in the resulting sstable.
> discovered by [~jakubzytka]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16619) Loss of commit log data possible after sstable ingest

[Benjamin Lerer (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332479#comment-17332479
 ] 

Benjamin Lerer edited comment on CASSANDRA-16619 at 4/26/21, 3:06 PM:
--

As {{4.0-rc1}} has been released we need to provide forward compatibility for 
data. I do not think it is the case with the current patch. Users using 
{{4.0-rc1}} will have an {{na}} format *without* hostID whereas when they will 
deploy {{4.0-rc2}} they will have an {{na}} format *with* hostID which will 
fail to read the {{na}} SSTables from the {{4.0-rc1}}. To fix that I think that 
we need to introduce a new {{nb}} version.

Regarding the patches should refactor the {{MetadataSerializerTest.testXReadY}} 
tests to ensure that they test all the possible combinations not only some of 
them.


was (Author: blerer):
As {{4.0-rc1}} has been released by consequence we need to provide forward 
compatibility for data. I do not think it is the case with the current patch. 
Users using {{4.0-rc1}} will have an {{na}} format *without* hostID whereas 
when they will deploy {{4.0-rc2}} they will have an {{na}} format *with* hostID 
which will fail to read the {{na}} SSTables from the {{4.0-rc1}}. To fix that I 
think that we need to introduce a new {{nb}} version.

Regarding the patches should refactor the {{MetadataSerializerTest.testXReadY}} 
tests to ensure that they test all the possible combinations not only some of 
them.

> Loss of commit log data possible after sstable ingest
> -
>
> Key: CASSANDRA-16619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16619
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Commit Log
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> SSTable metadata contains commit log positions of the sstable. These 
> positions are used to filter out mutations from the commit log on restart and 
> only make sense for the node on which the data was flushed.
> If an SSTable is moved between nodes they may cover regions that the 
> receiving node has not yet flushed, and result in valid data being lost 
> should these sections of the commit log need to be replayed.
> Solution:
> The chosen solution introduces a new sstable metadata (StatsMetadata) - 
> originatingHostId (UUID), which is the local host id of the node on which the 
> sstable was created, or null if not known. Commit log intervals from an 
> sstable are taken into account during Commit Log replay only when the 
> originatingHostId of the sstable matches the local node's hostId.
> For new sstables the originatingHostId is set according to StorageService's 
> local hostId.
> For compacted sstables the originatingHostId set according to 
> StorageService's local hostId, and only commit log intervals from local 
> sstables is preserved in the resulting sstable.
> discovered by [~jakubzytka]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16619) Loss of commit log data possible after sstable ingest

[Benjamin Lerer (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332479#comment-17332479
 ] 

Benjamin Lerer commented on CASSANDRA-16619:


As {{4.0-rc1}} has been released by consequence we need to provide forward 
compatibility for data. I do not think it is the case with the current patch. 
Users using {{4.0-rc1}} will have an {{na}} format *without* hostID whereas 
when they will deploy {{4.0-rc2}} they will have an {{na}} format *with* hostID 
which will fail to read the {{na}} SSTables from the {{4.0-rc1}}. To fix that I 
think that we need to introduce a new {{nb}} version.

Regarding the patches should refactor the testMcReadMc

> Loss of commit log data possible after sstable ingest
> -
>
> Key: CASSANDRA-16619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16619
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Commit Log
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> SSTable metadata contains commit log positions of the sstable. These 
> positions are used to filter out mutations from the commit log on restart and 
> only make sense for the node on which the data was flushed.
> If an SSTable is moved between nodes they may cover regions that the 
> receiving node has not yet flushed, and result in valid data being lost 
> should these sections of the commit log need to be replayed.
> Solution:
> The chosen solution introduces a new sstable metadata (StatsMetadata) - 
> originatingHostId (UUID), which is the local host id of the node on which the 
> sstable was created, or null if not known. Commit log intervals from an 
> sstable are taken into account during Commit Log replay only when the 
> originatingHostId of the sstable matches the local node's hostId.
> For new sstables the originatingHostId is set according to StorageService's 
> local hostId.
> For compacted sstables the originatingHostId set according to 
> StorageService's local hostId, and only commit log intervals from local 
> sstables is preserved in the resulting sstable.
> discovered by [~jakubzytka]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16558) Fix rat check (April 2021)

[Benedict Elliott Smith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332478#comment-17332478
 ] 

Benedict Elliott Smith edited comment on CASSANDRA-16558 at 4/26/21, 2:55 PM:
--

I appreciate all the work you're putting in to cleaning up the build.xml, but 
it can cause some headaches downstream, and this one unfortunately breaks my 
local build. In my case, rat now is very unhappy if you have anything in your 
working directory that is not committed to git, never will be, but is also not 
in .gitignore (which is a shared resource, so I don't typically add any local 
directories a retain in my working directory).

I had a quick go at fixing this by producing a list of files that are checked 
into git to provide as an includesfile, but unfortunately the list of 
inclusions seems to override the list of exclusions.




was (Author: benedict):
I appreciate all the work you're putting in to cleaning up the build.xml, but 
it can cause some headaches downstream, and breaks my local build. In my case, 
rat now is very unhappy if you have anything in your working directory that is 
not committed to git, never will be, but is also not in .gitignore (which is a 
shared resource, so I don't typically add any local directories a retain in my 
working directory).

I had a quick go at fixing this by producing a list of files that are checked 
into git to provide as an includesfile, but unfortunately the list of 
inclusions seems to override the list of exclusions.



> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16558) Fix rat check (April 2021)

[Benedict Elliott Smith (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332478#comment-17332478
 ] 

Benedict Elliott Smith commented on CASSANDRA-16558:


I appreciate all the work you're putting in to cleaning up the build.xml, but 
it can cause some headaches downstream, and breaks my local build. In my case, 
rat now is very unhappy if you have anything in your working directory that is 
not committed to git, never will be, but is also not in .gitignore (which is a 
shared resource, so I don't typically add any local directories a retain in my 
working directory).

I had a quick go at fixing this by producing a list of files that are checked 
into git to provide as an includesfile, but unfortunately the list of 
inclusions seems to override the list of exclusions.



> Fix rat check (April 2021)
> --
>
> Key: CASSANDRA-16558
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16558
> Project: Cassandra
>  Issue Type: Task
>  Components: Build, Packaging
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: High
> Fix For: 2.2.20, 3.0.25, 3.11.11, 4.0, 4.0-rc1
>
> Attachments: Screenshot 2021-04-18 at 15.34.01.png
>
>
> The rat plugin in build.xml is a mess and not properly catching missing 
> license headers.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Update release process documentation wrt bintray removal and new usage of ASF JFrog Artifactory ref: https://github.com/apache/cassandra/pull/985

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new ab91b1f  Update release process documentation wrt bintray removal and 
new usage of ASF JFrog Artifactory  ref: 
https://github.com/apache/cassandra/pull/985
ab91b1f is described below

commit ab91b1fdc7ce253fa00a8b68d45e4710b8819bd8
Author: Mick Semb Wever 
AuthorDate: Mon Apr 26 13:33:34 2021 +0200

Update release process documentation wrt bintray removal and new usage of 
ASF JFrog Artifactory
 ref: https://github.com/apache/cassandra/pull/985

 patch by Mick Semb Wever; reviewed by Ekaterina Dimitrova
---
 doc/source/development/release_process.rst | 10 +++---
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/doc/source/development/release_process.rst 
b/doc/source/development/release_process.rst
index a2bcf48..e409141 100644
--- a/doc/source/development/release_process.rst
+++ b/doc/source/development/release_process.rst
@@ -56,10 +56,10 @@ Include your public key in::
 
 Publish your GPG key in a PGP key server, such as `MIT Keyserver 
`_.
 
-Bintray account with access to Apache organisation
---
+Artifactory account with access to Apache organisation
+--
 
-Publishing a successfully voted upon release requires bintray access to the 
Apache organisation. Please verify that you have a bintray account and the 
Apache organisation is listed `here 
`_.
+Publishing a successfully voted upon release requires Artifactory access using 
your Apache LDAP credentials. Please verify that you have logged into 
Artifactory `here `_.
 
 
 Create Release Artifacts
@@ -159,10 +159,6 @@ Promote Nexus Repository
 * Select the "Releases" repository and click "Promote".
 * Next click on "Repositories", select the "Releases" repository and validate 
that your artifacts exist as you expect them.
 
-Publish the Bintray Uploaded Distribution Packages

-
-Log into bintray and publish the uploaded artifacts.
 
 Update and Publish Website
 --

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16135) Separate in-JVM test into smaller packages

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16135:
-
Status: Open  (was: Patch Available)

> Separate in-JVM test into smaller packages
> --
>
> Key: CASSANDRA-16135
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16135
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: High
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Introduce a structure similar to how tags are organised in Cassandra Jira for 
> corresponding in-jvm dtests to help people find a right place for their tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16135) Separate in-JVM test into smaller packages

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16135:
-
Status: Patch Available  (was: Ready to Commit)

> Separate in-JVM test into smaller packages
> --
>
> Key: CASSANDRA-16135
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16135
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest/java
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: High
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-rc
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Introduce a structure similar to how tags are organised in Cassandra Jira for 
> corresponding in-jvm dtests to help people find a right place for their tests.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16626) Improve how-to commit docs

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332447#comment-17332447
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16626:
-

I had one small suggestion but I don't mind If you ignore it. 

+1

> Improve how-to commit docs
> --
>
> Key: CASSANDRA-16626
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16626
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
> Attachments: index.jpg
>
>
> The how to commit page looks outdated where 3.3 is being merged into trunk 
> whereas it should be 3.11 instead. Also adding a comment on folding patches 
> to the forward merge.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16632:
-
Reviewers: Brandon Williams
   Status: Review In Progress  (was: Patch Available)

> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Assignee: Matt Fleming
>Priority: Normal
> Fix For: 4.0-rc
>
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.
> Patch here: 
> https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332421#comment-17332421
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16632:
-

Thanks [~mfleming], cc [~brandon.williams]

> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Assignee: Matt Fleming
>Priority: Normal
> Fix For: 4.0-rc
>
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.
> Patch here: 
> https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16631) Update README.asc

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16631:
-
Source Control Link: 
https://github.com/apache/cassandra/commit/e9cf21b3a7c0a9badd397ed9580e8798f916ed45
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed, thanks.

> Update README.asc
> -
>
> Key: CASSANDRA-16631
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16631
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Mitesh Gupta
>Assignee: Mitesh Gupta
>Priority: Normal
>  Labels: pull-request-available
> Fix For: NA
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Link to 'Partitioning' and 'Row store' shows Page Not Found, because it is 
> moved to another place.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16631) Update README.asc

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16631:
-
Reviewers: Brandon Williams
   Status: Review In Progress  (was: Patch Available)

> Update README.asc
> -
>
> Key: CASSANDRA-16631
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16631
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Mitesh Gupta
>Assignee: Mitesh Gupta
>Priority: Normal
>  Labels: pull-request-available
> Fix For: NA
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Link to 'Partitioning' and 'Row store' shows Page Not Found, because it is 
> moved to another place.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16631) Update README.asc

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16631:
-
Status: Ready to Commit  (was: Review In Progress)

> Update README.asc
> -
>
> Key: CASSANDRA-16631
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16631
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Mitesh Gupta
>Assignee: Mitesh Gupta
>Priority: Normal
>  Labels: pull-request-available
> Fix For: NA
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Link to 'Partitioning' and 'Row store' shows Page Not Found, because it is 
> moved to another place.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16632:

Test and Documentation Plan:  
[https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4]
 Status: Patch Available  (was: In Progress)

> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Assignee: Matt Fleming
>Priority: Normal
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.
> Patch here: 
> https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16632:

Fix Version/s: 4.0-rc

> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Assignee: Matt Fleming
>Priority: Normal
> Fix For: 4.0-rc
>
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.
> Patch here: 
> https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16632:

Change Category: Quality Assurance
 Complexity: Normal
   Assignee: Matt Fleming
 Status: Open  (was: Triage Needed)

> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Assignee: Matt Fleming
>Priority: Normal
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.
> Patch here: 
> https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 71f8fa043f96caf4d15260bcdc6042934e8e9c88
Merge: bd488e4 354b87a
Author: Brandon Williams 
AuthorDate: Mon Apr 26 08:47:11 2021 -0500

Merge branch 'cassandra-3.11' into trunk

 README.asc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)


-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated (bd488e4 -> 71f8fa0)

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a change to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from bd488e4  Increment version to 4.0-rc2
 new e9cf21b  Update README links
 new 354b87a  Merge branch 'cassandra-3.0' into cassandra-3.11
 new 71f8fa0  Merge branch 'cassandra-3.11' into trunk

The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 README.asc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-3.11 updated (9e1b8c0 -> 354b87a)

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a change to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git.


from 9e1b8c0  Merge branch 'cassandra-3.0' into cassandra-3.11
 new e9cf21b  Update README links
 new 354b87a  Merge branch 'cassandra-3.0' into cassandra-3.11

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 README.asc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git

commit 354b87a0f6a9d7da02645627ae4d21feda1286e5
Merge: 9e1b8c0 e9cf21b
Author: Brandon Williams 
AuthorDate: Mon Apr 26 08:46:56 2021 -0500

Merge branch 'cassandra-3.0' into cassandra-3.11

 README.asc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --cc README.asc
index 8a6bb48,4c76580..6224b42
--- a/README.asc
+++ b/README.asc
@@@ -1,11 -1,11 +1,11 @@@
 -Executive summary
 +Apache Cassandra
  -
  
 -Cassandra is a partitioned row store.  Rows are organized into tables with a 
required primary key.
 +Apache Cassandra is a highly-scalable partitioned row store. Rows are 
organized into tables with a required primary key.
  
- http://wiki.apache.org/cassandra/Partitioners[Partitioning] means that 
Cassandra can distribute your data across multiple machines in an 
application-transparent matter. Cassandra will automatically repartition as 
machines are added and removed from the cluster.
+ 
https://cwiki.apache.org/confluence/display/CASSANDRA2/Partitioners[Partitioning]
 means that Cassandra can distribute your data across multiple machines in an 
application-transparent matter. Cassandra will automatically repartition as 
machines are added and removed from the cluster.
  
- http://wiki.apache.org/cassandra/DataModel[Row store] means that like 
relational databases, Cassandra organizes data by rows and columns. The 
Cassandra Query Language (CQL) is a close relative of SQL.
+ https://cwiki.apache.org/confluence/display/CASSANDRA2/DataModel[Row store] 
means that like relational databases, Cassandra organizes data by rows and 
columns. The Cassandra Query Language (CQL) is a close relative of SQL.
  
  For more information, see http://cassandra.apache.org/[the Apache Cassandra 
web site].
  

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch cassandra-3.0 updated: Update README links

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
 new e9cf21b  Update README links
e9cf21b is described below

commit e9cf21b3a7c0a9badd397ed9580e8798f916ed45
Author: Mitesh 
AuthorDate: Mon Apr 26 08:43:47 2021 -0500

Update README links

Patch by Mitesh Gupta, reviewed by brandonwilliams for CASSANDRA-16631
---
 README.asc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/README.asc b/README.asc
index 3a48991..4c76580 100644
--- a/README.asc
+++ b/README.asc
@@ -3,9 +3,9 @@ Executive summary
 
 Cassandra is a partitioned row store.  Rows are organized into tables with a 
required primary key.
 
-http://wiki.apache.org/cassandra/Partitioners[Partitioning] means that 
Cassandra can distribute your data across multiple machines in an 
application-transparent matter.  Cassandra will automatically repartition as 
machines are added and removed from the cluster.
+https://cwiki.apache.org/confluence/display/CASSANDRA2/Partitioners[Partitioning]
 means that Cassandra can distribute your data across multiple machines in an 
application-transparent matter. Cassandra will automatically repartition as 
machines are added and removed from the cluster.
 
-http://wiki.apache.org/cassandra/DataModel[Row store] means that like 
relational databases, Cassandra organizes data by rows and columns.  The 
Cassandra Query Language (CQL) is a close relative of SQL.
+https://cwiki.apache.org/confluence/display/CASSANDRA2/DataModel[Row store] 
means that like relational databases, Cassandra organizes data by rows and 
columns. The Cassandra Query Language (CQL) is a close relative of SQL.
 
 For more information, see http://cassandra.apache.org/[the Apache Cassandra 
web site].
 

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16605) The dependencies in the Cassandra DEB package prevent installation on Ubuntu 20.04+

[Michael Kelly (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332377#comment-17332377
 ] 

Michael Kelly commented on CASSANDRA-16605:
---

Not quite.

If python3 is not supported until Cassandra 4.0, that's fine by me.

The issue is that cassandra3 has a dependency on 'python >= 2.7'.

In Ubuntu 20.04 (and later, I presume), no package can satisfy that dependency 
(the 'python' package was removed sometime after 18.04).

The python2 package provides 'python2=2.7.17'.

I guess the optimal solution would be to  change the dependency list, something 
like:

Depends: python (>= 2.7) | python2 (>= 2.7), ...

That should cover all currently supported versions of Ubuntu.

 

> The dependencies in the Cassandra DEB package prevent installation on Ubuntu 
> 20.04+
> ---
>
> Key: CASSANDRA-16605
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16605
> Project: Cassandra
>  Issue Type: Task
>  Components: Documentation/Website
>Reporter: Michael Kelly
>Assignee: Erick Ramirez
>Priority: Normal
> Fix For: 4.0.x
>
>
> The DEB package for Cassandra has a dependency on 'python >= 2.7'.
> In Ubuntu 18.04 there is a package called 'python' which installs python2.7.
> This package has been removed in Ubuntu 20.04 so the dependency cannot be 
> satisified.
>  
> Proposed solution:
> Change the dependency 'python >= 2.7' to something like 'python2 >= 2.7 OR 
> python3 >= 3.6'.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Matt Fleming (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Fleming updated CASSANDRA-16632:
-
Authors: Matt Fleming
Description: 
While working on CASSANDRA-16588 I had some tests that were very useful for 
getting the cluster gossip state into particular configurations and they even 
caught some oversights in the original suggestion for CASSANDRA-16588's fix.

Patch here: 
https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4

  was:While working on CASSANDRA-16588 I had some tests that were very useful 
for getting the cluster gossip state into particular configurations and they 
even caught some oversights in the original suggestion for CASSANDRA-16588's 
fix.


> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Priority: Normal
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.
> Patch here: 
> https://github.com/mfleming/cassandra-dtest/commit/f3eb50f33444da3ea599f2d51129b54f2024ead4



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Matt Fleming (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt Fleming updated CASSANDRA-16632:
-
Component/s: Test/dtest/python

> Add gossip tests from CASSANDRA-16588
> -
>
> Key: CASSANDRA-16632
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Test/dtest/python
>Reporter: Matt Fleming
>Priority: Normal
>
> While working on CASSANDRA-16588 I had some tests that were very useful for 
> getting the cluster gossip state into particular configurations and they even 
> caught some oversights in the original suggestion for CASSANDRA-16588's fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16632) Add gossip tests from CASSANDRA-16588

[Matt Fleming (Jira)]

Matt Fleming created CASSANDRA-16632:


 Summary: Add gossip tests from CASSANDRA-16588
 Key: CASSANDRA-16632
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16632
 Project: Cassandra
  Issue Type: Improvement
Reporter: Matt Fleming


While working on CASSANDRA-16588 I had some tests that were very useful for 
getting the cluster gossip state into particular configurations and they even 
caught some oversights in the original suggestion for CASSANDRA-16588's fix.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16614) Flaky test_pending_range

[Jira]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andres de la Peña updated CASSANDRA-16614:
--
  Fix Version/s: (was: 4.0-rc)
 4.0-rc2
  Since Version: 2.1.13
Source Control Link: 
https://github.com/apache/cassandra-dtest/commit/2032cb8503d9a3e90822de72458a09dd07d30b7e
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

> Flaky test_pending_range
> 
>
> Key: CASSANDRA-16614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-rc2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Flaky 
> [test_pending_range|https://ci-cassandra.apache.org/job/Cassandra-trunk/445/testReport/junit/dtest-large-novnode.pending_range_test/TestPendingRangeMovements/test_pending_range/]
> {noformat}
> Error Message
> AssertionError: assert None is not None  +  where None =  0x7f29dfa83b80>('127\\.0\\.0\\.1.*?Down.*?Moving', '\nDatacenter: 
> datacenter1\n==\nAddress RackStatus State   Load  
>   Owns   ...   rack1   Up Normal  90.86 KiB   
> 40.00%  5534023222112865484 \n\n\n  ')  + 
>where  = re.search
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16614) Flaky test_pending_range

[Jira]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332047#comment-17332047
 ] 

Andres de la Peña commented on CASSANDRA-16614:
---

[~bereng] great, thanks. I'm setting you as co-author since I wrote the patch 
on top of your changes and findings. Committed as 
[2032cb8503d9a3e90822de72458a09dd07d30b7e|https://github.com/apache/cassandra-dtest/commit/2032cb8503d9a3e90822de72458a09dd07d30b7e].

> Flaky test_pending_range
> 
>
> Key: CASSANDRA-16614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16614
> Project: Cassandra
>  Issue Type: Bug
>  Components: Test/dtest/python
>Reporter: Berenguer Blasi
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-rc
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Flaky 
> [test_pending_range|https://ci-cassandra.apache.org/job/Cassandra-trunk/445/testReport/junit/dtest-large-novnode.pending_range_test/TestPendingRangeMovements/test_pending_range/]
> {noformat}
> Error Message
> AssertionError: assert None is not None  +  where None =  0x7f29dfa83b80>('127\\.0\\.0\\.1.*?Down.*?Moving', '\nDatacenter: 
> datacenter1\n==\nAddress RackStatus State   Load  
>   Owns   ...   rack1   Up Normal  90.86 KiB   
> 40.00%  5534023222112865484 \n\n\n  ')  + 
>where  = re.search
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16627) Flaky DirectoriesTest.testSecondaryIndexDirectories

[Berenguer Blasi (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17332046#comment-17332046
 ] 

Berenguer Blasi commented on CASSANDRA-16627:
-

I want to give it another go with some {{Rule}} to repeat the whole class 
programmatically. It didn't work at shell level but maybe it will doing so...

> Flaky DirectoriesTest.testSecondaryIndexDirectories
> ---
>
> Key: CASSANDRA-16627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16627
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> See failure 
> [here|https://ci-cassandra.apache.org/job/Cassandra-trunk/462/testReport/junit/org.apache.cassandra.db/DirectoriesTest/testSecondaryIndexDirectories_cdc/]
> {noformat}
> Error Message
> expected:<1619094213489> but was:<1619094213485>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<1619094213489> but 
> was:<1619094213485>
>   at 
> org.apache.cassandra.db.DirectoriesTest.testSecondaryIndexDirectories(DirectoriesTest.java:216)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-16627) Flaky DirectoriesTest.testSecondaryIndexDirectories

[Berenguer Blasi (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Berenguer Blasi reassigned CASSANDRA-16627:
---

Assignee: Berenguer Blasi

> Flaky DirectoriesTest.testSecondaryIndexDirectories
> ---
>
> Key: CASSANDRA-16627
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16627
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/unit
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-rc
>
>
> See failure 
> [here|https://ci-cassandra.apache.org/job/Cassandra-trunk/462/testReport/junit/org.apache.cassandra.db/DirectoriesTest/testSecondaryIndexDirectories_cdc/]
> {noformat}
> Error Message
> expected:<1619094213489> but was:<1619094213485>
> Stacktrace
> junit.framework.AssertionFailedError: expected:<1619094213489> but 
> was:<1619094213485>
>   at 
> org.apache.cassandra.db.DirectoriesTest.testSecondaryIndexDirectories(DirectoriesTest.java:216)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-dtest] branch trunk updated: Fix flaky test_pending_range

[adelapena]

This is an automated email from the ASF dual-hosted git repository.

adelapena pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 2032cb8  Fix flaky test_pending_range
2032cb8 is described below

commit 2032cb8503d9a3e90822de72458a09dd07d30b7e
Author: Andrés de la Peña 
AuthorDate: Wed Apr 21 16:53:39 2021 +0100

Fix flaky test_pending_range

patch by Andrés de la Peña; reviewed by Berenguer Blasi for CASSANDRA-16614

Co-authored-by: Andrés de la Peña 
Co-authored-by: Bereng 
---
 README.md | 13 +
 pending_range_test.py | 33 +++--
 requirements.txt  |  1 +
 3 files changed, 33 insertions(+), 14 deletions(-)

diff --git a/README.md b/README.md
index b7efc05..efefb21 100644
--- a/README.md
+++ b/README.md
@@ -78,6 +78,19 @@ directory is to set `cassandra_dir` in 
`~/path/to/cassandra-dtest/pytest.ini`:
 The tests will use this directory by default, avoiding the need for any
 environment variable (that still will have precedence if given though).
 
+To run a specific test file, class or individual test, you only have to 
+pass its path as an argument:
+
+pytest --cassandra-dir=~/path/to/cassandra pending_range_test.py
+pytest --cassandra-dir=~/path/to/cassandra 
pending_range_test.py::TestPendingRangeMovements
+pytest --cassandra-dir=~/path/to/cassandra 
pending_range_test.py::TestPendingRangeMovements::test_pending_range
+
+When adding a new test or modifying an existing one, it's always a good idea to
+run it several times to make sure it is stable. This can be easily done with 
+the ``--count`` option. For example, to run a test class 10 times:
+
+pytest --count=10 --cassandra-dir=~/path/to/cassandra pending_range_test.py
+
 Existing tests are probably the best place to start to look at how to write
 tests.
 
diff --git a/pending_range_test.py b/pending_range_test.py
index 6371312..e643ad7 100644
--- a/pending_range_test.py
+++ b/pending_range_test.py
@@ -1,6 +1,7 @@
 import logging
 import pytest
 import re
+import threading
 
 from cassandra.query import SimpleStatement
 
@@ -23,7 +24,8 @@ class TestPendingRangeMovements(Tester):
 cluster.set_log_level('DEBUG')
 
 # Create 5 node cluster
-cluster.populate(5).start()
+ring_delay_ms = 3_600_000  # 1 hour
+
cluster.populate(5).start(jvm_args=['-Dcassandra.ring_delay_ms={}'.format(ring_delay_ms)])
 node1, node2 = cluster.nodelist()[0:2]
 
 # Set up RF=3 keyspace
@@ -46,27 +48,30 @@ class TestPendingRangeMovements(Tester):
 
 mark = node1.mark_log()
 
-# Move a node
-node1.nodetool('move {}'.format(token))
+# Move a node without waiting for the response of nodetool, so we 
don't have to wait for ring_delay
+threading.Thread(target=(lambda: node1.nodetool('move 
{}'.format(token.start()
 
 # Watch the log so we know when the node is moving
 node1.watch_log_for('Moving .* to {}'.format(token), timeout=10, 
from_mark=mark)
-node1.watch_log_for('Sleeping 3 ms before start streaming/fetching 
ranges', timeout=10, from_mark=mark)
-
-if cluster.version() >= '2.2':
-if cluster.version() >= '4.0':
-node2.watch_log_for('127.0.0.1:7000 state MOVING', timeout=10, 
filename='debug.log')
+node1.watch_log_for('Sleeping {} ms before start streaming/fetching 
ranges'.format(ring_delay_ms),
+timeout=10, from_mark=mark)
+
+# Watch the logs so we know when all the nodes see the status update 
to MOVING
+for node in cluster.nodelist():
+if cluster.version() >= '2.2':
+if cluster.version() >= '4.0':
+node.watch_log_for('127.0.0.1:7000 state MOVING', 
timeout=10, filename='debug.log')
+else:
+node.watch_log_for('127.0.0.1 state moving', timeout=10, 
filename='debug.log')
 else:
-node2.watch_log_for('127.0.0.1 state moving', timeout=10, 
filename='debug.log')
-else:
-# 2.1 doesn't have debug.log, so we are logging at trace, and look
-# in the system.log file
-node2.watch_log_for('127.0.0.1 state moving', timeout=10, 
filename='system.log')
+# 2.1 doesn't have debug.log, so we are logging at trace, and 
look
+# in the system.log file
+node.watch_log_for('127.0.0.1 state moving', timeout=10, 
filename='system.log')
 
 # Once the node is MOVING, kill it immediately, let the other nodes 
notice
 node1.stop(gently=False, wait_other_notice=True)
 
-# Verify other nodes believe this is Down/Moving
+# Verify other nodes believe that the killed node is Down/Moving
 out, _, _ = node2.nodetool('ring')

[jira] [Updated] (CASSANDRA-16360) CRC32 is inefficient on x86

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-16360:

Impacts: Clients  (was: None)

> CRC32 is inefficient on x86
> ---
>
> Key: CASSANDRA-16360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16360
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Avi Kivity
>Assignee: Alexey Zotov
>Priority: Normal
>  Labels: protocolv6
> Fix For: 4.0.x
>
>
> The client/server protocol specifies CRC24 and CRC32 as the checksum 
> algorithm (cql_protocol_V5_framing.asc). Those however are expensive to 
> compute; this affects both the client and the server.
>  
> A better checksum algorithm is CRC32C, which has hardware support on x86 (as 
> well as other modern architectures).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16360) CRC32 is inefficient on x86

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-16360:

Labels: protocolv6  (was: protocolv5)

> CRC32 is inefficient on x86
> ---
>
> Key: CASSANDRA-16360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16360
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Messaging/Client
>Reporter: Avi Kivity
>Assignee: Alexey Zotov
>Priority: Normal
>  Labels: protocolv6
> Fix For: 4.0.x
>
>
> The client/server protocol specifies CRC24 and CRC32 as the checksum 
> algorithm (cql_protocol_V5_framing.asc). Those however are expensive to 
> compute; this affects both the client and the server.
>  
> A better checksum algorithm is CRC32C, which has hardware support on x86 (as 
> well as other modern architectures).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-website] branch asf-staging updated (dd40a0e -> 92198fd)

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a change to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git.


 discard dd40a0e  hack in plausible tracking with
 new bb53788  hack new design website with latest 4.0-rc1 release
 new 92198fd  hack in plausible tracking with

This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version.  This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:

 * -- * -- B -- O -- O -- O   (dd40a0e)
\
 N -- N -- N   refs/heads/asf-staging (92198fd)

You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.

Any revisions marked "omit" are not gone; other references still
refer to them.  Any revisions marked "discard" are gone forever.

The 2 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 content/download/index.html | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org