date:20200615



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136209#comment-17136209
 ] 

Caleb Rackliffe commented on CASSANDRA-15872:
-

[~jasonstack] It looks like the fix should address the race between 
{{SyncTask}} completion and the assertion, but I think we can do this without 
modifying {{RepairJob}}. One alternative is that we could take 
{{MeasureableRepairSession}}, provide it a callback (perhaps allow it to be set 
int the test body), override {{syncComplete()}}, and hit the callback from that 
before we call {{super.syncComplete()}}. That would allow us to await on a 
latch in there or even directly make the assertion. (My assumption is that 
there will be at least one remote job.)

Making the test a bit more complex while simplifying the production code seems 
reasonable here. Thoughts?

> Flaky test: testNoTreesRetainedAfterDifference - 
> org.apache.cassandra.repair.RepairJobTest
> --
>
> Key: CASSANDRA-15872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15872
> Project: Cassandra
>  Issue Type: Task
>  Components: Consistency/Repair
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-rc
>
>
> link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15877) Followup on CASSANDRA-15600

2020-06-15 Thread Kornel Pal (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136198#comment-17136198
 ] 

Kornel Pal commented on CASSANDRA-15877:


Thank you [~e.dimitrova], for further improving on the previous change and even 
fixing a bug that previously resulted in a flaky unit test.

I've noticed that you added a new NoReplicationTokenAllocatorTest.failed field 
with assertions, but it does not seem to be set to true anywhere. Could you 
please check whether it is needed.

After the change from random tokens to splits, 
TokenAllocatorDiagnostics.randomTokensGenerated does not seem to be used 
anymore. Could you please consider removing it, if not needed.

> Followup on CASSANDRA-15600
> ---
>
> Key: CASSANDRA-15877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> As part of CASSANDRA-15600  generateSplits method replaced the 
> generateRandomTokens for NoReplicationAwareTokenAllocator.  generateSplits 
> should be used also in ReplicationAwareTokenAllocator.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15600) Algorithmic token allocation does not handle the racks = RF case well



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136177#comment-17136177
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15600:
-

CASSANDRA-15877

> Algorithmic token allocation does not handle the racks = RF case well
> -
>
> Key: CASSANDRA-15600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15600
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Branimir Lambov
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha4
>
> Attachments: CASSANDRA-15600 unit.txt, CASSANDRA-15600-jvm.txt.zip, 
> Screen Shot 2020-03-02 at 6.55.41 PM.png
>
>
> When the number of racks in a data centre is equal to the replication factor, 
> the node allocation algorithm may calculate the target ownership incorrectly 
> and allocate unsuitable tokens. This causes some inefficiency when the number 
> of nodes in the DC is small, and significant problems if the number of nodes 
> in each rack is different.
> In this case (racks count equal to replication factor) the load within each 
> rack is effectively distributed independently. The allocation algorithm 
> should reflect that, restricting the allocation space to the rack (instead of 
> the DC) and using the RF=1 solution.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15877) Followup on CASSANDRA-15600



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136175#comment-17136175
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15877 at 6/15/20, 10:14 PM:


[ [branch 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15600-followup]
 ] [ [Java8 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/216/workflows/2e1ae6fe-6b04-4099-a550-8e069cd030cb]
 ] [ [Java11 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/216/workflows/f98e8ecf-8852-43a6-90aa-c3078418cac1]
 ] [ [pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/44] ]

*Comments on the CI run:*
 - Unit tests Java8
testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest 
- failure already presented on trunk, CASSANDRA-15872
 - Unit tests Java11
testTimeout - org.apache.cassandra.net.ConnectionTest - looks unrelated
 - CQLSH tests Java8
test_compression_cql_options - compression_test.TestCompression - looks 
unrelated
 - DTests Java8
test_restart_node_localhost - pushed_notifications_test.TestPushedNotifications,
test_move_single_node_localhost - 
pushed_notifications_test.TestPushedNotifications - already on trunk, missed 
after a patch committed last week, contacted the author
test_disk_balance_after_joining_ring_lcs - disk_balance_test.TestDiskBalance - 
presented on trunk already
 - DTests Java11
test_speculative_data_request - read_repair_test.TestSpeculativeReadRepair - 
CASSANDRA-15792
test_restart_node_localhost - 
pushed_notifications_test.TestPushedNotifications, 
test_move_single_node_localhost - 
pushed_notifications_test.TestPushedNotifications - already on trunk, missed 
after a patch committed last week, contacted the author

*The first commit is the actual patch. The second commit applies our custom 
CircleCI config and shouldn't be committed! Kept for test purposes.*

 


was (Author: e.dimitrova):
[[branch 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15600-followup]]
 [[Java8 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/216/workflows/2e1ae6fe-6b04-4099-a550-8e069cd030cb]]
 [[Java11 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/216/workflows/f98e8ecf-8852-43a6-90aa-c3078418cac1]]
 [[pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/44]]

*Comments on the CI run:*
 - Unit tests Java8
testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest 
- failure already presented on trunk, CASSANDRA-15872
 - Unit tests Java11
testTimeout - org.apache.cassandra.net.ConnectionTest - looks unrelated
 - CQLSH tests Java8
test_compression_cql_options - compression_test.TestCompression - looks 
unrelated
 - DTests Java8
test_restart_node_localhost - pushed_notifications_test.TestPushedNotifications,
test_move_single_node_localhost - 
pushed_notifications_test.TestPushedNotifications - already on trunk, missed 
after a patch committed last week, contacted the author
test_disk_balance_after_joining_ring_lcs - disk_balance_test.TestDiskBalance - 
presented on trunk already
 - DTests Java11
test_speculative_data_request - read_repair_test.TestSpeculativeReadRepair - 
CASSANDRA-15792
test_restart_node_localhost - 
pushed_notifications_test.TestPushedNotifications, 
test_move_single_node_localhost - 
pushed_notifications_test.TestPushedNotifications - already on trunk, missed 
after a patch committed last week, contacted the author

*The first commit is the actual patch. The second commit applies our custom 
CircleCI config and shouldn't be committed! Kept for test purposes.*

 

> Followup on CASSANDRA-15600
> ---
>
> Key: CASSANDRA-15877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> As part of CASSANDRA-15600  generateSplits method replaced the 
> generateRandomTokens for NoReplicationAwareTokenAllocator.  generateSplits 
> should be used also in ReplicationAwareTokenAllocator.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15877) Followup on CASSANDRA-15600



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136175#comment-17136175
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15877:
-

[[branch 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15600-followup]]
 [[Java8 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/216/workflows/2e1ae6fe-6b04-4099-a550-8e069cd030cb]]
 [[Java11 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/216/workflows/f98e8ecf-8852-43a6-90aa-c3078418cac1]]
 [[pull request |https://github.com/ekaterinadimitrova2/cassandra/pull/44]]

*Comments on the CI run:*
 - Unit tests Java8
testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest 
- failure already presented on trunk, CASSANDRA-15872
 - Unit tests Java11
testTimeout - org.apache.cassandra.net.ConnectionTest - looks unrelated
 - CQLSH tests Java8
test_compression_cql_options - compression_test.TestCompression - looks 
unrelated
 - DTests Java8
test_restart_node_localhost - pushed_notifications_test.TestPushedNotifications,
test_move_single_node_localhost - 
pushed_notifications_test.TestPushedNotifications - already on trunk, missed 
after a patch committed last week, contacted the author
test_disk_balance_after_joining_ring_lcs - disk_balance_test.TestDiskBalance - 
presented on trunk already
 - DTests Java11
test_speculative_data_request - read_repair_test.TestSpeculativeReadRepair - 
CASSANDRA-15792
test_restart_node_localhost - 
pushed_notifications_test.TestPushedNotifications, 
test_move_single_node_localhost - 
pushed_notifications_test.TestPushedNotifications - already on trunk, missed 
after a patch committed last week, contacted the author

*The first commit is the actual patch. The second commit applies our custom 
CircleCI config and shouldn't be committed! Kept for test purposes.*

 

> Followup on CASSANDRA-15600
> ---
>
> Key: CASSANDRA-15877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> As part of CASSANDRA-15600  generateSplits method replaced the 
> generateRandomTokens for NoReplicationAwareTokenAllocator.  generateSplits 
> should be used also in ReplicationAwareTokenAllocator.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15877) Followup on CASSANDRA-15600



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15877:

Test and Documentation Plan: 
https://jira.apache.org/jira/browse/CASSANDRA-15877?focusedCommentId=17136175=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17136175
 Status: Patch Available  (was: In Progress)

> Followup on CASSANDRA-15600
> ---
>
> Key: CASSANDRA-15877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> As part of CASSANDRA-15600  generateSplits method replaced the 
> generateRandomTokens for NoReplicationAwareTokenAllocator.  generateSplits 
> should be used also in ReplicationAwareTokenAllocator.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15877) Followup on CASSANDRA-15600



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15877:

Description: As part of CASSANDRA-15600  generateSplits method replaced the 
generateRandomTokens for NoReplicationAwareTokenAllocator.  generateSplits 
should be used also in ReplicationAwareTokenAllocator.

> Followup on CASSANDRA-15600
> ---
>
> Key: CASSANDRA-15877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>
> As part of CASSANDRA-15600  generateSplits method replaced the 
> generateRandomTokens for NoReplicationAwareTokenAllocator.  generateSplits 
> should be used also in ReplicationAwareTokenAllocator.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15877) Followup on CASSANDRA-15600



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15877:

 Bug Category: Parent values: Correctness(12982)
   Complexity: Normal
  Component/s: Feature/Virtual Nodes
Discovered By: User Report
Fix Version/s: 4.0-alpha
   4.0
 Severity: Normal
 Assignee: Ekaterina Dimitrova
   Status: Open  (was: Triage Needed)

> Followup on CASSANDRA-15600
> ---
>
> Key: CASSANDRA-15877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15877
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-15877) Followup on CASSANDRA-15600

Ekaterina Dimitrova created CASSANDRA-15877:
---

 Summary: Followup on CASSANDRA-15600
 Key: CASSANDRA-15877
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15877
 Project: Cassandra
  Issue Type: Bug
Reporter: Ekaterina Dimitrova






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-15876) Accessors for SystemProperties



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova reassigned CASSANDRA-15876:
---

Assignee: Ekaterina Dimitrova

> Accessors for SystemProperties
> --
>
> Key: CASSANDRA-15876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15876
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-beta
>
>
> As part of CASSANDRA-15234, it was suggested a class of accessors for System 
> properties to be created for better clarity and maintainability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126344#comment-17126344
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/15/20, 10:03 PM:


[ [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase]
 ] [ [DTests 
|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] 
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] 
[ [JAVA8 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e]
 ] [ [JAVA11 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757]
 ]

Attached to the ticket is the log of the successful runs of the few failing 
JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as 
it didn't make sense to me to rerun the whole suite of tests.

*Last update:* after rebase and a few nits on my end, we have only 1 DTest and 
1 Unit test failing in JAVA11. Unrelated known failures.
 - added additional note on the possible units to be used as suffixes in both 
NEWS.txt and cassandra.yaml

CQLSH JAVA11 tests are not currently available in CircleCI.

*EDIT:* This work has been moved to a separate ticket - CASSANDRA-15876.

-The last part is all green after rebase and CI run.-
 - -"line.separator" getter added-
 -[ [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ 
[JAVA8 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6]
 ] [ [JAVA11 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7]
 ]-

-There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check 
for tickets logged.-

*NOTE:* Before commit return to the default config.yml and requirements.txt 
files


was (Author: e.dimitrova):
[ [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase 
] ] [ [DTests | 
https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] 
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] 
[ [JAVA8 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e]
 ] [ [JAVA11 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757
 ] ] 

Attached to the ticket is the log of the successful runs of the few failing 
JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as 
it didn't make sense to me to rerun the whole suite of tests. 

*Last update:* after rebase and a few nits on my end, we have only 1 DTest and 
1 Unit test failing in JAVA11. Unrelated known failures.

- added additional note on the possible units to be used as suffixes in both 
NEWS.txt and cassandra.yaml

CQLSH JAVA11 tests are not currently available in CircleCI.


The last part is all green after rebase and CI run. 
- "line.separator" getter added
[ [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ 
[JAVA8 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6]
 ] [ [JAVA11 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7]
 ] 

There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check 
for tickets logged. 

*NOTE:* Before commit return to the default config.yml and requirements.txt 
files

> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase, 
> both from the yams and JVM properties.  It would be nice to standardise the 
> naming (such as otc_ vs internode_) as well as the provision of values with 
> units - while maintaining perpetual backwards compatibility with the old 
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s, 
>

[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17125553#comment-17125553
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/15/20, 10:02 PM:


[ [Part 2 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3] ] [ 
[DTests 
|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] 
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] 
[ [CI run 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/187/workflows/615cdebf-1d49-47ab-93a4-efad25a247b1]
 ]

 - Unit tests updated

- Getters and Setters names in the DatabaseDescriptor were updated to follow 
the new names of the parameters updated
 - The conversion of some of the parameters values which is happening in their 
getters is kept as it is needed in runtime. Some parameters are used with 
different units in the different areas of the codebase. So I left this part 
untouched.

 - New class created, ParseAndConvertUnits - it pulls all the functionality 
needed to cover the addition of units suffixes to the Memory, Duration, and 
Rate parameters in the cassandra.yaml

- the cassandra.yaml file in this branch contains currently the new parameters 
with new names and unit suffixes but the arrangement of the sections is the old 
one as this was causing me issues every time I rebase/commit. If we confirm the 
format, this will be the last step to switch to the new format

- Added some additional parsing and supporting conversion functions for the 
double duration parameters.

- The 2 failing cqlsh tests were fixed locally and tested. It was formatting 
issue. 

-1 unit test failed - it doesn't look a related issue but will check the config 
loaded to ensure that tomorrow

- 8 failing dtest, it doesn't look related but I am gonna compare tomorrow 
morning

 - Current version of the code needs a final rebase/merge of the last 6 commits 
on trunk.  Will do tomorrow after I recheck the CI and current version

- Have to return the default requirements.txt and config.yml files. The current 
versions were required only for testing purposes with CircleCI

*WARNING:* we should be careful with the dtests and ccm. One missed check for 
old/new parameter and default value of the respective parameter from Config 
will be loaded. This might not lead to failing test and thus silently cover the 
miss.

*NOTE:* Custom Types, as they were suggested, don't really work with SnakeYAML, 
unfortunately.  SnakeYAML supports nothing more than mapping between the yaml 
and the Config class.
 * _So something like that: Memory max_value = new Memory("5ms") wouldn't work. 
If there is no value in the yaml, the config default  max_value will be parsed 
properly but if there is value assigned to max_value at the yaml, it won't be 
assigned properly in Config. Classes are used for nested parameters. Also, 
currently there won't be a big benefit of the custom types as the different 
parameters validations in the DatabaseDescriptor are taking different 
approaches, there is no general framework/approach per group of parameters 
(Memory, Duration, Rate)._

_-_

_*EDIT:* Part3 has been moved to a separate ticket - CASSANDRA-15876._

-The last part was updated as per the recommendations. CI ran successfully. 
Need to rebase and recheck nothing that might need the attention of this patch 
has changed in the meanwhile. Three weeks passed since I took care of this 
one.- 
 -[ [Part3 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [  
[CI JAVA8 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/146/workflows/fae5da98-acf5-4824-bf61-3ffab3d86cd6]
 ] [ [CI Java11 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/146/workflows/21b088c4-8984-4b20-8302-2ff8adaf55f7]
 ]-


was (Author: e.dimitrova):
[ [Part 2 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3] ] [ 
[DTests 
|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] 
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] 
[ [CI run 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/187/workflows/615cdebf-1d49-47ab-93a4-efad25a247b1]
 ]

 - Unit tests updated

- Getters and Setters names in the DatabaseDescriptor were updated to follow 
the new names of the parameters updated
 - The conversion of some of the parameters values which is happening in their 
getters is kept as it is needed in runtime. Some parameters are used with 
different units in the different areas of the codebase. So I left this part 
untouched.

 - New class created, ParseAndConvertUnits - it pulls all the functionality 
needed to cover the addition of units suffixes to the Memory, Duration, and 
Rate parameters in the cassandra.yaml

- the

[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17091157#comment-17091157
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/15/20, 10:01 PM:


Latest update:
 Part1
 - format issues fixed
 - The isOldYAML logger info changed to warn, and it lists also the old 
properties used
 - The format of yaml props is noun_verb. native_transport props adjusted to 
follow this. Max sent to the back as in native_transport_threads_max.
 - MAP_OLD_TO_NEW_PARAMETERS is not static anymore(so it is freed memory after 
startup)

Part2 - will be completed in the suggested way probably tomorrow (working 
asynchronously)

*EDIT:* Part3 has been moved to a separate ticket - CASSANDRA-15876.

-Part3 - anyone any thoughts whether it should be 1 class or maybe divided into 
3 classes, as suggested by- [~dcapwell]-?-


was (Author: e.dimitrova):
Latest update:
Part1
   - format issues fixed
   - The isOldYAML logger info changed to warn, and it lists also the old 
properties used
   - The format of yaml props is noun_verb. native_transport props adjusted to 
follow this. Max sent to the back as in native_transport_threads_max.
   - MAP_OLD_TO_NEW_PARAMETERS is not static anymore(so it is freed memory 
after startup) 

Part2 - will be completed in the suggested way probably tomorrow (working 
asynchronously)

Part3 - anyone any thoughts whether it should be 1 class or maybe divided into 
3 classes, as suggested by [~dcapwell]?

 

> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase, 
> both from the yams and JVM properties.  It would be nice to standardise the 
> naming (such as otc_ vs internode_) as well as the provision of values with 
> units - while maintaining perpetual backwards compatibility with the old 
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s, 
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or 
> powers of 1000 such as {{KB/s}}, given these are regularly used for either 
> their old or new definition e.g. {{KiB/s}}, or we could support them and 
> simply log the value in bytes/s.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17089156#comment-17089156
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/15/20, 10:01 PM:


@David, thanks for the review and the clarifications and Slack constructive 
discussion!

So looks like we have the same vision and I missed one thing. Let me wrap up 
here for the record and [~mck] who is also looking into it.
 Part2 - null is a legit issue and no one of us likes the idea of adding "" in 
the yaml file. Still have to spend time to think about a workaround and that we 
already have it there.

The Memory class is kind of what I meant when I said I have to pull the current 
implementation separately.
 So the suggestion is to pull into three additional classes - Memory, 
Throughput and TimeDuration (or something similar)

The confusion was about a thing I saw and forgot to take care. :\
 There are 11 getters in the DatabaseDescriptor. 
 They need to be changed to only getters without doing one more layer of 
conversion to bytes. Example:
 getMaxValueSizeInBytes()

{ return maxValue.toBytes(); }

-> Memory getMaxValueSize()

{ return maxValue}

This will clean the code and make it easy for maintenance.

*EDIT:* Part3 has been moved to a separate ticket - CASSANDRA-15876.

-Part 3 - code-wise not related to Part1 and 2, just additional small task 
which was requested in the previous comments, considering we are trying to do 
standardization as part of this ticket.- 
 -Whether we keep all of them in one class or we split is a matter of agreement 
I think. As soon as we get it, I will correct it. This is real quick.-


was (Author: e.dimitrova):
@David, thanks for the review and the clarifications and Slack constructive 
discussion!

So looks like we have the same vision and I missed one thing. Let me wrap up 
here for the record and [~mck] who is also looking into it.
Part2 - null is a legit issue and no one of us likes the idea of adding "" in 
the yaml file.  Still have to spend time to think about a workaround and that 
we already have it there. 

The Memory class is kind of what I meant when I said I have to pull the current 
implementation separately.
So the suggestion is to pull into three additional classes - Memory, Throughput 
and TimeDuration (or something similar)

The confusion was about a thing I saw and forgot to take care. :\
There are 11 getters in the DatabaseDescriptor. 
They need to be changed to only getters without doing one more layer of 
conversion to bytes. Example:
getMaxValueSizeInBytes() { return maxValue.toBytes(); }  -> Memory 
getMaxValueSize() { return maxValue}

This will clean the code and make it easy for maintenance.

Part 3 - code-wise not related to Part1 and 2, just additional small task which 
was requested in the previous comments, considering we are trying to do 
standardization as part of this ticket.  
Whether we keep all of them in one class or we split is a matter of agreement I 
think. As soon as we get it, I will correct it. This is real quick.

> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase, 
> both from the yams and JVM properties.  It would be nice to standardise the 
> naming (such as otc_ vs internode_) as well as the provision of values with 
> units - while maintaining perpetual backwards compatibility with the old 
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s, 
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or 
> powers of 1000 such as {{KB/s}}, given these are regularly used for either 
> their old or new definition e.g. {{KiB/s}}, or we could support them and 
> simply log the value in bytes/s.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters

[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17086118#comment-17086118
]

Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/15/20, 10:00 PM:

Done but with one caveat which was caught thankfully with the DTests. Unit
tests were even on 100% green at some runs. (And my unit test was not covering
the problem conversion explained below, I should add it!)
I am new to snakeyaml so I might be wrong, I would appreciate if someone who
has the experience proves me wrong.
From the DTest failures I realized that turning into Strings the values from
the YAML we hit one issue. To me it looks like we already have it actually with
Integer and String parameters, just they are not many so it didn't bite us up
to now. Not sure whether this was considered when the Config class was built or
not.

So the issue with int parameters in Config in my solution is:
Currently in trunk, when we parse to int variable, If a parameter is commented
in the cassandra.yaml, this is observed as null and it will raise an exception
in some cases. (i.e. commitlog_sync_period_in_ms) But if it is not commented,
just not assigned a value, then this is considered as 0 (as we parse to int).
The situation is different with String, Integer. They will be in both cases
null. No differentiator. But currently we don't have many parameters being
Integer and String and this wasn't a problem.
Now with the units attached to the parameters' values in the cassandra.yaml
file, we need to parse strings with snakeyaml and then convert to the proper
int values in Config.

A way to cope with this issue is to make the users use "". In this way, if we
have a parameter not commented, but not also having an assigned value in
cassandra.yaml, we can identify it by checking a string for being empty. If the
parameter is commented or not presented, it will be null. But this would be one
more new thing for the users to consider. That's why I came back to snakeyaml
again and tried to think of a different way utilizing it. Unfortunately, up to
now I didn't find any different solution. Also, the links to the examples in
the official snakeyaml documentation are broken. Not sure whether there they
had some interesting stuff. Maybe something with customized constructor but
also, I try to make as less as possible disruptive changes in the codebase in
order not to mess up something with the Config which might be extremely
unpleasant.

I will be happy to hear an experienced opinion here. Does anyone know a better
solution? Or are we ok to introduce "" in the yaml for empty parameters/Strings
recognition?

[Part1|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-2]
(This part was already revised by [~dcapwell] but I paste the explanation again
here for completeness).
- New cassandra.yaml ---> currently cassandra-new.yaml in the branch as I
wanted to test the backward compatibility on the old version (will have to
rename properly also these yaml files with version probably)
reorganization of the file renames to make the names more consistent
- Backward compatibility with the old names - my approach is:
Cassandra comes with the new version of the yaml to ensure any new builds will
be using it
During load of the yaml file I check whether old names are identified(means
someone upgraded to 4 and wants to still use the old yaml). If this is the
case, we load the values from the old yaml to the new variables (the already
renamed parameters). Also, there is a warning to the user that there is new
cassandra.yaml version and he/she can consider an update.

[Part2|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3]
- units added to the values in the cassandra.yaml file. Parsing and unit
conversion in place. One unit test added, should be further developed.

[DTests|https://github.com/ekaterinadimitrova2/cassandra-dtest] updated
accordingly for v.4

*EDIT:* Part3 has been moved to a separate ticket - CASSANDRA-15876.

-[Part3|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5]
- SysProperties class, accessors, hope I didn't miss anything... most of them
are actually just using System.getProperty method.-
TO DO: Extract the parser/converter in a separate class from Config

[~mck]? [~dcapwell]? [~benedict]? Anyone else any thoughts here?

NOTE: The branches are not fully cleaned, this is still work in progress but we
need to take a decision on the String issue first, I think.

was (Author: e.dimitrova):
Done but with one caveat which was caught thankfully with the DTests. Unit
tests were even on 100% green at some runs. (And my unit test was not covering
the problem conversion explained below, I should add it!)
I am new to snakeyaml so I might be wrong, I would appreciate if someone who
has the experience

[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17126974#comment-17126974
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 6/15/20, 9:59 PM:
---

*Summary:*
 This branch contains:
 - [first commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99]
 which contains parameters name changes as requested; backward compatibility 
with the old names; Proposed new structure into sections for cassandra.yaml.
 - the [second commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718]
 implements the functionality which gives the end users the opportunity to 
attach units suffixes to Memory, Duration, Rates parameters.
 - [third commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e]
 is a ninja fix of one line which was changed in a wrong way during the latest 
rebase.
 - [fourth commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044]
 is to return the default config.yml

This patch requires custom [ [DTests 
|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] 
] which run successfully with the following patch of [CCM 
|https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3]
 DTests - requirements.txt should be updated back to the original one before 
commit!

The DTests are not backward compatible with the yaml file without units 
suffixes.
 This part is covered by a unit test which tests the successful load of the 
values of the parameters without units as per the old format.

[ [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase]
 ] [ [DTests 
|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] 
] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] 
[ [JAVA8 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/718c8a04-385b-4ddc-ac2e-9291c37b1f1e]
 ] [ [JAVA11 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/193/workflows/6cd6048d-5daf-4490-a5d3-b5c3178a0757]
 ]

Attached to the ticket is the log of the successful runs of the few failing 
JAVA8 DTests in CircleCI. I ran them locally after fixing lost line of code as 
it didn't make sense to me to rerun the whole suite of tests.

*Last update:* after rebase and a few nits on my end, we have only 1 DTest and 
1 Unit test failing in JAVA11. Unrelated known failures.
 CQLSH JAVA11 tests are not currently available in CircleCI.

*WARNING:* Before commit return to the default requirements.txt file

*Order of commits:* 1) [ [CASSANDRA branch 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3-rebase]
 ] 2) [ [CCM branch 
|https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] 3) [ 
[DTests branch 
|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] 
]

*EDIT:* This work has been moved to a separate ticket - CASSANDRA-15876.

-This [branch 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] is a 
class of Accessors for System properties. *This patch could be considered 
separate of the previous one, it doesn't matter* *if it is committed* *first or 
second.* It also doesn't require custom DTests or custom CCM.-
 -[ [trunk 
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ 
[JAVA8 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/ebd771ac-d71c-49ea-872b-00b534d9d5c6]
 ] [ [JAVA11 CI 
|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/195/workflows/19a3254b-cd7a-4e4f-856f-f314546ff9d7]
 ]-

-There are 1 DTest and 1 Unit unrelated failing tests in JAVA11. Should check 
for tickets logged.-


was (Author: e.dimitrova):
*Summary:*
 This branch contains:
 - [first commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/a8a9145a032a9f2d9b74369f2e4474a210aa5b99]
 which contains parameters name changes as requested; backward compatibility 
with the old names; Proposed new structure into sections for cassandra.yaml.
 - the [second commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/032fbd8cca1d7b5b96738bd96d9672864426d718]
 implements the functionality which gives the end users the opportunity to 
attach units suffixes to Memory, Duration, Rates parameters.
 - [third commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/68f89503e6e6f30e8f6df1374cb553f243f0068e]
 is a ninja fix of one line which was changed in a wrong way during the latest 
rebase.
 - [fourth commit 
|https://github.com/ekaterinadimitrova2/cassandra/commit/8389010d704a9524a1c36d96f9e4d8179a8e2044]
 is to return the default config.yml

This patch requires

[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136171#comment-17136171
 ] 

Ekaterina Dimitrova commented on CASSANDRA-15234:
-

Thank you [~mck]

I agree with you, CASSANDRA-15876 was created to facilitate this work. It is 
also not a  4.0 Beta blocker(I set the fix version to beta there) and I think 
we might be even able to further develop it. Let's continue the discussion 
there. 

> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase, 
> both from the yams and JVM properties.  It would be nice to standardise the 
> naming (such as otc_ vs internode_) as well as the provision of values with 
> units - while maintaining perpetual backwards compatibility with the old 
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s, 
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or 
> powers of 1000 such as {{KB/s}}, given these are regularly used for either 
> their old or new definition e.g. {{KiB/s}}, or we could support them and 
> simply log the value in bytes/s.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15876) Accessors for SystemProperties



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15876:

Description: As part of CASSANDRA-15234, it was suggested a class of 
accessors for System properties to be created for better clarity and 
maintainability.

> Accessors for SystemProperties
> --
>
> Key: CASSANDRA-15876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15876
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-beta
>
>
> As part of CASSANDRA-15234, it was suggested a class of accessors for System 
> properties to be created for better clarity and maintainability.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15876) Accessors for SystemProperties



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15876:

Change Category: Code Clarity
 Complexity: Low Hanging Fruit
Component/s: Local/Config
   Priority: Low  (was: Normal)
 Status: Open  (was: Triage Needed)

> Accessors for SystemProperties
> --
>
> Key: CASSANDRA-15876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15876
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Ekaterina Dimitrova
>Priority: Low
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15876) Accessors for SystemProperties



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15876:

Fix Version/s: 4.0-beta
   4.0

> Accessors for SystemProperties
> --
>
> Key: CASSANDRA-15876
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15876
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Local/Config
>Reporter: Ekaterina Dimitrova
>Priority: Low
> Fix For: 4.0, 4.0-beta
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-15876) Accessors for SystemProperties

Ekaterina Dimitrova created CASSANDRA-15876:
---

 Summary: Accessors for SystemProperties
 Key: CASSANDRA-15876
 URL: https://issues.apache.org/jira/browse/CASSANDRA-15876
 Project: Cassandra
  Issue Type: Improvement
Reporter: Ekaterina Dimitrova






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRASC-27) CDC reader in Apache Cassandra Sidecar

2020-06-15 Thread Tharanga Sampath Gamaethige (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRASC-27?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136135#comment-17136135
 ] 

Tharanga Sampath Gamaethige commented on CASSANDRASC-27:


I added a proposal to the description. Feedback, comments, and thoughts are 
welcome.

> CDC reader in Apache Cassandra Sidecar
> --
>
> Key: CASSANDRASC-27
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-27
> Project: Sidecar for Apache Cassandra
>  Issue Type: New Feature
>Reporter: Vinay Chella
>Assignee: Tharanga Sampath Gamaethige
>Priority: Normal
>
> Apache Cassandra has the CDC (Change Data Capture) features since 3.8. This 
> is further enhanced with (CASS-12148) in Cassandra 4.0.
> However, there’s no generally available mechanism to stream changes out of a 
> Cassandra database; hence the utility of this feature is limited if not 
> absent.
> Many applications use Cassandra as their primary data store. For various 
> reasons(Caching, analyzing, indexing, etc), this data needs to be 
> synchronized with derived/secondary data stores.  We would like to emit 
> change streams in real-time to consumers so that changes to Cassandra can be 
> used for various purposes.
> *Goals*
>  * Enhance Apache Cassandra sidecar with a CDC reader that can read and emit 
> changes in real-time. Priority for the initial implementation is safety and 
> correctness, performance enhancements will follow in subsequent iterations
> *Nongoals*
>  * Modify Cassandra storage engine to emit changes
>  
> *Proposal*
> https://docs.google.com/document/d/11YywfJTm29szZOVOSRbtfvClbmMQtJ8WyCB7_CUgo-U/edit?usp=sharing
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRASC-27) CDC reader in Apache Cassandra Sidecar

2020-06-15 Thread Tharanga Sampath Gamaethige (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRASC-27?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tharanga Sampath Gamaethige updated CASSANDRASC-27:
---
Description: 
Apache Cassandra has the CDC (Change Data Capture) features since 3.8. This is 
further enhanced with (CASS-12148) in Cassandra 4.0.

However, there’s no generally available mechanism to stream changes out of a 
Cassandra database; hence the utility of this feature is limited if not absent.

Many applications use Cassandra as their primary data store. For various 
reasons(Caching, analyzing, indexing, etc), this data needs to be synchronized 
with derived/secondary data stores.  We would like to emit change streams in 
real-time to consumers so that changes to Cassandra can be used for various 
purposes.

*Goals*
 * Enhance Apache Cassandra sidecar with a CDC reader that can read and emit 
changes in real-time. Priority for the initial implementation is safety and 
correctness, performance enhancements will follow in subsequent iterations

*Nongoals*
 * Modify Cassandra storage engine to emit changes

 

*Proposal*

https://docs.google.com/document/d/11YywfJTm29szZOVOSRbtfvClbmMQtJ8WyCB7_CUgo-U/edit?usp=sharing

 

  was:
Apache Cassandra has the CDC (Change Data Capture) features since 3.8. This is 
further enhanced with (CASS-12148) in Cassandra 4.0.

However, there’s no generally available mechanism to stream changes out of a 
Cassandra database; hence the utility of this feature is limited if not absent.

Many applications use Cassandra as their primary data store. For various 
reasons(Caching, analyzing, indexing, etc), this data needs to be synchronized 
with derived/secondary data stores.  We would like to emit change streams in 
real-time to consumers so that changes to Cassandra can be used for various 
purposes.

*Goals*
 * Enhance Apache Cassandra sidecar with a CDC reader that can read and emit 
changes in real-time. Priority for the initial implementation is safety and 
correctness, performance enhancements will follow in subsequent iterations

*Nongoals*
 * Modify Cassandra storage engine to emit changes

 


> CDC reader in Apache Cassandra Sidecar
> --
>
> Key: CASSANDRASC-27
> URL: https://issues.apache.org/jira/browse/CASSANDRASC-27
> Project: Sidecar for Apache Cassandra
>  Issue Type: New Feature
>Reporter: Vinay Chella
>Assignee: Tharanga Sampath Gamaethige
>Priority: Normal
>
> Apache Cassandra has the CDC (Change Data Capture) features since 3.8. This 
> is further enhanced with (CASS-12148) in Cassandra 4.0.
> However, there’s no generally available mechanism to stream changes out of a 
> Cassandra database; hence the utility of this feature is limited if not 
> absent.
> Many applications use Cassandra as their primary data store. For various 
> reasons(Caching, analyzing, indexing, etc), this data needs to be 
> synchronized with derived/secondary data stores.  We would like to emit 
> change streams in real-time to consumers so that changes to Cassandra can be 
> used for various purposes.
> *Goals*
>  * Enhance Apache Cassandra sidecar with a CDC reader that can read and emit 
> changes in real-time. Priority for the initial implementation is safety and 
> correctness, performance enhancements will follow in subsequent iterations
> *Nongoals*
>  * Modify Cassandra storage engine to emit changes
>  
> *Proposal*
> https://docs.google.com/document/d/11YywfJTm29szZOVOSRbtfvClbmMQtJ8WyCB7_CUgo-U/edit?usp=sharing
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters

2020-06-15 Thread Michael Semb Wever (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136129#comment-17136129
 ] 

Michael Semb Wever commented on CASSANDRA-15234:


[~e.dimitrova], can we pull out the part 3 (Accessors for System properties) 
patch into a separate ticket.  If it has no dependencies on the other patches 
in this ticket, then there's no reason it can't get reviewed and committed 
separately.

> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase, 
> both from the yams and JVM properties.  It would be nice to standardise the 
> naming (such as otc_ vs internode_) as well as the provision of values with 
> units - while maintaining perpetual backwards compatibility with the old 
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s, 
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or 
> powers of 1000 such as {{KB/s}}, given these are regularly used for either 
> their old or new definition e.g. {{KiB/s}}, or we could support them and 
> simply log the value in bytes/s.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15600) Algorithmic token allocation does not handle the racks = RF case well

2020-06-15 Thread Lorina Poland (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lorina Poland updated CASSANDRA-15600:
--
Impacts: Docs  (was: None)

> Algorithmic token allocation does not handle the racks = RF case well
> -
>
> Key: CASSANDRA-15600
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15600
> Project: Cassandra
>  Issue Type: Bug
>  Components: Feature/Virtual Nodes
>Reporter: Branimir Lambov
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-alpha4
>
> Attachments: CASSANDRA-15600 unit.txt, CASSANDRA-15600-jvm.txt.zip, 
> Screen Shot 2020-03-02 at 6.55.41 PM.png
>
>
> When the number of racks in a data centre is equal to the replication factor, 
> the node allocation algorithm may calculate the target ownership incorrectly 
> and allocate unsuitable tokens. This causes some inefficiency when the number 
> of nodes in the DC is small, and significant problems if the number of nodes 
> in each rack is different.
> In this case (racks count equal to replication factor) the load within each 
> rack is effectively distributed independently. The allocation algorithm 
> should reflect that, restricting the allocation space to the rack (instead of 
> the DC) and using the RF=1 solution.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14587) TrueDiskSpaceUsed overcounts snapshots

2020-06-15 Thread Lorina Poland (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lorina Poland updated CASSANDRA-14587:
--
Impacts: Docs

> TrueDiskSpaceUsed overcounts snapshots
> --
>
> Key: CASSANDRA-14587
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14587
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Tool/nodetool
> Environment: Debian 8
> Cassandra 3.11.2
>Reporter: Elliott Sims
>Priority: Low
>
> Running 'nodetool listsnapshots' seems to overcount "TrueDiskSpaceUsed" under 
> some circumstances.  Specifically when there's a large number of snapshots.  
> I suspect that it's not deduplicating space used when multiple snapshots 
> share sstables that are not part of the current table.
> Results of "nodetool listsnapshots":
> Total TrueDiskSpaceUsed: 396.11 MiB
> Results of "du -hcs" on the table's directory:
> 18M    total
> This is 50+ snapshots (every minute) run with "-t  -sf 
> --column-family  "
> The results of a "du -hcs -L  "TrueDiskSpaceUsed"
> I have only tested against 3.11.2, but have no reason to believe it's unique 
> to that version or even 3.x.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37

2020-06-15 Thread Matt Davis (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136035#comment-17136035
 ] 

Matt Davis commented on CASSANDRA-15868:


Also, would it be out of scope for this issue to update Cassandra 4 from 
netty-tcnative-boringssl-static 2.0.25 to 2.0.31? This latest version also 
includes native libraries for Arm processors.

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37

2020-06-15 Thread Matt Davis (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17136033#comment-17136033
 ] 

Matt Davis commented on CASSANDRA-15868:


[~stefan.miklosovic] - looks like you've already added the JAR for 4.1.50, only 
change would be to remove 4.1.37.

[~aleksey] - I've added a PR for Cassandra 3.11.7: 
[https://github.com/apache/cassandra/pull/634]

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe reassigned CASSANDRA-15872:
---

Assignee: ZhaoYang

> Flaky test: testNoTreesRetainedAfterDifference - 
> org.apache.cassandra.repair.RepairJobTest
> --
>
> Key: CASSANDRA-15872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15872
> Project: Cassandra
>  Issue Type: Task
>  Components: Consistency/Repair
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-rc
>
>
> link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15872:

Reviewers: Caleb Rackliffe

> Flaky test: testNoTreesRetainedAfterDifference - 
> org.apache.cassandra.repair.RepairJobTest
> --
>
> Key: CASSANDRA-15872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15872
> Project: Cassandra
>  Issue Type: Task
>  Components: Consistency/Repair
>Reporter: ZhaoYang
>Priority: Normal
> Fix For: 4.0-rc
>
>
> link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135970#comment-17135970
 ] 

Caleb Rackliffe commented on CASSANDRA-15872:
-

I can probably take a look this week.

> Flaky test: testNoTreesRetainedAfterDifference - 
> org.apache.cassandra.repair.RepairJobTest
> --
>
> Key: CASSANDRA-15872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15872
> Project: Cassandra
>  Issue Type: Task
>  Components: Consistency/Repair
>Reporter: ZhaoYang
>Priority: Normal
> Fix For: 4.0-rc
>
>
> link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15868:
--
Status: Changes Suggested  (was: Review In Progress)

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15868:
--
Reviewers: Aleksey Yeschenko

Hi. Could you please update the bundled jar (in lib/) too? Cheers.

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15868:
--
Reviewers: Aleksey Yeschenko, Aleksey Yeschenko  (was: Aleksey Yeschenko)
   Aleksey Yeschenko, Aleksey Yeschenko  (was: Aleksey Yeschenko)
   Status: Review In Progress  (was: Patch Available)

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15868:
--
Test and Documentation Plan: N/A
 Status: Patch Available  (was: Open)

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15868:
--
Change Category: Operability
 Complexity: Low Hanging Fruit
 Status: Open  (was: Triage Needed)

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-15868) Update Netty version to 4.1.50 because there are security issues in 4.1.37



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko reassigned CASSANDRA-15868:
-

Assignee: Stefan Miklosovic

> Update Netty version to 4.1.50 because there are security issues in 4.1.37
> --
>
> Key: CASSANDRA-15868
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15868
> Project: Cassandra
>  Issue Type: Task
>  Components: Dependencies
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.11.7, 4.0-beta
>
> Attachments: dependency-check-report.html
>
>
> Please see attached HTML report from OWASP dependency check.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15864) WARN [CompactionExecutor:5570] 2020-06-09 02:27:01,011 LeveledCompactionStrategy.java:144 - Could not acquire references for compacting SSTables



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15864:
--
Resolution: Not A Problem
Status: Resolved  (was: Triage Needed)

Hi Muhammad. This isn't necessarily a problem. Does it happen frequently for 
you? If so, please feel free to reopen the JIRA.

> WARN  [CompactionExecutor:5570] 2020-06-09 02:27:01,011 
> LeveledCompactionStrategy.java:144 - Could not acquire references for 
> compacting SSTables
> -
>
> Key: CASSANDRA-15864
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15864
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction/LCS, Local/SSTable, Tool/sstable
>Reporter: Muhammad Zubair Saleem
>Priority: Normal
>
> WARN [CompactionExecutor:5570] 2020-06-09 02:27:01,011 
> LeveledCompactionStrategy.java:144 - Could not acquire references for 
> compacting SSTables 
> [BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-118925-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119279-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119319-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119254-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119229-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119264-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119149-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119269-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119214-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119314-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119204-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119239-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119351-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119249-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119194-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119294-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119304-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119125-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119234-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119190-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119324-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119259-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119284-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119219-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119209-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119274-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119289-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119224-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119309-big-Data.db'),
>  
> BigTableReader(path='/var/lib/cassandra-3/data/KEYSPACE/TABLE-27b25230f4a111e88e158fc9322074d6/mc-119199-big-Data.db'),
>  
>

[jira] [Updated] (CASSANDRA-15866) stream sstable attached index files entirely with data file



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15866:
--
Change Category: Performance
 Complexity: Normal
 Status: Open  (was: Triage Needed)

> stream sstable attached index files entirely with data file
> ---
>
> Key: CASSANDRA-15866
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15866
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Consistency/Streaming
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
>
> When sstable is streamed entirely, there is no need to rebuild sstable 
> attached index on receiver if index files can be streamed entirely.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15872) Flaky test: testNoTreesRetainedAfterDifference - org.apache.cassandra.repair.RepairJobTest



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15872:
--
Change Category: Operability
 Complexity: Normal
 Status: Open  (was: Triage Needed)

[~maedhroz][~bdeggleston] Either of you guys want to have a look? A small patch.

> Flaky test: testNoTreesRetainedAfterDifference - 
> org.apache.cassandra.repair.RepairJobTest
> --
>
> Key: CASSANDRA-15872
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15872
> Project: Cassandra
>  Issue Type: Task
>  Components: Consistency/Repair
>Reporter: ZhaoYang
>Priority: Normal
> Fix For: 4.0-rc
>
>
> link: [https://circleci.com/gh/maedhroz/cassandra/21#tests/containers/95]
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15861) Mutating sstable STATS metadata may race with entire-sstable-streaming(ZCS) causing checksum validation failure

2020-06-15 Thread ZhaoYang (Jira)



[ 
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17135889#comment-17135889
 ] 

ZhaoYang commented on CASSANDRA-15861:
--

[~marcuse] [~djoshi] it looks like you are pretty experienced with compaction 
code, do you see any issue with first proposal? 

> Mutating sstable STATS metadata may race with entire-sstable-streaming(ZCS) 
> causing checksum validation failure
> ---
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair, Consistency/Streaming
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator - 
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> In the above test, it executes "nodetool repair" on node1 and kills node2 
> during repair. At the end, node3 reports checksum validation failure on 
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies 
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be 
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1 
> starts to broadcast repair-failure-message to all participants in 
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair 
> sessions at {{LocalSessions#failSession}} which triggers async background 
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and 
> pending repair id to null via  
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more 
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS 
> component size is different from the original size recorded in the manifest.
> 7. At the end, node3 reports checksum validation failure when it tries to 
> mutate sstable level and "isTransient" attribute in 
> {{CassandraEntireSSTableStreamReader#read}}.
> {code}
> I believe similar race may happen with level compaction where it may directly 
> mutate a sstable's level if it doesn't overlap with sstables at next level. 
> (Note: this isn't a problem in legacy streaming as STATS file length didn't 
> matter.)
> Ideally it will be great to make sstable STATS metadata immutable, just like 
> other sstable components, so we don't have to worry this special case. 
> I can think of two ways:
> # Change {{RepairFinishedCompactionTask}}, {{AntiCompaction}} and 
> {{SingleSSTableLCSTask}} to create hard link on the compacting sstable 
> components with a new descriptor, except STATS files which will be copied 
> entirely. Then mutation will be applied on the new STATS file. At the end, 
> old sstable will be released. This ensures all sstable components are 
> immutable and shouldn't make these special compaction tasks slower.
> # Hacky approach: load the small STATS file into memory when initializing 
> {{CassandraOutgoingFile}} instead of relying on mutable on-disk STATS file.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15861) Mutating sstable STATS metadata may race with entire-sstable-streaming(ZCS) causing checksum validation failure

2020-06-15 Thread ZhaoYang (Jira)



 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhaoYang updated CASSANDRA-15861:
-
Description: 
Flaky dtest: [test_dead_sync_initiator - 
repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]

In the above test, it executes "nodetool repair" on node1 and kills node2 
during repair. At the end, node3 reports checksum validation failure on sstable 
transferred from node1.
{code:java|title=what happened}
1. When repair started on node1, it performs anti-compaction which modifies 
sstable's repairAt to 0 and pending repair id to session-id.
2. Then node1 creates {{ComponentManifest}} which contains file lengths to be 
transferred to node3.
3. Before node1 actually sends the files to node3, node2 is killed and node1 
starts to broadcast repair-failure-message to all participants in 
{{CoordinatorSession#fail}}
4. Node1 receives its own repair-failure-message and fails its local repair 
sessions at {{LocalSessions#failSession}} which triggers async background 
compaction.
5. Node1's background compaction will mutate sstable's repairAt to 0 and 
pending repair id to null via  
{{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more 
in-progress repair.
6. Node1 actually sends the sstable to node3 where the sstable's STATS 
component size is different from the original size recorded in the manifest.
7. At the end, node3 reports checksum validation failure when it tries to 
mutate sstable level and "isTransient" attribute in 
{{CassandraEntireSSTableStreamReader#read}}.
{code}

I believe similar race may happen with level compaction where it may directly 
mutate a sstable's level if it doesn't overlap with sstables at next level. 
(Note: this isn't a problem in legacy streaming as STATS file length didn't 
matter.)

Ideally it will be great to make sstable STATS metadata immutable, just like 
other sstable components, so we don't have to worry this special case. 

I can think of two ways:
# Change {{RepairFinishedCompactionTask}}, {{AntiCompaction}} and 
{{SingleSSTableLCSTask}} to create hard link on the compacting sstable 
components with a new descriptor, except STATS files which will be copied 
entirely. Then mutation will be applied on the new STATS file. At the end, old 
sstable will be released. This ensures all sstable components are immutable and 
shouldn't make these special compaction tasks slower.
# Hacky approach: load the small STATS file into memory when initializing 
{{CassandraOutgoingFile}} instead of relying on mutable on-disk STATS file.

  was:
Flaky dtest: [test_dead_sync_initiator - 
repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]

In the above test, it executes "nodetool repair" on node1 and kills node2 
during repair. At the end, node3 reports checksum validation failure on sstable 
transferred from node1.
{code:java|title=what happened}
1. When repair started on node1, it performs anti-compaction which modifies 
sstable's repairAt to 0 and pending repair id to session-id.
2. Then node1 creates {{ComponentManifest}} which contains file lengths to be 
transferred to node3.
3. Before node1 actually sends the files to node3, node2 is killed and node1 
starts to broadcast repair-failure-message to all participants in 
{{CoordinatorSession#fail}}
4. Node1 receives its own repair-failure-message and fails its local repair 
sessions at {{LocalSessions#failSession}} which triggers async background 
compaction.
5. Node1's background compaction will mutate sstable's repairAt to 0 and 
pending repair id to null via  
{{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more 
in-progress repair.
6. Node1 actually sends the sstable to node3 where the sstable's STATS 
component size is different from the original size recorded in the manifest.
7. At the end, node3 reports checksum validation failure when it tries to 
mutate sstable level and "isTransient" attribute in 
{{CassandraEntireSSTableStreamReader#read}}.
{code}

I believe similar race may happen with level compaction where it may directly 
mutate a sstable's level if it doesn't overlap with sstables at next level. 
(Note: this isn't a problem in legacy streaming as STATS file length didn't 
matter.)

Ideally it will be great to make sstable STATS metadata immutable, just like 
other sstable components, so we don't have to worry this special case. 

I can think of two ways:
# Change {{RepairFinishedCompactionTask}} and {{SingleSSTableLCSTask}} to 
create hard link on the compacting sstable components with a new descriptor, 
except STATS files which will be copied entirely. Then mutation will be applied 
on

[jira] [Commented] (CASSANDRA-15262) Update defaults for server and client TLS settings