[jira] [Created] (CASSANDRA-16066) Update and rework cassandra-website material to work with Antora

[Anthony Grasso (Jira)]

Anthony Grasso created CASSANDRA-16066:
--

 Summary: Update and rework cassandra-website material to work with 
Antora
 Key: CASSANDRA-16066
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16066
 Project: Cassandra
  Issue Type: Task
  Components: Documentation/Website
Reporter: Anthony Grasso


*We want to modernise the way the website is built*
 Rework the cassandra-website repository to generate a UI bundle containing 
resources that Antora will use when generating the Cassandra documents and 
website.

*The existing method is starting to become dated*
 The documentation and website templates are currently in markdown format. 
Sphinx is used to generate the Cassandra documentation and Jekyll generates the 
website content. One of the major issues with the existing website tooling is 
that the live preview server (render site as it is being updated) is really 
slow. There is a preview server that is really fast, however it is unable to 
detect changes to the content and render automatically.

*We are migrating the docs to be rendered with Antora*
 The work in CASSANDRA-16029 is converting the document templates to AsciiDoc 
format. Sphinx is being replaced by Antora to generate the documentation 
content. This change has two advantages:
 * More flexibility if the Apache Cassandra documentation look and feel needs 
to be updated or redesigned.
 * More modern look and feel to the documentation.

*We can use Antora to generate the website as well*
 Antora could also be used to generate the Cassandra website content. As 
suggested on the [mailing 
list|https://www.mail-archive.com/dev@cassandra.apache.org/msg15577.html] this 
would require the existing markdown templates to be converted to AsciiDoc as 
well.

*Antora needs a UI bundle to style content*
 For Antora to generate the document content and potentially the website 
content it requires a UI bundle (ui-bundle.zip). The UI bundle contains the 
HTML templates (layouts, partials, and helpers), CSS, JavaScript, fonts, and 
(site-wide) images. As such, it provides both the visual theme and user 
interactions for the documentation. Effectively the UI bundle is the templates 
and styling that are applied to the documentation and website content.

*The [cassandra-website|https://github.com/apache/cassandra-website] repository 
can be used to generate the UI bundle*
 All the resources associated with templating and styling the documentation and 
website can be placed in the 
[cassandra-website|https://github.com/apache/cassandra-website] repository. In 
this case the repository would serve two purposes;
 * Generation of the UI bundle resources.
 * Serve the production website content.

*The [cassandra|https://github.com/apache/cassandra] repository would contain 
the site material*
 All other material that is used to generate documentation, blogs and project 
information would live in the [cassandra|https://github.com/apache/cassandra] 
repository. In this case Antora would run on the 
[cassandra/doc|https://github.com/apache/cassandra/doc] directory and pull in a 
UI bundle released on the GitHub 
[cassandra-website|https://github.com/apache/cassandra-website] repository. The 
content generated by Antora would be placed in the 
[cassandra-website/content|https://github.com/apache/cassandra-website/content] 
directory

*Design, content, and layout would remain the same*
 This proposal means the roles of each repository will change. In addition, the 
workflow to publish material will change as well. However, the website design, 
content, and layout will remain the same.

As an added bonus the tooling would allow for a live preview server that is 
fast and responsive. However, the time to build and generate the {{nodetool}} 
and Cassandra YAML AssciDoc files will still take in the order of minutes to 
complete.

*The [cassandra-website|https://github.com/apache/cassandra-website] repository 
would need to be modified*
 The following changes would need to be made to the 
[cassandra-website|https://github.com/apache/cassandra-website] repository:
||File/Directory||Action||Reason||
|[content/|https://github.com/apache/cassandra-website/content]|keep|Production 
site content served from this directory|
|[src/_data/|https://github.com/apache/cassandra-website/src/_data]|delete|_site.yml_
 and _antora.yml_ include this info|
|[src/_includes/|https://github.com/apache/cassandra-website/src/_includes]|delete|Replace
 with UI bundle components|
|[src/_layout/|https://github.com/apache/cassandra-website/src/_layout]|delete|Replace
 with UI bundle components|
|[src/_plugins/|https://github.com/apache/cassandra-website/src/_plugins]|delete|Replace
 with UI bundle components|
|[src/_posts/|https://github.com/apache/cassandra-website/src/_posts]|move|Convert
 to AsciiDoc format and place in 
[cassandra|https://github.com/apache/cassandra] 

[jira] [Updated] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15909:

Status: Changes Suggested  (was: Review In Progress)

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-beta
>
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181474#comment-17181474
 ] 

Caleb Rackliffe commented on CASSANDRA-16064:
-

bq. I should backport this to 3.0

Fair. Either way, I'll try to have feedback on trunk by tomorrow.

> Add test which validates that Message serializedSize(version) == 
> serialize(out, version).length
> ---
>
> Key: CASSANDRA-16064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16064
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> In 4.0 we require serializedSize(version) == serialize(out, version).length 
> for correctness in post40 message format as we write it into the message 
> header.  Given that this is a strong requirement for correct deserialization 
> of the message, we should have tests which help enforce this property.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16065) Distinguish partition errors published by Cassandra-Diff between source and target

[Yifan Cai (Jira)]

Yifan Cai created CASSANDRA-16065:
-

 Summary: Distinguish partition errors published by Cassandra-Diff 
between source and target
 Key: CASSANDRA-16065
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16065
 Project: Cassandra
  Issue Type: Improvement
  Components: Tool/diff
Reporter: Yifan Cai


The partition errors found during diff are persisted without the error origin 
information. 

Therefore, I am proposing to add an error origin indicator, (e.g. 0 for source 
and 1 for target) when persisting the partition error details. 




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length

[David Capwell (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181464#comment-17181464
 ] 

David Capwell commented on CASSANDRA-16064:
---

I also feel I should backport this to 3.0.  I started this in 3.0 and found a 
bug where we don't include the table name's size in our estimate.  We didn't 
use sizeOf for correctness like we do in 4.0, so less critical.

> Add test which validates that Message serializedSize(version) == 
> serialize(out, version).length
> ---
>
> Key: CASSANDRA-16064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16064
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> In 4.0 we require serializedSize(version) == serialize(out, version).length 
> for correctness in post40 message format as we write it into the message 
> header.  Given that this is a strong requirement for correct deserialization 
> of the message, we should have tests which help enforce this property.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-16064:

Reviewers: Caleb Rackliffe

> Add test which validates that Message serializedSize(version) == 
> serialize(out, version).length
> ---
>
> Key: CASSANDRA-16064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16064
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> In 4.0 we require serializedSize(version) == serialize(out, version).length 
> for correctness in post40 message format as we write it into the message 
> header.  Given that this is a strong requirement for correct deserialization 
> of the message, we should have tests which help enforce this property.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length

[David Capwell (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Capwell updated CASSANDRA-16064:
--
 Bug Category: Parent values: Correctness(12982)Level 1 values: Test 
Failure(12990)
   Complexity: Low Hanging Fruit
Discovered By: Unit Test
Fix Version/s: 4.0-beta
 Severity: Normal
   Status: Open  (was: Triage Needed)

> Add test which validates that Message serializedSize(version) == 
> serialize(out, version).length
> ---
>
> Key: CASSANDRA-16064
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16064
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0-beta
>
>
> In 4.0 we require serializedSize(version) == serialize(out, version).length 
> for correctness in post40 message format as we write it into the message 
> header.  Given that this is a strong requirement for correct deserialization 
> of the message, we should have tests which help enforce this property.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length

[David Capwell (Jira)]

David Capwell created CASSANDRA-16064:
-

 Summary: Add test which validates that Message 
serializedSize(version) == serialize(out, version).length
 Key: CASSANDRA-16064
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16064
 Project: Cassandra
  Issue Type: Bug
  Components: Messaging/Internode
Reporter: David Capwell
Assignee: David Capwell


In 4.0 we require serializedSize(version) == serialize(out, version).length for 
correctness in post40 message format as we write it into the message header.  
Given that this is a strong requirement for correct deserialization of the 
message, we should have tests which help enforce this property.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181445#comment-17181445
 ] 

Caleb Rackliffe commented on CASSANDRA-15909:
-

I've made a pass at review and left my comments 
[inline|https://github.com/spmallette/cassandra/pull/2]. Most of my suggestions 
are simple clean-ups and readability enhancements. [~spmallette] When those are 
in a good place, I'd be more than willing to kick off the necessary CI runs.

>From the discussion I've seen so far, it doesn't look like we have any 
>appetite to do this in 4.0, but I'd really push to remove the "alias" metric 
>name factory in {{TableMetrics}}, which is what is making this patch more 
>complex than it could be, in our next major release. At minimum, we should use 
>CASSANDRA-15821 to indicate very clearly (via {{metrics.rst}}) that both the 
>aliases we've added here and the "ColumnFamily"-based paths are going away 
>eventually.

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-beta
>
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16053) cqlsh COPY functions broken in Python 3.8 on Mac

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16053:
---
Fix Version/s: 4.0

> cqlsh COPY functions broken in Python 3.8 on Mac
> 
>
> Key: CASSANDRA-16053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0, 4.0-beta2
>
>
> {noformat}
> aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
> 3.8.2
> aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
> Using CQL driver:  '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
> Using connect timeout: 5 seconds
> Using 'utf-8' encoding
> Using ssl: False
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> copy test.testcopyto to 'asdf';
> Detected 12 core(s)
> Using 11 child processes
> Starting copy of test.testcopyto with columns [a, b, c, d].
> Traceback (most recent call last):
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
> self.handle_statement(st, statementtext)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in 
> handle_statement
> return custom_handler(parsed)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in 
> do_copy
> task.run()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 669, in run
> self.start_processes()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 471, in start_processes
> process.start()
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py",
>  line 121, in start
> self._popen = self._Popen(self)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 224, in _Popen
> return _default_context.get_context().Process._Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 283, in _Popen
> return Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 32, in __init__
> super().__init__(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py",
>  line 19, in __init__
> self._launch(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 47, in _launch
> reduction.dump(process_obj, fp)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py",
>  line 60, in dump
> ForkingPickler(file, protocol).dump(obj)
> TypeError: cannot pickle '_thread.lock' object
> cqlsh>
> {noformat}
> multiprocessing uses a different default start method on Mac, and pickling 
> fails trying to serialize the Cluster object.
> https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
> https://bugs.python.org/issue33725



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16053) cqlsh COPY functions broken in Python 3.8 on Mac

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16053:
-
Reviewers: Brandon Williams, Ekaterina Dimitrova  (was: Ekaterina Dimitrova)

> cqlsh COPY functions broken in Python 3.8 on Mac
> 
>
> Key: CASSANDRA-16053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta2
>
>
> {noformat}
> aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
> 3.8.2
> aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
> Using CQL driver:  '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
> Using connect timeout: 5 seconds
> Using 'utf-8' encoding
> Using ssl: False
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> copy test.testcopyto to 'asdf';
> Detected 12 core(s)
> Using 11 child processes
> Starting copy of test.testcopyto with columns [a, b, c, d].
> Traceback (most recent call last):
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
> self.handle_statement(st, statementtext)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in 
> handle_statement
> return custom_handler(parsed)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in 
> do_copy
> task.run()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 669, in run
> self.start_processes()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 471, in start_processes
> process.start()
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py",
>  line 121, in start
> self._popen = self._Popen(self)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 224, in _Popen
> return _default_context.get_context().Process._Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 283, in _Popen
> return Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 32, in __init__
> super().__init__(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py",
>  line 19, in __init__
> self._launch(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 47, in _launch
> reduction.dump(process_obj, fp)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py",
>  line 60, in dump
> ForkingPickler(file, protocol).dump(obj)
> TypeError: cannot pickle '_thread.lock' object
> cqlsh>
> {noformat}
> multiprocessing uses a different default start method on Mac, and pickling 
> fails trying to serialize the Cluster object.
> https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
> https://bugs.python.org/issue33725



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16053) cqlsh COPY functions broken in Python 3.8 on Mac

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16053:
-
  Since Version: 4.0-alpha1
Source Control Link: 
https://github.com/apache/cassandra/commit/0e53379ec945e07c38aed03048ba8f76d42bdd42
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed, thanks.

> cqlsh COPY functions broken in Python 3.8 on Mac
> 
>
> Key: CASSANDRA-16053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta2
>
>
> {noformat}
> aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
> 3.8.2
> aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
> Using CQL driver:  '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
> Using connect timeout: 5 seconds
> Using 'utf-8' encoding
> Using ssl: False
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> copy test.testcopyto to 'asdf';
> Detected 12 core(s)
> Using 11 child processes
> Starting copy of test.testcopyto with columns [a, b, c, d].
> Traceback (most recent call last):
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
> self.handle_statement(st, statementtext)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in 
> handle_statement
> return custom_handler(parsed)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in 
> do_copy
> task.run()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 669, in run
> self.start_processes()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 471, in start_processes
> process.start()
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py",
>  line 121, in start
> self._popen = self._Popen(self)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 224, in _Popen
> return _default_context.get_context().Process._Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 283, in _Popen
> return Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 32, in __init__
> super().__init__(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py",
>  line 19, in __init__
> self._launch(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 47, in _launch
> reduction.dump(process_obj, fp)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py",
>  line 60, in dump
> ForkingPickler(file, protocol).dump(obj)
> TypeError: cannot pickle '_thread.lock' object
> cqlsh>
> {noformat}
> multiprocessing uses a different default start method on Mac, and pickling 
> fails trying to serialize the Cluster object.
> https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
> https://bugs.python.org/issue33725



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16053) cqlsh COPY functions broken in Python 3.8 on Mac

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16053:
-
Status: Ready to Commit  (was: Review In Progress)

> cqlsh COPY functions broken in Python 3.8 on Mac
> 
>
> Key: CASSANDRA-16053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta2
>
>
> {noformat}
> aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
> 3.8.2
> aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
> Using CQL driver:  '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
> Using connect timeout: 5 seconds
> Using 'utf-8' encoding
> Using ssl: False
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> copy test.testcopyto to 'asdf';
> Detected 12 core(s)
> Using 11 child processes
> Starting copy of test.testcopyto with columns [a, b, c, d].
> Traceback (most recent call last):
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
> self.handle_statement(st, statementtext)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in 
> handle_statement
> return custom_handler(parsed)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in 
> do_copy
> task.run()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 669, in run
> self.start_processes()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 471, in start_processes
> process.start()
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py",
>  line 121, in start
> self._popen = self._Popen(self)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 224, in _Popen
> return _default_context.get_context().Process._Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 283, in _Popen
> return Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 32, in __init__
> super().__init__(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py",
>  line 19, in __init__
> self._launch(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 47, in _launch
> reduction.dump(process_obj, fp)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py",
>  line 60, in dump
> ForkingPickler(file, protocol).dump(obj)
> TypeError: cannot pickle '_thread.lock' object
> cqlsh>
> {noformat}
> multiprocessing uses a different default start method on Mac, and pickling 
> fails trying to serialize the Cluster object.
> https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
> https://bugs.python.org/issue33725



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: stop passing unpicklable Cluster object to spawned child processes in cqlsh

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 0e53379  stop passing unpicklable Cluster object to spawned child 
processes in cqlsh
0e53379 is described below

commit 0e53379ec945e07c38aed03048ba8f76d42bdd42
Author: Adam Holmberg 
AuthorDate: Mon Aug 17 15:01:23 2020 -0500

stop passing unpicklable Cluster object to spawned child processes in
cqlsh

Patch by Adam Holmberg, reviewed by Ekaterina Dimitrova and
brandonwilliams for CASSANDRA-16053
---
 CHANGES.txt|  1 +
 pylib/cqlshlib/copyutil.py | 23 ---
 2 files changed, 5 insertions(+), 19 deletions(-)

diff --git a/CHANGES.txt b/CHANGES.txt
index 3272226..f4efcbd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-beta2
+ * fix cqlsh COPY functions in Python 3.8 on Mac (CASSANDRA-16053)
  * Strip comment blocks from cqlsh input before processing statements 
(CASSANDRA-15802)
  * Fix unicode chars error input (CASSANDRA-15990)
  * Improved testability for CacheMetrics and ChunkCacheMetrics 
(CASSANDRA-15788)
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 169a6e0..ce4d35d 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -67,7 +67,6 @@ PROFILE_ON = False
 STRACE_ON = False
 DEBUG = False  # This may be set to True when initializing the task
 IS_LINUX = platform.system() == 'Linux'
-IS_WINDOWS = platform.system() == 'Windows'
 
 CopyOptions = namedtuple('CopyOptions', 'copy dialect unrecognized')
 
@@ -480,10 +479,8 @@ class CopyTask(object):
 def make_params(self):
 """
 Return a dictionary of parameters to be used by the worker processes.
-On Windows this dictionary must be pickle-able, therefore we do not 
pass the
-parent connection since it may not be pickle-able. Also, on Windows 
child
-processes are spawned and not forked, and therefore we don't need to 
shutdown
-the parent connection anyway, see CASSANDRA-11749 for more details.
+On platforms using 'spawn' as the default multiprocessing start method,
+this dictionary must be picklable.
 """
 shell = self.shell
 
@@ -497,7 +494,6 @@ class CopyTask(object):
 port=shell.port,
 ssl=shell.ssl,
 auth_provider=shell.auth_provider,
-parent_cluster=shell.conn if not IS_WINDOWS else None,
 cql_version=shell.conn.cql_version,
 config_file=self.config_file,
 protocol_version=self.protocol_version,
@@ -1171,8 +1167,7 @@ class ImportTask(CopyTask):
 self.processes.append(ImportProcess(self.update_params(params, 
i)))
 
 feeder = FeedingProcess(self.outmsg.pipes[-1], 
self.inmsg.pipes[-1],
-self.outmsg.pipes[:-1], self.fname, 
self.options,
-self.shell.conn if not IS_WINDOWS else 
None)
+self.outmsg.pipes[:-1], self.fname, 
self.options)
 self.processes.append(feeder)
 
 self.start_processes()
@@ -1291,7 +1286,7 @@ class FeedingProcess(mp.Process):
 """
 A process that reads from import sources and sends chunks to worker 
processes.
 """
-def __init__(self, inpipe, outpipe, worker_pipes, fname, options, 
parent_cluster):
+def __init__(self, inpipe, outpipe, worker_pipes, fname, options):
 super(FeedingProcess, self).__init__(target=self.run)
 self.inpipe = inpipe
 self.outpipe = outpipe
@@ -1305,7 +1300,6 @@ class FeedingProcess(mp.Process):
 self.num_worker_processes = options.copy['numprocesses']
 self.max_pending_chunks = options.copy['maxpendingchunks']
 self.chunk_id = 0
-self.parent_cluster = parent_cluster
 
 def on_fork(self):
 """
@@ -1316,10 +1310,6 @@ class FeedingProcess(mp.Process):
 self.outmsg = SendingChannel(self.outpipe)
 self.worker_channels = [SendingChannel(p) for p in self.worker_pipes]
 
-if self.parent_cluster:
-printdebugmsg("Closing parent cluster sockets")
-self.parent_cluster.shutdown()
-
 def run(self):
 pr = profile_on() if PROFILE_ON else None
 
@@ -1419,7 +1409,6 @@ class ChildProcess(mp.Process):
 self.connect_timeout = params['connect_timeout']
 self.cql_version = params['cql_version']
 self.auth_provider = params['auth_provider']
-self.parent_cluster = params['parent_cluster']
 self.ssl = params['ssl']
 self.protocol_version = params['protocol_version']
 self.config_file = params['config_file']
@@ -1451,10 +1440,6 @@ class ChildProcess(mp.Process):
 self.inmsg = 

[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Brandon Williams (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181414#comment-17181414
 ] 

Brandon Williams commented on CASSANDRA-16063:
--

I agree and would even say that running upgradesstables as part of the upgrade 
process is the normal case, not the exception.

> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Priority: Normal
>
> The code to handle compact tables has been removed from 4.0, and the intended 
> upgrade path to 4.0 for users having compact tables on 3.x is that they must 
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables 
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table) 
> and may try upgrading despite still having compact tables. If they do so, the 
> intent is that the node will _not_ start, with a message clearly indicating 
> the pre-upgrade step the user has missed. The user will then downgrade back 
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and 
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables 
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within 
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
>   But by then, we've _at least_ called 
> {{SystemKeyspace.persistLocalMetadata()}}} and 
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log, 
> and even possibly flush new {{na}} format sstables. As a results, a user 
> might not be able to seemlessly restart the node on 3.x (to drop compact 
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0 
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0 
> with compact storage, you can downgrade back with no intervention whatsoever).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Jordan West (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181413#comment-17181413
 ] 

Jordan West edited comment on CASSANDRA-16063 at 8/20/20, 7:54 PM:
---

Another concern I have with the existing approach is if users run upgrade 
sstables as part of their upgrade process instead of letting the node do it 
while online. We should ensure the check is there as well. 


was (Author: jrwest):
Another concern I would have with this approach is if users run upgrade 
sstables as part of their upgrade process instead of letting the node do it 
while online. We should ensure the check is there as well. 

> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Priority: Normal
>
> The code to handle compact tables has been removed from 4.0, and the intended 
> upgrade path to 4.0 for users having compact tables on 3.x is that they must 
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables 
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table) 
> and may try upgrading despite still having compact tables. If they do so, the 
> intent is that the node will _not_ start, with a message clearly indicating 
> the pre-upgrade step the user has missed. The user will then downgrade back 
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and 
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables 
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within 
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
>   But by then, we've _at least_ called 
> {{SystemKeyspace.persistLocalMetadata()}}} and 
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log, 
> and even possibly flush new {{na}} format sstables. As a results, a user 
> might not be able to seemlessly restart the node on 3.x (to drop compact 
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0 
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0 
> with compact storage, you can downgrade back with no intervention whatsoever).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Jordan West (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181413#comment-17181413
 ] 

Jordan West commented on CASSANDRA-16063:
-

Another concern I would have with this approach is if users run upgrade 
sstables as part of their upgrade process instead of letting the node do it 
while online. We should ensure the check is there as well. 

> Fix user experience when upgrading to 4.0 with compact tables
> -
>
> Key: CASSANDRA-16063
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
> Project: Cassandra
>  Issue Type: Bug
>  Components: Legacy/CQL
>Reporter: Sylvain Lebresne
>Priority: Normal
>
> The code to handle compact tables has been removed from 4.0, and the intended 
> upgrade path to 4.0 for users having compact tables on 3.x is that they must 
> execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables 
> *before* attempting the upgrade.
> Obviously, some users won't read the upgrade instructions (or miss a table) 
> and may try upgrading despite still having compact tables. If they do so, the 
> intent is that the node will _not_ start, with a message clearly indicating 
> the pre-upgrade step the user has missed. The user will then downgrade back 
> the node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and 
> then upgrade again.
> But while 4.0 does currently fail startup when finding any compact tables 
> with a decent message, I believe the check is done too late during startup.
> Namely, that check is done as we read the tables schema, so within 
> [{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
>   But by then, we've _at least_ called 
> {{SystemKeyspace.persistLocalMetadata()}}} and 
> {{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log, 
> and even possibly flush new {{na}} format sstables. As a results, a user 
> might not be able to seemlessly restart the node on 3.x (to drop compact 
> storage on the appropriate tables).
> Basically, we should make sure the check for compact tables done at 4.0 
> startup is done as a {{StartupCheck}}, before the node does anything.
> We should also add a test for this (checking that if you try upgrading to 4.0 
> with compact storage, you can downgrade back with no intervention whatsoever).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns

[Jordan West (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181410#comment-17181410
 ] 

Jordan West edited comment on CASSANDRA-16048 at 8/20/20, 7:49 PM:
---

{quote}this has user visible consequences (no more "WITH COMPACT STORAGE" 
mention when using DESCRIBED and removed DENSE flag), even if they can arguably 
be considered minor.{quote}.

Agreed. I should have been more specific that I was referring to visible 
changes to query results as shown by the included test. 

{quote}Without this patch, users are obligated to manually run DROP COMPACT 
STORAGE on all tables, so no risk of surprise exists. Which is kind of a 
feature in that case imo.{quote}

In practice, I am finding this to not be the case. Instead, its a blocker to 
upgrading that will cause a significant burden and time commitment including 
user outreach, etc. 

{quote} to ask a few power users to run more manual DROP COMPACT STORAGE (it's 
not like it can't be scripted externally easily if you know what you're doing) 
{quote}

This is an option I explored as well. While the scripting might be easy, the 
risk to organizations doing so is more complex. It also requires identifying 
the types of tables that are safe to call {{DROP COMPACT STORAGE}} on vs the 
ones that will necessarily result in a user visible query result change like 
this patch does. That can be done in the script but I've found it more 
straightforward to implement and test w/in the database as done in this patch. 

I'll take a look at CASSANDRA-16063. Thanks. 



was (Author: jrwest):
{quote}this has user visible consequences (no more "WITH COMPACT STORAGE" 
mention when using DESCRIBED and removed DENSE flag), even if they can arguably 
be considered minor.{/quote}.

Agreed. I should have been more specific that I was referring to visible 
changes to query results as shown by the included test. 

{quote}Without this patch, users are obligated to manually run DROP COMPACT 
STORAGE on all tables, so no risk of surprise exists. Which is kind of a 
feature in that case imo.{quote}

In practice, I am finding this to not be the case. Instead, its a blocker to 
upgrading that will cause a significant burden and time commitment including 
user outreach, etc. 

{quote} to ask a few power users to run more manual DROP COMPACT STORAGE (it's 
not like it can't be scripted externally easily if you know what you're doing) 
{quote}

This is an option I explored as well. While the scripting might be easy, the 
risk to organizations doing so is more complex. It also requires identifying 
the types of tables that are safe to call {{DROP COMPACT STORAGE}} on vs the 
ones that will necessarily result in a user visible query result change like 
this patch does. That can be done in the script but I've found it more 
straightforward to implement and test w/in the database as done in this patch. 

I'll take a look at CASSANDRA-16063. Thanks. 


> Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and 
> Value Columns
> --
>
> Key: CASSANDRA-16048
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16048
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/CQL
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Some compact storage tables, specifically those where the user has defined 
> both at least one clustering and the value column, can be safely handled in 
> 4.0 because besides the DENSE flag they are not materially different post 3.0 
> and there is no visible change to the user facing schema after dropping 
> compact storage. We can detect this case and allow these tables to silently 
> drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE 
> tables that don’t meet the criteria. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns

[Jordan West (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181410#comment-17181410
 ] 

Jordan West commented on CASSANDRA-16048:
-

{quote}this has user visible consequences (no more "WITH COMPACT STORAGE" 
mention when using DESCRIBED and removed DENSE flag), even if they can arguably 
be considered minor.{/quote}.

Agreed. I should have been more specific that I was referring to visible 
changes to query results as shown by the included test. 

{quote}Without this patch, users are obligated to manually run DROP COMPACT 
STORAGE on all tables, so no risk of surprise exists. Which is kind of a 
feature in that case imo.{quote}

In practice, I am finding this to not be the case. Instead, its a blocker to 
upgrading that will cause a significant burden and time commitment including 
user outreach, etc. 

{quote} to ask a few power users to run more manual DROP COMPACT STORAGE (it's 
not like it can't be scripted externally easily if you know what you're doing) 
{quote}

This is an option I explored as well. While the scripting might be easy, the 
risk to organizations doing so is more complex. It also requires identifying 
the types of tables that are safe to call {{DROP COMPACT STORAGE}} on vs the 
ones that will necessarily result in a user visible query result change like 
this patch does. That can be done in the script but I've found it more 
straightforward to implement and test w/in the database as done in this patch. 

I'll take a look at CASSANDRA-16063. Thanks. 


> Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and 
> Value Columns
> --
>
> Key: CASSANDRA-16048
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16048
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/CQL
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Some compact storage tables, specifically those where the user has defined 
> both at least one clustering and the value column, can be safely handled in 
> 4.0 because besides the DENSE flag they are not materially different post 3.0 
> and there is no visible change to the user facing schema after dropping 
> compact storage. We can detect this case and allow these tables to silently 
> drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE 
> tables that don’t meet the criteria. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15821) Metrics Documentation Enhancements

[Caleb Rackliffe (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181399#comment-17181399
 ] 

Caleb Rackliffe commented on CASSANDRA-15821:
-

[~spmallette] Would deprecating the "ColumnFamily" type in favor of "Table", at 
least in the documentation, be part of the scope of this Jira?

> Metrics Documentation Enhancements
> --
>
> Key: CASSANDRA-15821
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15821
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-beta
>
>
> CASSANDRA-15582 involves quality around metrics and it was mentioned that 
> reviewing and [improving 
> documentation|https://github.com/apache/cassandra/blob/trunk/doc/source/operating/metrics.rst]
>  around metrics would fall into that scope. Please consider some of this 
> analysis in determining what improvements to make here:
> Please see [this 
> spreadsheet|https://docs.google.com/spreadsheets/d/1iPWfCMIG75CI6LbYuDtCTjEOvZw-5dyH-e08bc63QnI/edit?usp=sharing]
>  that itemizes almost all of cassandra's metrics and whether they are 
> documented or not (and other notes).  That spreadsheet is "almost all" 
> because there are some metrics that don't seem to initialize as part of 
> Cassandra startup (i was able to trigger some to initialize, but all were not 
> immediately obvious). The missing metrics seem to be related to the following:
> * ThreadPool metrics - only some initialize at startup the list of which 
> follow below
> * Streaming Metrics
> * HintedHandoff Metrics
> * HintsService Metrics
> Here are the ThreadPool scopes that get listed:
> {code}
> AntiEntropyStage
> CacheCleanupExecutor
> CompactionExecutor
> GossipStage
> HintsDispatcher
> MemtableFlushWriter
> MemtablePostFlush
> MemtableReclaimMemory
> MigrationStage
> MutationStage
> Native-Transport-Requests
> PendingRangeCalculator
> PerDiskMemtableFlushWriter_0
> ReadStage
> Repair-Task
> RequestResponseStage
> Sampler
> SecondaryIndexManagement
> ValidationExecutor
> ViewBuildExecutor
> {code}
> I noticed that Keyspace Metrics have this note: "Most of these metrics are 
> the same as the Table Metrics above, only they are aggregated at the Keyspace 
> level." I think I've isolated those metrics on table that are not on keyspace 
> to specifically be:
> {code}
> BloomFilterFalsePositives
> BloomFilterFalseRatio
> BytesAnticompacted
> BytesFlushed
> BytesMutatedAnticompaction
> BytesPendingRepair
> BytesRepaired
> BytesUnrepaired
> CompactionBytesWritten
> CompressionRatio
> CoordinatorReadLatency
> CoordinatorScanLatency
> CoordinatorWriteLatency
> EstimatedColumnCountHistogram
> EstimatedPartitionCount
> EstimatedPartitionSizeHistogram
> KeyCacheHitRate
> LiveSSTableCount
> MaxPartitionSize
> MeanPartitionSize
> MinPartitionSize
> MutatedAnticompactionGauge
> PercentRepaired
> RowCacheHitOutOfRange
> RowCacheHit
> RowCacheMiss
> SpeculativeSampleLatencyNanos
> SyncTime
> WaitingOnFreeMemtableSpace
> DroppedMutations
> {code}
> Someone with greater knowledge of this area might consider it worth the 
> effort to see if any of these metrics should be aggregated to the keyspace 
> level in case they were inadvertently missed. In any case, perhaps the 
> documentation could easily now reflect which metric names could be expected 
> on Keyspace.
> The DroppedMessage metrics have a much larger body of scopes than just what 
> were documented:
> {code}
> ASYMMETRIC_SYNC_REQ
> BATCH_REMOVE_REQ
> BATCH_REMOVE_RSP
> BATCH_STORE_REQ
> BATCH_STORE_RSP
> CLEANUP_MSG
> COUNTER_MUTATION_REQ
> COUNTER_MUTATION_RSP
> ECHO_REQ
> ECHO_RSP
> FAILED_SESSION_MSG
> FAILURE_RSP
> FINALIZE_COMMIT_MSG
> FINALIZE_PROMISE_MSG
> FINALIZE_PROPOSE_MSG
> GOSSIP_DIGEST_ACK
> GOSSIP_DIGEST_ACK2
> GOSSIP_DIGEST_SYN
> GOSSIP_SHUTDOWN
> HINT_REQ
> HINT_RSP
> INTERNAL_RSP
> MUTATION_REQ
> MUTATION_RSP
> PAXOS_COMMIT_REQ
> PAXOS_COMMIT_RSP
> PAXOS_PREPARE_REQ
> PAXOS_PREPARE_RSP
> PAXOS_PROPOSE_REQ
> PAXOS_PROPOSE_RSP
> PING_REQ
> PING_RSP
> PREPARE_CONSISTENT_REQ
> PREPARE_CONSISTENT_RSP
> PREPARE_MSG
> RANGE_REQ
> RANGE_RSP
> READ_REPAIR_REQ
> READ_REPAIR_RSP
> READ_REQ
> READ_RSP
> REPAIR_RSP
> REPLICATION_DONE_REQ
> REPLICATION_DONE_RSP
> REQUEST_RSP
> SCHEMA_PULL_REQ
> SCHEMA_PULL_RSP
> SCHEMA_PUSH_REQ
> SCHEMA_PUSH_RSP
> SCHEMA_VERSION_REQ
> SCHEMA_VERSION_RSP
> SNAPSHOT_MSG
> SNAPSHOT_REQ
> SNAPSHOT_RSP
> STATUS_REQ
> STATUS_RSP
> SYNC_REQ
> SYNC_RSP
> TRUNCATE_REQ
> TRUNCATE_RSP
> VALIDATION_REQ
> VALIDATION_RSP
> _SAMPLE
> _TEST_1
> _TEST_2
> _TRACE
> {code}
> I suppose I may yet be missing some metrics as my knowledge of what's 
> available is limited to what I can get from JMX after cassandra 
> initialization (and some 

[jira] [Updated] (CASSANDRA-15406) Show the progress of data streaming and index build

[Stefan Miklosovic (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-15406:
--
Test and Documentation Plan: [https://github.com/apache/cassandra/pull/711] 
 (was: 
[https://github.com/apache/cassandra/pull/558|https://github.com/apache/cassandra/pull/711])

> Show the progress of data streaming and index build 
> 
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Consistency/Streaming, Legacy/Streaming and Messaging, 
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0, 4.x
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> I found that we should supply a command to show the progress of streaming 
> when we do the operation of bootstrap/move/decommission/removenode. For when 
> do data streaming , noboday knows which steps there program are in , so I 
> think a command to show the joing/leaving node's is needed .
>  
> PR [https://github.com/apache/cassandra/pull/558]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15406) Show the progress of data streaming and index build

[Stefan Miklosovic (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Stefan Miklosovic updated CASSANDRA-15406:
--
Test and Documentation Plan: 
[https://github.com/apache/cassandra/pull/558|https://github.com/apache/cassandra/pull/711]
  (was: https://github.com/apache/cassandra/pull/558)

> Show the progress of data streaming and index build 
> 
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Consistency/Streaming, Legacy/Streaming and Messaging, 
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0, 4.x
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> I found that we should supply a command to show the progress of streaming 
> when we do the operation of bootstrap/move/decommission/removenode. For when 
> do data streaming , noboday knows which steps there program are in , so I 
> think a command to show the joing/leaving node's is needed .
>  
> PR [https://github.com/apache/cassandra/pull/558]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15406) Show the progress of data streaming and index build

[Stefan Miklosovic (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181389#comment-17181389
 ] 

Stefan Miklosovic edited comment on CASSANDRA-15406 at 8/20/20, 7:32 PM:
-

Build: [https://ci-cassandra.apache.org/job/Cassandra-devbranch/255/]

PR: [https://github.com/apache/cassandra/pull/711]

I have found a bug during this implementation. In a nutshell, I hope it helps 
you during review, the issue is that the size in 
CassandraOutgoingFile#getSize() is computed from header.size() but that is 
wrong because size from header is done in CassandraStreamHeader#calculateSize 
and that is either based on compression info or otherwise it just computes 
stuff from "sections". However there is a weird clash when this is computed 
because Netstats reports the total bytes to be sent as total over this 
"sections size" even it is sent over like compressed, it is compressed by 
default by CassandraCompressedStreamWriter and the individual "total bytes" per 
each item to be streamed is taken from its totalSize method and there it is 
computed as compressed so the numbers dont match.

What I did was that I computed CassandraOutgoingFile#getSize() in advance just 
once with help of header.calculateCompressionInfo in its constructor.

The PR is consisting of two commits, the second one tries to simplify the logic 
related to the computation of size and it reduce that logic to one central 
place.

Tests are testing bootstrapping and normal repair, compression turned on / off 
and entire sstable streaming on / off. Btw, I am not able to test as of now the 
output of the bootstrapping node. My suspicion is that the output is there, 
sure, even for the bootstrapping one, but dtest api and things around are hairy 
a bit so nothing is shown, I am getting some errors while doing nodetool 
netstats against a node which is under bootstrap.
  
 For percentage progress for building indexes, the progress is there, I have 
managed to push this through some time ago so that one is just fine 
[https://github.com/apache/cassandra/commit/38f5f9caccabee2601ca5e95884d83857d22bf33]

Once this is merged, issues described in CASSANDRA-14192 will be fixed as well. 
I found out the exact same reason as Vincent White in his first comment and 
that is what is included in this patch as well.


was (Author: stefan.miklosovic):
Build: [https://ci-cassandra.apache.org/job/Cassandra-devbranch/255/]

PR: [https://github.com/apache/cassandra/pull/711]

I have found a bug during this implementation. In a nutshell, I hope it helps 
you during review, the issue is that the size in 
CassandraOutgoingFile#getSize() is computed from header.size() but that is 
wrong because size from header is done in CassandraStreamHeader#calculateSize 
and that is either based on compression info or otherwise it just computes 
stuff from "sections". However there is a weird clash when this is computed 
because Netstats reports the total bytes to be sent as total over this 
"sections size" even it is sent over like compressed, it is compressed by 
default by CassandraCompressedStreamWriter and the individual "total bytes" per 
each item to be streamed is taken from its totalSize method and there it is 
computed as compressed so the numbers dont match.

What I did was that I computed CassandraOutgoingFile#getSize() in advance just 
once with help of header.calculateCompressionInfo in its constructor.

The PR is consisting of two commits, the second one tries to simplify the logic 
related to the computation of size and it reduce that logic to one central 
place.

Tests are testing bootstrapping and normal repair, compression turned on / off 
and entire sstable streaming on / off. Btw, I am not able to test as of now the 
output of the bootstrapping node. My suspicion is that the output is there, 
sure, even for the bootstrapping one, but dtest api and things around are hairy 
a bit so nothing is shown, I am getting some errors while doing nodetool 
netstats against a node which is under bootstrap.
 
For percentage progress for building indexes, the progress is there, I have 
managed to push this through some time ago so that one is just fine 
[https://github.com/apache/cassandra/commit/38f5f9caccabee2601ca5e95884d83857d22bf33]

> Show the progress of data streaming and index build 
> 
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Consistency/Streaming, Legacy/Streaming and Messaging, 
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0, 4.x
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> I found that we should supply a 

[jira] [Commented] (CASSANDRA-15406) Show the progress of data streaming and index build

[Stefan Miklosovic (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181389#comment-17181389
 ] 

Stefan Miklosovic commented on CASSANDRA-15406:
---

Build: [https://ci-cassandra.apache.org/job/Cassandra-devbranch/255/]

PR: [https://github.com/apache/cassandra/pull/711]

I have found a bug during this implementation. In a nutshell, I hope it helps 
you during review, the issue is that the size in 
CassandraOutgoingFile#getSize() is computed from header.size() but that is 
wrong because size from header is done in CassandraStreamHeader#calculateSize 
and that is either based on compression info or otherwise it just computes 
stuff from "sections". However there is a weird clash when this is computed 
because Netstats reports the total bytes to be sent as total over this 
"sections size" even it is sent over like compressed, it is compressed by 
default by CassandraCompressedStreamWriter and the individual "total bytes" per 
each item to be streamed is taken from its totalSize method and there it is 
computed as compressed so the numbers dont match.

What I did was that I computed CassandraOutgoingFile#getSize() in advance just 
once with help of header.calculateCompressionInfo in its constructor.

The PR is consisting of two commits, the second one tries to simplify the logic 
related to the computation of size and it reduce that logic to one central 
place.

Tests are testing bootstrapping and normal repair, compression turned on / off 
and entire sstable streaming on / off. Btw, I am not able to test as of now the 
output of the bootstrapping node. My suspicion is that the output is there, 
sure, even for the bootstrapping one, but dtest api and things around are hairy 
a bit so nothing is shown, I am getting some errors while doing nodetool 
netstats against a node which is under bootstrap.
 
For percentage progress for building indexes, the progress is there, I have 
managed to push this through some time ago so that one is just fine 
[https://github.com/apache/cassandra/commit/38f5f9caccabee2601ca5e95884d83857d22bf33]

> Show the progress of data streaming and index build 
> 
>
> Key: CASSANDRA-15406
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15406
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Consistency/Streaming, Legacy/Streaming and Messaging, 
> Tool/nodetool
>Reporter: maxwellguo
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0, 4.x
>
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> I found that we should supply a command to show the progress of streaming 
> when we do the operation of bootstrap/move/decommission/removenode. For when 
> do data streaming , noboday knows which steps there program are in , so I 
> think a command to show the joing/leaving node's is needed .
>  
> PR [https://github.com/apache/cassandra/pull/558]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16053) cqlsh COPY functions broken in Python 3.8 on Mac

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181388#comment-17181388
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16053:
-

I am not a driver person but from everything I read in the referenced links 
around python and drivers, the patch looks good to me. 
No new CI failures, I read you iterated the reproduction test quite some time 
and it doesn't fail anymore.
+1 from me, you need a committer to check it too and possibly commit.

> cqlsh COPY functions broken in Python 3.8 on Mac
> 
>
> Key: CASSANDRA-16053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta2
>
>
> {noformat}
> aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
> 3.8.2
> aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
> Using CQL driver:  '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
> Using connect timeout: 5 seconds
> Using 'utf-8' encoding
> Using ssl: False
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> copy test.testcopyto to 'asdf';
> Detected 12 core(s)
> Using 11 child processes
> Starting copy of test.testcopyto with columns [a, b, c, d].
> Traceback (most recent call last):
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
> self.handle_statement(st, statementtext)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in 
> handle_statement
> return custom_handler(parsed)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in 
> do_copy
> task.run()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 669, in run
> self.start_processes()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 471, in start_processes
> process.start()
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py",
>  line 121, in start
> self._popen = self._Popen(self)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 224, in _Popen
> return _default_context.get_context().Process._Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 283, in _Popen
> return Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 32, in __init__
> super().__init__(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py",
>  line 19, in __init__
> self._launch(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 47, in _launch
> reduction.dump(process_obj, fp)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py",
>  line 60, in dump
> ForkingPickler(file, protocol).dump(obj)
> TypeError: cannot pickle '_thread.lock' object
> cqlsh>
> {noformat}
> multiprocessing uses a different default start method on Mac, and pickling 
> fails trying to serialize the Cluster object.
> https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
> https://bugs.python.org/issue33725



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15802) Commented-out lines that end in a semicolon cause an error.

[Rens Groothuijsen (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181387#comment-17181387
 ] 

Rens Groothuijsen commented on CASSANDRA-15802:
---

[~mck] Nice! Thanks for your help in getting me there as well.

> Commented-out lines that end in a semicolon cause an error.
> ---
>
> Key: CASSANDRA-15802
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15802
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Interpreter, CQL/Syntax
>Reporter: null 
>Assignee: Rens Groothuijsen
>Priority: Normal
> Fix For: 4.0-beta2
>
> Attachments: cqlsh.png
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Commented-out lines that end in a semicolon cause an error.
> For example:
> /*
> describe keyspaces;
> */
>  
> This produces an error:
> SyntaxException: line 2:22 no viable alternative at input ' (...*
> describe keyspaces;...)
>  
> It works as expected if you use syntax:
> -- describe keyspaces;
>  
> Environment:
> python:3.7.7-slim-stretch (docker image)
>  
> I found that this was first seen here, and was patched, but the bug appears 
> to have resurfaced:
> https://issues.apache.org/jira/browse/CASSANDRA-2488



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-16000:
---
Fix Version/s: (was: 4.0-beta)
   4.0-beta2

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta2
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181359#comment-17181359
 ] 

Michael Semb Wever commented on CASSANDRA-16000:


[~e.dimitrova], in cassandra-website there is a macro for the latest version, 
so changes like this don't have to be made:
 https://github.com/apache/cassandra-website/blob/master/src/_data/releases.yaml

Something similar does exist for the in-tree docs (though I've never used it):
 https://github.com/apache/cassandra/blob/trunk/doc/source/conf.py#L81-L84

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181347#comment-17181347
 ] 

Ekaterina Dimitrova edited comment on CASSANDRA-16000 at 8/20/20, 5:19 PM:
---

Thanks [~brandon.williams]. Actually I wanted to confirm is this gonna be 
published with the new view of the web page before beta2 or not?
As if it is gonna be roll out with beta2, it needs to be changed to 4.0-beta2. 
[~lor...@datastax.com], can you advise, please?
Thinking out loud, should we make this update part of the release process so we 
update this page on release? [~mck2], does this make sense?


was (Author: e.dimitrova):
Hey, actually I wanted to confirm is this gonna be published with the new 
view of the web page before beta2 or not?
As if it is gonna be roll out with beta2, it needs to be changed to 4.0-beta2. 
[~lor...@datastax.com], can you advise, please?
Thinking out loud, should we make this update part of the release process so we 
update this page on release? [~mck2], does this make sense?

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181347#comment-17181347
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16000:
-

Hey, actually I wanted to confirm is this gonna be published with the new 
view of the web page before beta2 or not?
As if it is gonna be roll out with beta2, it needs to be changed to 4.0-beta2. 
[~lor...@datastax.com], can you advise, please?
Thinking out loud, should we make this update part of the release process so we 
update this page on release? [~mck2], does this make sense?

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16053) cqlsh COPY functions broken in Python 3.8 on Mac

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16053:

Reviewers: Ekaterina Dimitrova, Ekaterina Dimitrova  (was: Ekaterina 
Dimitrova)
   Ekaterina Dimitrova, Ekaterina Dimitrova
   Status: Review In Progress  (was: Patch Available)

> cqlsh COPY functions broken in Python 3.8 on Mac
> 
>
> Key: CASSANDRA-16053
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16053
> Project: Cassandra
>  Issue Type: Bug
>  Components: Tool/cqlsh
>Reporter: Adam Holmberg
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta2
>
>
> {noformat}
> aholmberg-rmbp16:cassandra adamholmberg$ pyenv shell
> 3.8.2
> aholmberg-rmbp16:cassandra adamholmberg$ bin/cqlsh --debug
> Using CQL driver:  '/Users/adamholmberg/code/cassandra/bin/../lib/cassandra-driver-internal-only-3.23.0.post0-1a184b99.zip/cassandra-driver-3.23.0.post0-1a184b99/cassandra/__init__.py'>
> Using connect timeout: 5 seconds
> Using 'utf-8' encoding
> Using ssl: False
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 4.0-beta2-SNAPSHOT | CQL spec 3.4.5 | Native 
> protocol v4]
> Use HELP for help.
> cqlsh> copy test.testcopyto to 'asdf';
> Detected 12 core(s)
> Using 11 child processes
> Starting copy of test.testcopyto with columns [a, b, c, d].
> Traceback (most recent call last):
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 937, in onecmd
> self.handle_statement(st, statementtext)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 974, in 
> handle_statement
> return custom_handler(parsed)
>   File "/Users/adamholmberg/code/cassandra/bin/cqlsh.py", line 1563, in 
> do_copy
> task.run()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 669, in run
> self.start_processes()
>   File 
> "/Users/adamholmberg/code/cassandra/bin/../pylib/cqlshlib/copyutil.py", line 
> 471, in start_processes
> process.start()
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/process.py",
>  line 121, in start
> self._popen = self._Popen(self)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 224, in _Popen
> return _default_context.get_context().Process._Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/context.py",
>  line 283, in _Popen
> return Popen(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 32, in __init__
> super().__init__(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_fork.py",
>  line 19, in __init__
> self._launch(process_obj)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/popen_spawn_posix.py",
>  line 47, in _launch
> reduction.dump(process_obj, fp)
>   File 
> "/Users/adamholmberg/.pyenv/versions/3.8.2/lib/python3.8/multiprocessing/reduction.py",
>  line 60, in dump
> ForkingPickler(file, protocol).dump(obj)
> TypeError: cannot pickle '_thread.lock' object
> cqlsh>
> {noformat}
> multiprocessing uses a different default start method on Mac, and pickling 
> fails trying to serialize the Cluster object.
> https://github.com/python/cpython/blob/db098bc1f05bd0773943e59f83489f05f28dedf8/Lib/multiprocessing/context.py#L313-L318
> https://bugs.python.org/issue33725



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15986) Repair tests flakiness

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15986:

Description: 
Repair tests come up in test failure reports every now and then. I have tried 
to repro the 
[latest|https://ci-cassandra.apache.org/job/Cassandra-trunk/241/testReport/junit/dtest-novnode.repair_tests.repair_test/TestRepair/test_simple_sequential_repair/]
 locally 100 times with no luck.
_dtest-novnode.repair_tests.repair_test/TestRepair/test_simple_sequential_repair_

Still from experience from fixing other flaky tests I have some intuition where 
the problems may lie. The proposed fix should add no harm if merged. We can 
reopen the ticket if repair tests keep failing.

  was:
Repair tests come up in test failure reports every now and then. I have tried 
to repro the 
[latest|https://ci-cassandra.apache.org/job/Cassandra-trunk/241/testReport/junit/dtest-novnode.repair_tests.repair_test/TestRepair/test_simple_sequential_repair/]
 locally 100 times with no luck.

Still from experience from fixing other flaky tests I have some intuition where 
the problems may lie. The proposed fix should add no harm if merged. We can 
reopen the ticket if repair tests keep failing.


> Repair tests flakiness
> --
>
> Key: CASSANDRA-15986
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15986
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Repair tests come up in test failure reports every now and then. I have tried 
> to repro the 
> [latest|https://ci-cassandra.apache.org/job/Cassandra-trunk/241/testReport/junit/dtest-novnode.repair_tests.repair_test/TestRepair/test_simple_sequential_repair/]
>  locally 100 times with no luck.
> _dtest-novnode.repair_tests.repair_test/TestRepair/test_simple_sequential_repair_
> Still from experience from fixing other flaky tests I have some intuition 
> where the problems may lie. The proposed fix should add no harm if merged. We 
> can reopen the ticket if repair tests keep failing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16048) Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and Value Columns

[Sylvain Lebresne (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181308#comment-17181308
 ] 

Sylvain Lebresne commented on CASSANDRA-16048:
--

bq. Regarding the DENSE flag, this is the same as if "DROP COMPACT STORAGE" was 
run, and the behavior prior to this patch is that the database would fail to 
start, so I'm not sure there will be much surprise to the user or that its of 
much concern.

This patch silently drop compact storage on (some) user tables. And as 
mentionned, this _has_ user visible consequences (no more "WITH COMPACT 
STORAGE" mention when using {{DESCRIBED}} and removed {{DENSE}} flag), even if 
they can arguably be considered minor.

The user surprise I'm talking about is a user noticing one of those 
consequences some time post 4.0 upgrade, and having no clue why this happened 
(since again, the drop is automatic and basically silent\[1\]). I mean, in 
general, silent changes tends to be surprising.

Without this patch, users are obligated to _manually_ run {{DROP COMPACT 
STORAGE}} on all tables, so no risk of surprise exists. Which is kind of a 
feature in that case imo.

I suppose the opinion I'm pushing here is that it might be better to ask a few 
power users to run more manual {{DROP COMPACT STORAGE}} (it's not like it can't 
be scripted externally easily if you know what you're doing), than risking to 
confuse some users, even slightly, around something already kind of confusing 
like compact tables. Obviously, I don't know how many and how much users might 
get confused by this. And probably not a lot. Is the risk worth the upside for 
users in general? I lean "no" as of now, but definitively a very weakly held 
position.

bq. As an aside, I am also concerned about the current behavior that we halt 
the database from starting when we detect compact storage tables

This may or may not be what you meant here, but fwiw, I created CASSANDRA-16063 
which is relevant (long story short, I think the general behavior is sensible 
but I don't think it's implemented right; if you're concerned by the general 
behavior and have alternative to propose, feel free to hijack that ticket for 
that purpose).


\[1\]: on that "silent" subject: I think that if we do end up doing this, we 
should at least clearly log when we automatically migrating a table, which the 
current patch don't do, so it's not completely silent. Tbc, doing so wouldn't 
entirely assuage my concerns because I don't think everyone reads their log 
carefully.


> Safely Ignore Compact Storage Tables Where Users Have Defined Clustering and 
> Value Columns
> --
>
> Key: CASSANDRA-16048
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16048
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Legacy/CQL
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Some compact storage tables, specifically those where the user has defined 
> both at least one clustering and the value column, can be safely handled in 
> 4.0 because besides the DENSE flag they are not materially different post 3.0 
> and there is no visible change to the user facing schema after dropping 
> compact storage. We can detect this case and allow these tables to silently 
> drop the DENSE flag while still throwing a start-up error for COMPACT STORAGE 
> tables that don’t meet the criteria. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16062) Avoid NPE in getCompactionInfo

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16062:
-
Reviewers: Brandon Williams, Brandon Williams  (was: Brandon Williams)
   Brandon Williams, Brandon Williams
   Status: Review In Progress  (was: Patch Available)

> Avoid NPE in getCompactionInfo
> --
>
> Key: CASSANDRA-16062
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16062
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We currently initialize {{sstables}} after calling {{beginCompaction(this)}} 
> in the {{CompactionIterator}} constructor which creates a window where we can 
> get NPE creating the {{CompactionInfo}} for cancelling compactions



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16062) Avoid NPE in getCompactionInfo

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16062:
-
Status: Ready to Commit  (was: Review In Progress)

+1

> Avoid NPE in getCompactionInfo
> --
>
> Key: CASSANDRA-16062
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16062
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We currently initialize {{sstables}} after calling {{beginCompaction(this)}} 
> in the {{CompactionIterator}} constructor which creates a window where we can 
> get NPE creating the {{CompactionInfo}} for cancelling compactions



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15909) Make Table/Keyspace Metric Names Consistent With Each Other

[Caleb Rackliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Caleb Rackliffe updated CASSANDRA-15909:

Reviewers: Caleb Rackliffe, Caleb Rackliffe  (was: Caleb Rackliffe)
   Caleb Rackliffe, Caleb Rackliffe
   Status: Review In Progress  (was: Patch Available)

> Make Table/Keyspace Metric Names Consistent With Each Other
> ---
>
> Key: CASSANDRA-15909
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15909
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Observability/Metrics
>Reporter: Stephen Mallette
>Assignee: Stephen Mallette
>Priority: Normal
> Fix For: 4.0-beta
>
>
> As part of CASSANDRA-15821 it became apparent that certain metric names found 
> in keyspace and tables had different names but were in fact the same metric - 
> they are as follows:
> * Table.SyncTime == Keyspace.RepairSyncTime
> * Table.RepairedDataTrackingOverreadRows == Keyspace.RepairedOverreadRows
> * Table.RepairedDataTrackingOverreadTime == Keyspace.RepairedOverreadTime
> * Table.AllMemtablesHeapSize == Keyspace.AllMemtablesOnHeapDataSize
> * Table.AllMemtablesOffHeapSize == Keyspace.AllMemtablesOffHeapDataSize
> * Table.MemtableOnHeapSize == Keyspace.MemtableOnHeapDataSize
> * Table.MemtableOffHeapSize == Keyspace.MemtableOffHeapDataSize
> Also, client metrics are the only metrics to start with a lower case letter. 
> Change those to upper case to match all the other metrics.
> Unifying this naming would help make metrics more consistent as part of 
> CASSANDRA-15582



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16000:
-
Status: Ready to Commit  (was: Review In Progress)

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16000:
-
  Since Version: 4.0-beta1
Source Control Link: 
https://github.com/apache/cassandra/commit/102a54a6a11aea4078e6f7e694c8798b48ff352c
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Thanks, committed.

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Brandon Williams (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brandon Williams updated CASSANDRA-16000:
-
Reviewers: Brandon Williams, Brandon Williams  (was: Brandon Williams)
   Brandon Williams, Brandon Williams
   Status: Review In Progress  (was: Patch Available)

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Update 4.0.0 to 4.0-beta1 at Documentation/Installing

[brandonwilliams]

This is an automated email from the ASF dual-hosted git repository.

brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 102a54a  Update 4.0.0 to 4.0-beta1 at Documentation/Installing
102a54a is described below

commit 102a54a6a11aea4078e6f7e694c8798b48ff352c
Author: Ekaterina Dimitrova 
AuthorDate: Thu Aug 20 11:36:02 2020 -0400

Update 4.0.0 to 4.0-beta1 at Documentation/Installing

Patch by Ekaterina Dimitrova, reviewed by brandonwilliams for 
CASSANDRA-16000
---
 doc/source/getting_started/installing.rst | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/doc/source/getting_started/installing.rst 
b/doc/source/getting_started/installing.rst
index 1d59b8b..5c5239d 100644
--- a/doc/source/getting_started/installing.rst
+++ b/doc/source/getting_started/installing.rst
@@ -77,7 +77,7 @@ Installing the binary tarball
 
 ::
 
-   $ curl -OL 
http://apache.mirror.digitalpacific.com.au/cassandra/4.0.0/apache-cassandra-4.0.0-bin.tar.gz
+   $ curl -OL 
http://apache.mirror.digitalpacific.com.au/cassandra/4.0-beta1/apache-cassandra-4.0-beta1-bin.tar.gz
 
 NOTE: The mirrors only host the latest versions of each major supported 
release. To download an earlier
 version of Cassandra, visit the `Apache Archives 
`__.
@@ -87,24 +87,24 @@ version of Cassandra, visit the `Apache Archives 

[jira] [Assigned] (CASSANDRA-15986) Repair tests flakiness

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova reassigned CASSANDRA-15986:
---

Assignee: Ekaterina Dimitrova

> Repair tests flakiness
> --
>
> Key: CASSANDRA-15986
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15986
> Project: Cassandra
>  Issue Type: Task
>  Components: Test/dtest
>Reporter: Berenguer Blasi
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Repair tests come up in test failure reports every now and then. I have tried 
> to repro the 
> [latest|https://ci-cassandra.apache.org/job/Cassandra-trunk/241/testReport/junit/dtest-novnode.repair_tests.repair_test/TestRepair/test_simple_sequential_repair/]
>  locally 100 times with no luck.
> Still from experience from fixing other flaky tests I have some intuition 
> where the problems may lie. The proposed fix should add no harm if merged. We 
> can reopen the ticket if repair tests keep failing.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Ekaterina Dimitrova (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181291#comment-17181291
 ] 

Ekaterina Dimitrova commented on CASSANDRA-16000:
-

[Pull request 
|https://github.com/ekaterinadimitrova2/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-16000?expand=1]

[~mck2], [~dcapwell] , [~brandon.williams], anyone has a free cycle to check 
this quick one?

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-16000:

Test and Documentation Plan: 
https://issues.apache.org/jira/browse/CASSANDRA-16000?focusedCommentId=17181291=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17181291
 Status: Patch Available  (was: In Progress)

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova reassigned CASSANDRA-16000:
---

Assignee: Ekaterina Dimitrova

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15971) full query log needs improvement

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova updated CASSANDRA-15971:

Status: Changes Suggested  (was: Review In Progress)

> full query log needs improvement
> 
>
> Key: CASSANDRA-15971
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15971
> Project: Cassandra
>  Issue Type: Improvement
>  Components: Documentation/Website, Tool/fql
>Reporter: Jonathan Shook
>Assignee: Lorina Poland
>Priority: Normal
>  Labels: pull-request-available
> Attachments: st1.txt
>
>
> When trying out full query logging as a possible integration for nosqlbench 
> usage, I ran across many issues which would make it painful for users. Since 
> there were several, they will be added to a single issue for now. This issue 
> can be broken up if needed.
> 
> FQL doesn't work on my system, even though it says it is logging queries. 
> With the following configuration in cassandra.yaml:
>  
> {noformat}
> full_query_logging_options:
>     log_dir: /REDACTED/fullquerylogs
>     roll_cycle: HOURLY
>     block: true
>     max_queue_weight: 268435456 # 256 MiB
>     max_log_size: 17179869184 # 16 GiB
>     ## archive command is "/path/to/script.sh %path" where %path is replaced 
> with the file being rolled:
>     # archive_command:
>     # max_archive_retries: 10
> {noformat}
> which appears to be the minimal configuration needed to enable fql, only a 
> single file `directory-listing.cq4t` is created, which is a 64K sized file of 
> zeroes.
>  
> 
> Calling bin/nodetool enablefullquerylog throws an error initially.
> [jshook@cass4 bin]$ ./nodetool enablefullquerylog
>  
> {noformat}
> error: sun.nio.ch.FileChannelImpl.map0(int,long,long) 
> -- StackTrace -- 
> java.lang.NoSuchMethodException: 
> sun.nio.ch.FileChannelImpl.map0(int,long,long) 
>     at java.base/java.lang.Class.getDeclaredMethod(Class.java:2553) 
>     at net.openhft.chronicle.core.OS.lambda$static$0(OS.java:51) 
>     at 
> net.openhft.chronicle.core.ClassLocal.computeValue(ClassLocal.java:53){noformat}
> (full stack trace attached to this ticket)
>  
> Subsequent calls produce normal output:
>  
> {noformat}
> [jshook@cass4 c4b1]$ bin/nodetool enablefullquerylog 
> nodetool: Already logging to /home/jshook/c4b1/data/fullquerylogs 
> See 'nodetool help' or 'nodetool help '.{noformat}
>  
> 
> nodetool missing getfullquerylog makes it difficult to verify current 
> fullquerylog state without changing it. The conventions for nodetool commands 
> should be followed to avoid confusing users.
> 
> (maybe)
> {noformat}
> tools/bin/fqltool help{noformat}
> should print out help for all fqltool commands rather than simply repeating 
> the default The most commonly used fqltool commands are..
> 
> [https://cassandra.apache.org/doc/latest/new/fqllogging.html] is 
> malformatted, mixing the appearance of configuration with comments, which is 
> confusing at best.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova reassigned CASSANDRA-16000:
---

Assignee: (was: Ekaterina Dimitrova)

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Assigned] (CASSANDRA-16000) Updates needed after 4.0-beta1 release

[Ekaterina Dimitrova (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ekaterina Dimitrova reassigned CASSANDRA-16000:
---

Assignee: Ekaterina Dimitrova

> Updates needed after 4.0-beta1 release
> --
>
> Key: CASSANDRA-16000
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16000
> Project: Cassandra
>  Issue Type: Bug
>  Components: Documentation/Website
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0, 4.0-beta
>
>
> We need to update for beta the Documentation/Installing Cassandra page (also 
> the curl link provided)
>  
> /CC [~lor...@datastax.com]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16063) Fix user experience when upgrading to 4.0 with compact tables

[Sylvain Lebresne (Jira)]

Sylvain Lebresne created CASSANDRA-16063:


 Summary: Fix user experience when upgrading to 4.0 with compact 
tables
 Key: CASSANDRA-16063
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16063
 Project: Cassandra
  Issue Type: Bug
  Components: Legacy/CQL
Reporter: Sylvain Lebresne


The code to handle compact tables has been removed from 4.0, and the intended 
upgrade path to 4.0 for users having compact tables on 3.x is that they must 
execute {{ALTER ... DROP COMPACT STORAGE}} on all of their compact tables 
*before* attempting the upgrade.

Obviously, some users won't read the upgrade instructions (or miss a table) and 
may try upgrading despite still having compact tables. If they do so, the 
intent is that the node will _not_ start, with a message clearly indicating the 
pre-upgrade step the user has missed. The user will then downgrade back the 
node(s) to 3.x, run the proper {{ALTER ... DROP COMPACT STORAGE}}, and then 
upgrade again.

But while 4.0 does currently fail startup when finding any compact tables with 
a decent message, I believe the check is done too late during startup.

Namely, that check is done as we read the tables schema, so within 
[{{Schema.instance.loadFromDisk()}}|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/service/CassandraDaemon.java#L241].
  But by then, we've _at least_ called 
{{SystemKeyspace.persistLocalMetadata()}}} and 
{{SystemKeyspaceMigrator40.migrate()}}, which will get into the commit log, and 
even possibly flush new {{na}} format sstables. As a results, a user might not 
be able to seemlessly restart the node on 3.x (to drop compact storage on the 
appropriate tables).

Basically, we should make sure the check for compact tables done at 4.0 startup 
is done as a {{StartupCheck}}, before the node does anything.

We should also add a test for this (checking that if you try upgrading to 4.0 
with compact storage, you can downgrade back with no intervention whatsoever).




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Commented] (CASSANDRA-15962) Digest for some queries is different depending whether the data are retrieved from sstable or memtable

[Sylvain Lebresne (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181221#comment-17181221
 ] 

Sylvain Lebresne commented on CASSANDRA-15962:
--

The approach lgtm. Just a few remarks:
* Could you include the {{DigestTest}} you wrote into the PR?
* Changes to {{SelectStatement}} (special casing of the filter for super 
columns on range queries, as is done for slice ones) looks legit, but unrelated 
and a problem on 3.0. We should either split to another ticket (probably 
ideal), or have a 3.0 branch with that fix. It would also be good to have a 
test (one that shows the problem that exists and is fixed by this change).
* Similarly, the change to {{AbstractReadExecutor}} also look legit but 
unrelated and exists in 3.0. That one is pretty minor since I don't think it 
has visible consequence, so not worth a ticket of its own, but would nice to 
include with whatever we do for my previous point so it gets into 3.0.
* BTreeRow#filter: we should reuse/extract the 
{{!queriedByUserTester.test(column)}} test from the {{isSkippable}} initializer 
as value for {{shouldSkipValue}} instead of redoing the work by calling 
{{!filter.fetchedColumnIsQueried(column)}}.
* ComplexColumnData#filter:
** I'd store the result of {{filter.fetchedColumnIsQueried(column))}} in a 
variable at the beginning of the function, to avoid potentially repeating the 
call multiple times. Mostly because it'll be imo more readable, but it also 
avoid bad cases where we repeat it for vary many cells.
** Nit: The {{path != null}} in the {{shouldSkipValue}} initializer is 
unecessary: cells of complex columns are guaranteed to have a non-null path 
(see assertion in {{BufferCell}} ctor).
** It would be nice to avoid the repetition of 
{{cellTest.fetchedCellIsQueried(path)}} as well, readability wise. I'd suggest 
something like:
   {noformat}
   CellPath path = cell.path();
   boolean isForDropped = ...;
   boolean isShardowed = ...;
   boolean isFetchedCell = cellTester == null || cellTest.fetches(path);
   boolean isQueriedCell = isQueriedColumn && isFetchedCell && (cellTester == 
null || cellTester.fetchedCellIsQueried(path));
   boolean isSkippableCell = !isFetchedCell || (!isQueriedCell && 
cell.timestamp() < rowLiveness.timestamp());
   if (isForDropped ||| isShadowed || isSkippable)
   return null;

   // If the cell is only fetched but not queried, we need the cell but never 
the value. So, when reading from sstables, we
   // "skip" the value of such cells as an optimiation (see Cell#deserialize). 
We _must_ thus do the same here to avoid
   // disrepancies between data coming from memtables or sstables, which would 
lead to digest mismatches.
   return isQueriedCell ? cell : cell.withSkippedValue();
   {noformat}


With those addressed, if you could set up both a 3.11 branch and a trunk one 
and run CI, this would be ideal.


> Digest for some queries is different depending whether the data are retrieved 
> from sstable or memtable
> --
>
> Key: CASSANDRA-15962
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15962
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Coordination
>Reporter: Jacek Lewandowski
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 4.0, 3.11.x
>
> Attachments: DigestTest.java
>
>
> Not sure into which category should I assign this ticket.
>  
> Basically when reading using certain column filters, the digest is different 
> depending whether we read from sstable and memtable. This happens on 
> {{trunk}} and {{cassandra-3.11}} branches. However it works properly on 
> {{cassandra-3.0}} branch.
>  
> I'm attaching a simple test for trunk to demonstrate what I mean. 
>  
> Please verify my test and my conclusions
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15816) Transports are stopped in the wrong order

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-15816:

Reviewers: Marcus Eriksson, Robert Stupp  (was: Robert Stupp)

> Transports are stopped in the wrong order
> -
>
> Key: CASSANDRA-15816
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15816
> Project: Cassandra
>  Issue Type: Bug
>  Components: Messaging/Internode
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Stopping gossip while native is running is almost always wrong, change the 
> order of shutdown and log a warning when done manually



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15817) Prevent repair from overrunning compaction

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-15817:

Reviewers: David Capwell, Marcus Eriksson  (was: David Capwell)

> Prevent repair from overrunning compaction
> --
>
> Key: CASSANDRA-15817
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15817
> Project: Cassandra
>  Issue Type: Bug
>  Components: Consistency/Repair
>Reporter: Jeff Jirsa
>Assignee: Jeff Jirsa
>Priority: Low
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Repair can run and stream faster than a host can compact. At some point, if a 
> host is sufficiently out of sync, or compaction is especially expensive, it 
> makes sense to intentionally block repair so that compaction can catch up



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15802) Commented-out lines that end in a semicolon cause an error.

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15802:
---
  Fix Version/s: (was: 4.0-beta)
 4.0-beta2
  Since Version: 0.8 beta 1
Source Control Link: 
https://github.com/apache/cassandra/commit/1b1b87cfe3a9a93c393d1f3c1e003394260edeb5
 Resolution: Fixed
 Status: Resolved  (was: Ready to Commit)

Committed as 
[1b1b87cfe3a9a93c393d1f3c1e003394260edeb5|https://github.com/apache/cassandra/commit/1b1b87cfe3a9a93c393d1f3c1e003394260edeb5]

> Commented-out lines that end in a semicolon cause an error.
> ---
>
> Key: CASSANDRA-15802
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15802
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Interpreter, CQL/Syntax
>Reporter: null 
>Assignee: Rens Groothuijsen
>Priority: Normal
> Fix For: 4.0-beta2
>
> Attachments: cqlsh.png
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Commented-out lines that end in a semicolon cause an error.
> For example:
> /*
> describe keyspaces;
> */
>  
> This produces an error:
> SyntaxException: line 2:22 no viable alternative at input ' (...*
> describe keyspaces;...)
>  
> It works as expected if you use syntax:
> -- describe keyspaces;
>  
> Environment:
> python:3.7.7-slim-stretch (docker image)
>  
> I found that this was first seen here, and was patched, but the bug appears 
> to have resurfaced:
> https://issues.apache.org/jira/browse/CASSANDRA-2488



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Comment Edited] (CASSANDRA-15802) Commented-out lines that end in a semicolon cause an error.

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181141#comment-17181141
 ] 

Michael Semb Wever edited comment on CASSANDRA-15802 at 8/20/20, 12:03 PM:
---

Committed as 
[1b1b87cfe3a9a93c393d1f3c1e003394260edeb5|https://github.com/apache/cassandra/commit/1b1b87cfe3a9a93c393d1f3c1e003394260edeb5].

Thanks for taking a few rounds on the patch [~RensGroothuijsen]!


was (Author: michaelsembwever):
Committed as 
[1b1b87cfe3a9a93c393d1f3c1e003394260edeb5|https://github.com/apache/cassandra/commit/1b1b87cfe3a9a93c393d1f3c1e003394260edeb5]

> Commented-out lines that end in a semicolon cause an error.
> ---
>
> Key: CASSANDRA-15802
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15802
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Interpreter, CQL/Syntax
>Reporter: null 
>Assignee: Rens Groothuijsen
>Priority: Normal
> Fix For: 4.0-beta2
>
> Attachments: cqlsh.png
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Commented-out lines that end in a semicolon cause an error.
> For example:
> /*
> describe keyspaces;
> */
>  
> This produces an error:
> SyntaxException: line 2:22 no viable alternative at input ' (...*
> describe keyspaces;...)
>  
> It works as expected if you use syntax:
> -- describe keyspaces;
>  
> Environment:
> python:3.7.7-slim-stretch (docker image)
>  
> I found that this was first seen here, and was patched, but the bug appears 
> to have resurfaced:
> https://issues.apache.org/jira/browse/CASSANDRA-2488



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra] branch trunk updated: Strip comment blocks from cqlsh input before processing statements

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git


The following commit(s) were added to refs/heads/trunk by this push:
 new 1b1b87c  Strip comment blocks from cqlsh input before processing 
statements
1b1b87c is described below

commit 1b1b87cfe3a9a93c393d1f3c1e003394260edeb5
Author: Rens Groothuijsen 
AuthorDate: Wed May 20 20:31:14 2020 +0200

Strip comment blocks from cqlsh input before processing statements

 patch by Rens Groothuijsen; reviewed by Mick Semb Wever for CASSANDRA-15802
---
 CHANGES.txt |  1 +
 bin/cqlsh.py| 19 ++
 pylib/cqlshlib/test/test_cql_parsing.py | 65 +
 3 files changed, 85 insertions(+)

diff --git a/CHANGES.txt b/CHANGES.txt
index 3e23053..3272226 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
 4.0-beta2
+ * Strip comment blocks from cqlsh input before processing statements 
(CASSANDRA-15802)
  * Fix unicode chars error input (CASSANDRA-15990)
  * Improved testability for CacheMetrics and ChunkCacheMetrics 
(CASSANDRA-15788)
  * Handle errors in StreamSession#prepare (CASSANDRA-15852)
diff --git a/bin/cqlsh.py b/bin/cqlsh.py
index 1ef0c91..3f60094 100644
--- a/bin/cqlsh.py
+++ b/bin/cqlsh.py
@@ -39,6 +39,7 @@ import getpass
 import optparse
 import os
 import platform
+import re
 import sys
 import traceback
 import warnings
@@ -910,12 +911,30 @@ class Shell(cmd.Cmd):
 self.reset_statement()
 print('')
 
+def strip_comment_blocks(self, statementtext):
+comment_block_in_literal_string = 
re.search('["].*[/][*].*[*][/].*["]', statementtext)
+if not comment_block_in_literal_string:
+result = re.sub('[/][*].*[*][/]', "", statementtext)
+if '*/' in result and '/*' not in result and not self.in_comment:
+raise SyntaxError("Encountered comment block terminator 
without being in comment block")
+if '/*' in result:
+result = re.sub('[/][*].*', "", result)
+self.in_comment = True
+if '*/' in result:
+result = re.sub('.*[*][/]', "", result)
+self.in_comment = False
+if self.in_comment and not re.findall('[/][*]|[*][/]', 
statementtext):
+result = ''
+return result
+return statementtext
+
 def onecmd(self, statementtext):
 """
 Returns true if the statement is complete and was handled (meaning it
 can be reset).
 """
 statementtext = ensure_text(statementtext)
+statementtext = self.strip_comment_blocks(statementtext)
 try:
 statements, endtoken_escaped = 
cqlruleset.cql_split_statements(statementtext)
 except pylexotron.LexingError as e:
diff --git a/pylib/cqlshlib/test/test_cql_parsing.py 
b/pylib/cqlshlib/test/test_cql_parsing.py
index 10be99f..8631d7a 100644
--- a/pylib/cqlshlib/test/test_cql_parsing.py
+++ b/pylib/cqlshlib/test/test_cql_parsing.py
@@ -711,6 +711,71 @@ class TestCqlParsing(TestCase):
 def test_parse_select_token(self):
 pass
 
+def test_strip_comment_blocks_from_input(self):
+
+parsed = parse_cqlsh_statements('SELECT FROM /* comment block */ 
"MyTable";')
+self.assertSequenceEqual(tokens_with_types(parsed),
+ [('SELECT', 'reserved_identifier'),
+  ('FROM', 'reserved_identifier'),
+  ('"MyTable"', 'quotedName'),
+  (';', 'endtoken')])
+
+parsed = parse_cqlsh_statements('SELECT FROM /* \n comment block 
starts here; \n and continues here \n */ "MyTable";')
+self.assertSequenceEqual(tokens_with_types(parsed),
+ [('SELECT', 'reserved_identifier'),
+  ('FROM', 'reserved_identifier'),
+  ('"MyTable"', 'quotedName'),
+  (';', 'endtoken')])
+
+parsed = parse_cqlsh_statements('''
+SELECT FROM /*
+ comment block starts here;
+ and continues here
+ */ "MyTable";
+''')
+self.assertSequenceEqual(tokens_with_types(parsed),
+ [('SELECT', 'reserved_identifier'),
+  ('FROM', 'reserved_identifier'),
+  ('"MyTable"', 'quotedName'),
+  (';', 'endtoken')])
+
+parsed = parse_cqlsh_statements('''
+/* comment block */
+

[jira] [Commented] (CASSANDRA-15802) Commented-out lines that end in a semicolon cause an error.

[Michael Semb Wever (Jira)]

[ 
https://issues.apache.org/jira/browse/CASSANDRA-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17181125#comment-17181125
 ] 

Michael Semb Wever commented on CASSANDRA-15802:


Latest CI run 
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/258/pipeline/].

> Commented-out lines that end in a semicolon cause an error.
> ---
>
> Key: CASSANDRA-15802
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15802
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Interpreter, CQL/Syntax
>Reporter: null 
>Assignee: Rens Groothuijsen
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: cqlsh.png
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Commented-out lines that end in a semicolon cause an error.
> For example:
> /*
> describe keyspaces;
> */
>  
> This produces an error:
> SyntaxException: line 2:22 no viable alternative at input ' (...*
> describe keyspaces;...)
>  
> It works as expected if you use syntax:
> -- describe keyspaces;
>  
> Environment:
> python:3.7.7-slim-stretch (docker image)
>  
> I found that this was first seen here, and was patched, but the bug appears 
> to have resurfaced:
> https://issues.apache.org/jira/browse/CASSANDRA-2488



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15899) Dropping a column can break queries until the schema is fully propagated

[Sam Tunnicliffe (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sam Tunnicliffe updated CASSANDRA-15899:

Reviewers: Sam Tunnicliffe, Sam Tunnicliffe  (was: Sam Tunnicliffe)
   Sam Tunnicliffe, Sam Tunnicliffe  (was: Sam Tunnicliffe)
   Status: Review In Progress  (was: Patch Available)

> Dropping a column can break queries until the schema is fully propagated
> 
>
> Key: CASSANDRA-15899
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15899
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Schema
>Reporter: Marcus Eriksson
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 3.0.x
>
>
> With a table like:
> {code}
> CREATE TABLE ks.tbl (id int primary key, v1 int, v2 int, v3 int)
> {code}
> and we drop {{v2}}, we get this exception on the replicas which haven't seen 
> the schema change:
> {code}
> ERROR [SharedPool-Worker-1] node2 2020-06-24 09:49:08,107 
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread 
> Thread[SharedPool-Worker-1,5,node2]
> java.lang.IllegalStateException: [ColumnDefinition{name=v1, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}, 
> ColumnDefinition{name=v2, type=org.apache.cassandra.db.marshal.Int32Type, 
> kind=REGULAR, position=-1}, ColumnDefinition{name=v3, 
> type=org.apache.cassandra.db.marshal.Int32Type, kind=REGULAR, position=-1}] 
> is not a subset of [v1 v3]
>   at 
> org.apache.cassandra.db.Columns$Serializer.encodeBitmap(Columns.java:546) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.Columns$Serializer.serializeSubset(Columns.java:478) 
> ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:184)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:114)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredSerializer.serialize(UnfilteredSerializer.java:102)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:132)
>  ~[main/:na]
>   at 
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
>  ~[main/:na]
> ...
> {code}
> Note that it doesn't matter if we {{SELECT *}} or {{SELECT id, v1}}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping

[Aleksey Yeschenko (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15158:
--
Reviewers: Aleksey Yeschenko, Blake Eggleston  (was: Blake Eggleston)

> Wait for schema agreement rather than in flight schema requests when 
> bootstrapping
> --
>
> Key: CASSANDRA-15158
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15158
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip, Cluster/Schema
>Reporter: Vincent White
>Assignee: Blake Eggleston
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently when a node is bootstrapping we use a set of latches 
> (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of 
> in-flight schema pull requests, and we don't proceed with 
> bootstrapping/stream until all the latches are released (or we timeout 
> waiting for each one). One issue with this is that if we have a large schema, 
> or the retrieval of the schema from the other nodes was unexpectedly slow 
> then we have no explicit check in place to ensure we have actually received a 
> schema before we proceed.
> While it's possible to increase "migration_task_wait_in_seconds" to force the 
> node to wait on each latche longer, there are cases where this doesn't help 
> because the callbacks for the schema pull requests have expired off the 
> messaging service's callback map 
> (org.apache.cassandra.net.MessagingService#callbacks) after 
> request_timeout_in_ms (default 10 seconds) before the other nodes were able 
> to respond to the new node.
> This patch checks for schema agreement between the bootstrapping node and the 
> rest of the live nodes before proceeding with bootstrapping. It also adds a 
> check to prevent the new node from flooding existing nodes with simultaneous 
> schema pull requests as can happen in large clusters.
> Removing the latch system should also prevent new nodes in large clusters 
> getting stuck for extended amounts of time as they wait 
> `migration_task_wait_in_seconds` on each of the latches left orphaned by the 
> timed out callbacks.
>  
> ||3.11||
> |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]|
> |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]|
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-15158) Wait for schema agreement rather than in flight schema requests when bootstrapping

[Aleksey Yeschenko (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Yeschenko updated CASSANDRA-15158:
--
Reviewers: Aleksey Yeschenko, Blake Eggleston, Aleksey Yeschenko  (was: 
Aleksey Yeschenko, Blake Eggleston)
   Aleksey Yeschenko, Blake Eggleston, Aleksey Yeschenko  (was: 
Aleksey Yeschenko, Blake Eggleston)
   Status: Review In Progress  (was: Patch Available)

> Wait for schema agreement rather than in flight schema requests when 
> bootstrapping
> --
>
> Key: CASSANDRA-15158
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15158
> Project: Cassandra
>  Issue Type: Bug
>  Components: Cluster/Gossip, Cluster/Schema
>Reporter: Vincent White
>Assignee: Blake Eggleston
>Priority: Normal
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently when a node is bootstrapping we use a set of latches 
> (org.apache.cassandra.service.MigrationTask#inflightTasks) to keep track of 
> in-flight schema pull requests, and we don't proceed with 
> bootstrapping/stream until all the latches are released (or we timeout 
> waiting for each one). One issue with this is that if we have a large schema, 
> or the retrieval of the schema from the other nodes was unexpectedly slow 
> then we have no explicit check in place to ensure we have actually received a 
> schema before we proceed.
> While it's possible to increase "migration_task_wait_in_seconds" to force the 
> node to wait on each latche longer, there are cases where this doesn't help 
> because the callbacks for the schema pull requests have expired off the 
> messaging service's callback map 
> (org.apache.cassandra.net.MessagingService#callbacks) after 
> request_timeout_in_ms (default 10 seconds) before the other nodes were able 
> to respond to the new node.
> This patch checks for schema agreement between the bootstrapping node and the 
> rest of the live nodes before proceeding with bootstrapping. It also adds a 
> check to prevent the new node from flooding existing nodes with simultaneous 
> schema pull requests as can happen in large clusters.
> Removing the latch system should also prevent new nodes in large clusters 
> getting stuck for extended amounts of time as they wait 
> `migration_task_wait_in_seconds` on each of the latches left orphaned by the 
> timed out callbacks.
>  
> ||3.11||
> |[PoC|https://github.com/apache/cassandra/compare/cassandra-3.11...vincewhite:check_for_schema]|
> |[dtest|https://github.com/apache/cassandra-dtest/compare/master...vincewhite:wait_for_schema_agreement]|
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[cassandra-builds] branch master updated: Jenkins jobs for jvm-dtest-upgrade builds – update cassandra_pipeline.groovy to use new jvm-dtest-upgrade jobs

[mck]

This is an automated email from the ASF dual-hosted git repository.

mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git


The following commit(s) were added to refs/heads/master by this push:
 new dc1e655  Jenkins jobs for jvm-dtest-upgrade builds – update 
cassandra_pipeline.groovy to use new jvm-dtest-upgrade jobs
dc1e655 is described below

commit dc1e65550c8eb2249589abcef73e5235c1755883
Author: mck 
AuthorDate: Thu Aug 20 10:33:44 2020 +0200

Jenkins jobs for jvm-dtest-upgrade builds – update 
cassandra_pipeline.groovy to use new jvm-dtest-upgrade jobs
---
 jenkins-dsl/cassandra_pipeline.groovy | 121 +-
 1 file changed, 59 insertions(+), 62 deletions(-)

diff --git a/jenkins-dsl/cassandra_pipeline.groovy 
b/jenkins-dsl/cassandra_pipeline.groovy
index 408dbde..e8eb114 100644
--- a/jenkins-dsl/cassandra_pipeline.groovy
+++ b/jenkins-dsl/cassandra_pipeline.groovy
@@ -28,12 +28,10 @@ pipeline {
   parallel {
 stage('stress') {
   steps {
-  warnError('Tests unstable') {
-  script {
-stress = build job: "${env.JOB_NAME}-stress-test", 
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', 
value: params.BRANCH)], propagate: false
-if (stress.result != 'SUCCESS') unstable('stress test 
failures')
-  }
-  }
+script {
+  stress = build job: "${env.JOB_NAME}-stress-test", 
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', 
value: params.BRANCH)], propagate: false
+  if (stress.result != 'SUCCESS') unstable('stress test 
failures')
+}
   }
   post {
 always {
@@ -47,12 +45,10 @@ pipeline {
 }
 stage('fqltool') {
   steps {
-  warnError('Tests unstable') {
-  script {
-fqltool = build job: "${env.JOB_NAME}-fqltool-test", 
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', 
value: params.BRANCH)], propagate: false
-if (fqltool.result != 'SUCCESS') unstable('fqltool 
test failures')
-  }
-  }
+script {
+  fqltool = build job: "${env.JOB_NAME}-fqltool-test", 
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', 
value: params.BRANCH)], propagate: false
+  if (fqltool.result != 'SUCCESS') unstable('fqltool test 
failures')
+}
   }
   post {
 always {
@@ -64,14 +60,12 @@ pipeline {
 }
   }
 }
-stage('JVM DTests') {
+stage('jvm-dtest') {
   steps {
-  warnError('Tests unstable') {
-  script {
-jvm_dtest = build job: "${env.JOB_NAME}-jvm-dtest", 
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', 
value: params.BRANCH)], propagate: false
-if (jvm_dtest.result != 'SUCCESS') unstable('jvm-dtest 
failures')
-  }
-  }
+script {
+  jvm_dtest = build job: "${env.JOB_NAME}-jvm-dtest", 
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', 
value: params.BRANCH)], propagate: false
+  if (jvm_dtest.result != 'SUCCESS') unstable('jvm-dtest 
failures')
+}
   }
   post {
 always {
@@ -83,15 +77,30 @@ pipeline {
 }
   }
 }
+stage('jvm-dtest-upgrade') {
+  steps {
+script {
+  jvm_dtest_upgrade = build job: 
"${env.JOB_NAME}-jvm-dtest-upgrade", parameters: [string(name: 'REPO', value: 
params.REPO), string(name: 'BRANCH', value: params.BRANCH)], propagate: false
+  if (jvm_dtest_upgrade.result != 'SUCCESS') 
unstable('jvm-dtest-upgrade failures')
+}
+  }
+  post {
+always {
+warnError('missing test xml files') {
+script {
+copyTestResults('jvm-dtest-upgrade', 
jvm_dtest_upgrade.getNumber())
+}
+}
+}
+  }
+}
 stage('units') {
-steps {
-  warnError('Tests unstable') {
-  script {
-test = build job: "${env.JOB_NAME}-test", parameters: 
[string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', value: 
params.BRANCH)], propagate: false
-if (test.result != 

[jira] [Updated] (CASSANDRA-15802) Commented-out lines that end in a semicolon cause an error.

[Michael Semb Wever (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Michael Semb Wever updated CASSANDRA-15802:
---
Status: Ready to Commit  (was: Changes Suggested)

> Commented-out lines that end in a semicolon cause an error.
> ---
>
> Key: CASSANDRA-15802
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15802
> Project: Cassandra
>  Issue Type: Bug
>  Components: CQL/Interpreter, CQL/Syntax
>Reporter: null 
>Assignee: Rens Groothuijsen
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: cqlsh.png
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Commented-out lines that end in a semicolon cause an error.
> For example:
> /*
> describe keyspaces;
> */
>  
> This produces an error:
> SyntaxException: line 2:22 no viable alternative at input ' (...*
> describe keyspaces;...)
>  
> It works as expected if you use syntax:
> -- describe keyspaces;
>  
> Environment:
> python:3.7.7-slim-stretch (docker image)
>  
> I found that this was first seen here, and was patched, but the bug appears 
> to have resurfaced:
> https://issues.apache.org/jira/browse/CASSANDRA-2488



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-14939) fix some operational holes in incremental repair

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-14939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-14939:

Status: Ready to Commit  (was: Review In Progress)

+1

bq. in getPendingStats when checking if (!Iterables.any(ranges, r -> 
r.intersects(session.ranges))) session could be null - there is a null check 
right below which we could move up.
Probably better to just skip it if the session is null 
({{LocalSessions#getPendingStats}}, line 278), feel free to fix on commit if 
you agree.

> fix some operational holes in incremental repair
> 
>
> Key: CASSANDRA-14939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14939
> Project: Cassandra
>  Issue Type: Bug
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0
>
>
> Incremental repair has a few operational rough spots that make it more 
> difficult to fully automate and operate at scale than it should be.
> * Visibility into whether pending repair data exists for a given token range.
> * Ability to force promotion/demotion of data for completed sessions instead 
> of waiting for compaction.
> * Get the most recent repairedAt timestamp for a given token range.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16062) Avoid NPE in getCompactionInfo

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-16062:

Test and Documentation Plan: cci run
 Status: Patch Available  (was: Open)

https://app.circleci.com/pipelines/github/krummas/cassandra/482/workflows/86158fb6-34aa-4e8a-9d28-e4fc440421b0
https://github.com/krummas/cassandra/commits/marcuse/16062

> Avoid NPE in getCompactionInfo
> --
>
> Key: CASSANDRA-16062
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16062
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We currently initialize {{sstables}} after calling {{beginCompaction(this)}} 
> in the {{CompactionIterator}} constructor which creates a window where we can 
> get NPE creating the {{CompactionInfo}} for cancelling compactions



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Created] (CASSANDRA-16062) Avoid NPE in getCompactionInfo

[Marcus Eriksson (Jira)]

Marcus Eriksson created CASSANDRA-16062:
---

 Summary: Avoid NPE in getCompactionInfo
 Key: CASSANDRA-16062
 URL: https://issues.apache.org/jira/browse/CASSANDRA-16062
 Project: Cassandra
  Issue Type: Bug
  Components: Local/Compaction
Reporter: Marcus Eriksson
Assignee: Marcus Eriksson


We currently initialize {{sstables}} after calling {{beginCompaction(this)}} in 
the {{CompactionIterator}} constructor which creates a window where we can get 
NPE creating the {{CompactionInfo}} for cancelling compactions



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org

[jira] [Updated] (CASSANDRA-16062) Avoid NPE in getCompactionInfo

[Marcus Eriksson (Jira)]

 [ 
https://issues.apache.org/jira/browse/CASSANDRA-16062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcus Eriksson updated CASSANDRA-16062:

 Bug Category: Parent values: Degradation(12984)Level 1 values: Other 
Exception(12998)
   Complexity: Low Hanging Fruit
Discovered By: Unit Test
Fix Version/s: 4.0-beta
 Severity: Low
   Status: Open  (was: Triage Needed)

> Avoid NPE in getCompactionInfo
> --
>
> Key: CASSANDRA-16062
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16062
> Project: Cassandra
>  Issue Type: Bug
>  Components: Local/Compaction
>Reporter: Marcus Eriksson
>Assignee: Marcus Eriksson
>Priority: Normal
> Fix For: 4.0-beta
>
>
> We currently initialize {{sstables}} after calling {{beginCompaction(this)}} 
> in the {{CompactionIterator}} constructor which creates a window where we can 
> get NPE creating the {{CompactionInfo}} for cancelling compactions



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org