[jira] [Comment Edited] (CASSANDRA-15706) Fix cqlsh completion test
[
https://issues.apache.org/jira/browse/CASSANDRA-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078039#comment-17078039
]
Eduard Tudenhoefner edited comment on CASSANDRA-15706 at 4/9/20, 7:01 AM:
--
Relevant cqlsh test on Jenkins is
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch-cqlsh-tests/34/]
was (Author: eduard.tudenhoefner):
Tests on Jenkins
[here|https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/35/pipeline/]
> Fix cqlsh completion test
> -
>
> Key: CASSANDRA-15706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15706
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This has been failing for a while now because *system_views* and
> *system_virtual_schema* are occuring in the completion.
> {code}
> cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_drop_keyspace
> (from nosetests)
> Failing for the past 1 build (Since Unstable#42 )
> Took 2 sec.
> Error Message
> Items in the second set but not the first:
> 'system_views'
> 'system_virtual_schema'
> """Fail immediately, with the given message."""
> >> raise self.failureException("Items in the second set but not the
> >> first:\n'system_views'\n'system_virtual_schema'")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15707) Fix cqlsh output test
[
https://issues.apache.org/jira/browse/CASSANDRA-15707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078988#comment-17078988
]
Eduard Tudenhoefner commented on CASSANDRA-15707:
-
Relevant cqlsh test on Jenkins is
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch-cqlsh-tests/35/],
which shows that the test is fixed
> Fix cqlsh output test
> -
>
> Key: CASSANDRA-15707
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15707
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> https://ci-cassandra.apache.org/view/branches/job/Cassandra-trunk/45/testReport/cqlshlib.test.test_cqlsh_output/TestCqlshOutput/
> {code}
> Sequences differ: ['CRE[438 chars]nt', ") WITH additional_write_policy =
> '99p'",[710 chars], ''] != ['CRE[438 chars]nt', ') WITH
> bloom_filter_fp_chance = 0.01', "[711 chars], '']
> First differing element 17:
> ") WITH additional_write_policy = '99p'"
> ') WITH bloom_filter_fp_chance = 0.01'
> Diff is 1475 characters long. Set self.maxDiff to None to see it.
> """Fail immediately, with the given message."""
> >> raise self.failureException('Sequences differ: [\'CRE[438 chars]nt\', ")
> >> WITH additional_write_policy = \'99p\'",[710 chars], \'\'] != [\'CRE[438
> >> chars]nt\', \') WITH bloom_filter_fp_chance = 0.01\', "[711 chars],
> >> \'\']\n\nFirst differing element 17:\n") WITH additional_write_policy =
> >> \'99p\'"\n\') WITH bloom_filter_fp_chance = 0.01\'\n\nDiff is 1475
> >> characters long. Set self.maxDiff to None to see it.')
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15706) Fix cqlsh completion test
[
https://issues.apache.org/jira/browse/CASSANDRA-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17078039#comment-17078039
]
Eduard Tudenhoefner edited comment on CASSANDRA-15706 at 4/9/20, 7:04 AM:
--
Relevant cqlsh test on Jenkins is
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch-cqlsh-tests/34/],
which shows that the test is fixed
was (Author: eduard.tudenhoefner):
Relevant cqlsh test on Jenkins is
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch-cqlsh-tests/34/]
> Fix cqlsh completion test
> -
>
> Key: CASSANDRA-15706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15706
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This has been failing for a while now because *system_views* and
> *system_virtual_schema* are occuring in the completion.
> {code}
> cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_drop_keyspace
> (from nosetests)
> Failing for the past 1 build (Since Unstable#42 )
> Took 2 sec.
> Error Message
> Items in the second set but not the first:
> 'system_views'
> 'system_virtual_schema'
> """Fail immediately, with the given message."""
> >> raise self.failureException("Items in the second set but not the
> >> first:\n'system_views'\n'system_virtual_schema'")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15573) Python 3.8 fails to execute cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077015#comment-17077015
]
Eduard Tudenhoefner edited comment on CASSANDRA-15573 at 4/9/20, 7:19 AM:
--
||[Patch|https://github.com/apache/cassandra/pull/518]||[CircleCI|https://app.circleci.com/pipelines/github/nastra/cassandra?branch=CASSANDRA-15573]||[Docker
Image update|https://github.com/apache/cassandra-builds/pull/17]||
was (Author: eduard.tudenhoefner):
||[Patch|https://github.com/apache/cassandra/pull/518]||[CircleCI|https://app.circleci.com/pipelines/github/nastra/cassandra/18/workflows/0549d320-a53d-4bff-b8d8-56d1a4ccc8bf]||[Docker
Image update|https://github.com/apache/cassandra-builds/pull/17]||
> Python 3.8 fails to execute cqlsh
> -
>
> Key: CASSANDRA-15573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15573
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/cqlsh
>Reporter: Yuki Morishita
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Python 3.8 renamed sre_parse.Pattern to sre_parse.State (see
> [https://bugs.python.org/issue34681] and corresponding pull request
> [https://github.com/python/cpython/pull/9310])
> So when executing cqlsh with Python 3.8, it throws error:
> {code:java}
> Traceback (most recent call last):
> File ".\bin\cqlsh.py", line 175, in
> from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling,
> cqlshhandling
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cql3handling.py", line 19,
> in
> from cqlshlib.cqlhandling import CqlParsingRuleSet, Hint
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cqlhandling.py", line 23,
> in
> from cqlshlib import pylexotron, util
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 342,
> in
> class ParsingRuleSet:
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 343,
> in ParsingRuleSet
> RuleSpecScanner = SaferScanner([
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\saferscanner.py", line 74,
> in __init__
> s = re.sre_parse.Pattern()
> AttributeError: module 'sre_parse' has no attribute 'Pattern'
> {code}
> h2. Summary of Work that was done
> Added a Python 3.8 compatible SaferScanner implementation ([diff
> here|https://github.com/apache/cassandra/pull/518/commits/2e6813f0ef5817e5d8d655052d61ce75a5fc062c]).
> Note that the changes from CASSANDRA-15659 are required in order to verify
> that the issue is fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15710) DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
Erick Ramirez created CASSANDRA-15710:
-
Summary: DOC - Add cqlsh Python 3 support in NEWS.txt and tools
section of website
Key: CASSANDRA-15710
URL: https://issues.apache.org/jira/browse/CASSANDRA-15710
Project: Cassandra
Issue Type: Improvement
Components: Documentation/NEWS.txt
Reporter: Erick Ramirez
h2. Background
I found out by accident that CASSANDRA-10190 is not documented in {{NEWS.txt}}.
h2. Scope
* {{NEWS.txt}} - Add info on Python 3 support for cqlsh.
* Docs - include new {{--python}} option
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15706) Fix cqlsh completion test
[
https://issues.apache.org/jira/browse/CASSANDRA-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15706:
---
Reviewers: Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Status: Review In Progress (was: Patch Available)
> Fix cqlsh completion test
> -
>
> Key: CASSANDRA-15706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15706
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This has been failing for a while now because *system_views* and
> *system_virtual_schema* are occuring in the completion.
> {code}
> cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_drop_keyspace
> (from nosetests)
> Failing for the past 1 build (Since Unstable#42 )
> Took 2 sec.
> Error Message
> Items in the second set but not the first:
> 'system_views'
> 'system_virtual_schema'
> """Fail immediately, with the given message."""
> >> raise self.failureException("Items in the second set but not the
> >> first:\n'system_views'\n'system_virtual_schema'")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15710) DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
[
https://issues.apache.org/jira/browse/CASSANDRA-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15710:
--
Authors: Erick Ramirez
Reviewers: David Capwell, Dinesh Joshi, Jon Haddad, Patrick Bannister
> DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
> -
>
> Key: CASSANDRA-15710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15710
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Priority: Normal
>
> h2. Background
> I found out by accident that CASSANDRA-10190 is not documented in
> {{NEWS.txt}}.
> h2. Scope
> * {{NEWS.txt}} - Add info on Python 3 support for cqlsh.
> * Docs - include new {{--python}} option
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14512) DESCRIBE behavior is broken for Virtual Keyspaces/Tables (CQLSH)
[
https://issues.apache.org/jira/browse/CASSANDRA-14512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-14512:
---
Resolution: Duplicate
Status: Resolved (was: Open)
> DESCRIBE behavior is broken for Virtual Keyspaces/Tables (CQLSH)
>
>
> Key: CASSANDRA-14512
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14512
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL, Legacy/Observability
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Labels: virtual-tables
> Fix For: 4.0-alpha
>
>
> The {{DESCRIBE}} command in CQLSH does not work properly for virtual
> keyspaces/tables.
> # For the {{DESCRIBE KEYSPACES}} the virtual keyspaces are correctly added to
> the list but for {{DESCRIBE TABLES}} only the non virtual tables are
> displayed.
> # {{DESCRIBE system_views}} return the error: {{'system_views' not found in
> keyspaces}}. Similar error for {{DESCRIBE system_virtual_schema}}.
> # {{DESCRIBE KEYSPACE system_views}} or {{DESCRIBE KEYSPACE
> system_virtual_schema}} return the error: {{'NoneType' object has no
> attribute 'export_for_schema'}}
> The {{DESCRIBE TABLE}} command works fine but the output might be confusing
> as it is a {{CREATE}} statement.
> {code}
> cqlsh> DESCRIBE TABLE system_virtual_schema.tables;
> CREATE TABLE system_virtual_schema.tables (
> comment text,
> keyspace_name text,
> table_name text,
> PRIMARY KEY (keyspace_name, table_name)
> ) WITH CLUSTERING ORDER BY (table_name ASC)
> AND compaction = {'class': 'None'}
> AND compression = {};
> {code}
> I would be in favor or replacing the {{CREATE TABLE}} by a {{VIRTUAL TABLE}}.
> [~cnlwsu], [~iamaleksey] What do you think?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15710) DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
[
https://issues.apache.org/jira/browse/CASSANDRA-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15710:
--
Change Category: Code Clarity
Complexity: Normal
Assignee: Erick Ramirez
Status: Open (was: Triage Needed)
> DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
> -
>
> Key: CASSANDRA-15710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15710
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
>
> h2. Background
> I found out by accident that CASSANDRA-10190 is not documented in
> {{NEWS.txt}}.
> h2. Scope
> * {{NEWS.txt}} - Add info on Python 3 support for cqlsh.
> * Docs - include new {{--python}} option
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14512) DESCRIBE behavior is broken for Virtual Keyspaces/Tables (CQLSH)
[
https://issues.apache.org/jira/browse/CASSANDRA-14512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-14512:
---
Fix Version/s: 4.0-alpha
> DESCRIBE behavior is broken for Virtual Keyspaces/Tables (CQLSH)
>
>
> Key: CASSANDRA-14512
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14512
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL, Legacy/Observability
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Labels: virtual-tables
> Fix For: 4.0-alpha
>
>
> The {{DESCRIBE}} command in CQLSH does not work properly for virtual
> keyspaces/tables.
> # For the {{DESCRIBE KEYSPACES}} the virtual keyspaces are correctly added to
> the list but for {{DESCRIBE TABLES}} only the non virtual tables are
> displayed.
> # {{DESCRIBE system_views}} return the error: {{'system_views' not found in
> keyspaces}}. Similar error for {{DESCRIBE system_virtual_schema}}.
> # {{DESCRIBE KEYSPACE system_views}} or {{DESCRIBE KEYSPACE
> system_virtual_schema}} return the error: {{'NoneType' object has no
> attribute 'export_for_schema'}}
> The {{DESCRIBE TABLE}} command works fine but the output might be confusing
> as it is a {{CREATE}} statement.
> {code}
> cqlsh> DESCRIBE TABLE system_virtual_schema.tables;
> CREATE TABLE system_virtual_schema.tables (
> comment text,
> keyspace_name text,
> table_name text,
> PRIMARY KEY (keyspace_name, table_name)
> ) WITH CLUSTERING ORDER BY (table_name ASC)
> AND compaction = {'class': 'None'}
> AND compression = {};
> {code}
> I would be in favor or replacing the {{CREATE TABLE}} by a {{VIRTUAL TABLE}}.
> [~cnlwsu], [~iamaleksey] What do you think?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15710) DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
[
https://issues.apache.org/jira/browse/CASSANDRA-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15710:
--
Attachment: 15710-trunk.txt
> DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
> -
>
> Key: CASSANDRA-15710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15710
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
> Attachments: 15710-trunk.txt
>
>
> h2. Background
> I found out by accident that CASSANDRA-10190 is not documented in
> {{NEWS.txt}}.
> h2. Scope
> * {{NEWS.txt}} - Add info on Python 3 support for cqlsh.
> * Docs - include new {{--python}} option
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15710) DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
[
https://issues.apache.org/jira/browse/CASSANDRA-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15710:
---
Labels: pull-request-available (was: )
> DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
> -
>
> Key: CASSANDRA-15710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15710
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
> Labels: pull-request-available
> Attachments: 15710-trunk.txt
>
>
> h2. Background
> I found out by accident that CASSANDRA-10190 is not documented in
> {{NEWS.txt}}.
> h2. Scope
> * {{NEWS.txt}} - Add info on Python 3 support for cqlsh.
> * Docs - include new {{--python}} option
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15710) DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
[
https://issues.apache.org/jira/browse/CASSANDRA-15710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Ramirez updated CASSANDRA-15710:
--
Test and Documentation Plan: CASSANDRA-15710 Added Python 3 support to
NEWS.txt, cqlsh doc
Status: Patch Available (was: In Progress)
Patch available here for review:
* Pull request - [PR #529|https://github.com/apache/cassandra/pull/529]
* Attached - [^15710-trunk.txt]
> DOC - Add cqlsh Python 3 support in NEWS.txt and tools section of website
> -
>
> Key: CASSANDRA-15710
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15710
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/NEWS.txt
>Reporter: Erick Ramirez
>Assignee: Erick Ramirez
>Priority: Normal
> Labels: pull-request-available
> Attachments: 15710-trunk.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h2. Background
> I found out by accident that CASSANDRA-10190 is not documented in
> {{NEWS.txt}}.
> h2. Scope
> * {{NEWS.txt}} - Add info on Python 3 support for cqlsh.
> * Docs - include new {{--python}} option
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Mark system_views/system_virtual_schema as system keyspaces in cqlsh
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new b3f2738 Mark system_views/system_virtual_schema as system keyspaces
in cqlsh
b3f2738 is described below
commit b3f27381a5efcf3d06bceb211c72479abcd6004a
Author: Eduard Tudenhoefner
AuthorDate: Wed Apr 8 11:43:39 2020 +0200
Mark system_views/system_virtual_schema as system keyspaces in cqlsh
patch by Eduard Tudenhoefner; reviewed by Benjamin Lerer for
CASSANDRA-15706
This is so that those keyspaces don't show up in tab completions. This
also fixes the cqlsh completion tests.
---
pylib/cqlshlib/cql3handling.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index eabb059..8738f10 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -34,7 +34,7 @@ class UnexpectedTableStructure(UserWarning):
return 'Unexpected table structure; may not translate correctly to
CQL. ' + self.msg
-SYSTEM_KEYSPACES = ('system', 'system_schema', 'system_traces', 'system_auth',
'system_distributed')
+SYSTEM_KEYSPACES = ('system', 'system_schema', 'system_traces', 'system_auth',
'system_distributed', 'system_views', 'system_virtual_schema')
NONALTERBALE_KEYSPACES = ('system', 'system_schema')
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15706) Fix cqlsh completion test
[
https://issues.apache.org/jira/browse/CASSANDRA-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15706:
---
Status: Ready to Commit (was: Review In Progress)
> Fix cqlsh completion test
> -
>
> Key: CASSANDRA-15706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15706
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This has been failing for a while now because *system_views* and
> *system_virtual_schema* are occuring in the completion.
> {code}
> cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_drop_keyspace
> (from nosetests)
> Failing for the past 1 build (Since Unstable#42 )
> Took 2 sec.
> Error Message
> Items in the second set but not the first:
> 'system_views'
> 'system_virtual_schema'
> """Fail immediately, with the given message."""
> >> raise self.failureException("Items in the second set but not the
> >> first:\n'system_views'\n'system_virtual_schema'")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15706) Fix cqlsh completion test
[
https://issues.apache.org/jira/browse/CASSANDRA-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15706:
---
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/b3f27381a5efcf3d06bceb211c72479abcd6004a
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed into trunk at b3f27381a5efcf3d06bceb211c72479abcd6004a
> Fix cqlsh completion test
> -
>
> Key: CASSANDRA-15706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15706
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This has been failing for a while now because *system_views* and
> *system_virtual_schema* are occuring in the completion.
> {code}
> cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_drop_keyspace
> (from nosetests)
> Failing for the past 1 build (Since Unstable#42 )
> Took 2 sec.
> Error Message
> Items in the second set but not the first:
> 'system_views'
> 'system_virtual_schema'
> """Fail immediately, with the given message."""
> >> raise self.failureException("Items in the second set but not the
> >> first:\n'system_views'\n'system_virtual_schema'")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
Benjamin Lerer created CASSANDRA-15711:
--
Summary: cqlsh ALTER KEYSPACE completion show virtual table
keyspaces
Key: CASSANDRA-15711
URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
Project: Cassandra
Issue Type: Bug
Components: Tool/cqlsh
Reporter: Benjamin Lerer
{code}
cqlsh> ALTER KEYSPACE
system_traces system_virtual_schema system_auth system_views
system_distributed
{code}
The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual tables
keyspaces that cannot be altered and by consequence they should not appear in
this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15711:
---
Bug Category: Parent values: Correctness(12982)
Complexity: Low Hanging Fruit
Discovered By: Code Inspection
Severity: Low
Since Version: 4.0-alpha
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Priority: Low
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15711:
---
Fix Version/s: 4.0
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Priority: Low
> Fix For: 4.0
>
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15659) Better support of Python 3 for cqlsh
[ https://issues.apache.org/jira/browse/CASSANDRA-15659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077009#comment-17077009 ] Eduard Tudenhoefner edited comment on CASSANDRA-15659 at 4/9/20, 9:25 AM: -- ||Pull Request||Tests||Docker Image||Jenkins|| |[PR#505|https://github.com/apache/cassandra/pull/505]|[CircleCI|https://app.circleci.com/pipelines/github/nastra/cassandra/19/workflows/dc0a8771-321f-40bf-a5e8-6e05b20bbca1]|[PR|https://github.com/apache/cassandra-builds/pull/17]|[Jenkins tests|https://ci-cassandra.apache.org/job/Cassandra-devbranch/42/]| Note that the test failures when testing with Python 3.8 are due to CASSANDRA-15573 and should be fixed in [this|https://app.circleci.com/pipelines/github/nastra/cassandra/18/workflows/0549d320-a53d-4bff-b8d8-56d1a4ccc8bf] test run. was (Author: eduard.tudenhoefner): ||Pull Request||Tests||Docker Image|| |[PR#505|https://github.com/apache/cassandra/pull/505]|[CircleCI|https://app.circleci.com/pipelines/github/nastra/cassandra/19/workflows/dc0a8771-321f-40bf-a5e8-6e05b20bbca1]|[PR|https://github.com/apache/cassandra-builds/pull/17] Note that the test failures when testing with Python 3.8 are due to CASSANDRA-15573 and should be fixed in [this|https://app.circleci.com/pipelines/github/nastra/cassandra/18/workflows/0549d320-a53d-4bff-b8d8-56d1a4ccc8bf] test run. > Better support of Python 3 for cqlsh > > > Key: CASSANDRA-15659 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15659 > Project: Cassandra > Issue Type: Task > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Assignee: Eduard Tudenhoefner >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 40m > Remaining Estimate: 0h > > h2. From mailing list: > [https://lists.apache.org/thread.html/r377099b632c62b641e4feef5b738084fc5369b0c7157fae867853597%40%3Cdev.cassandra.apache.org%3E] > > As of today (24/3/2020) and current trunk, there is Python 3.6 supported (1) > but there is not any 3.6 version ootb in Debian for example. E.g. Buster has > Python 3.7 and other (recent) releases have version 2.7. This means that if > one wants to use Python 3 in Debian, he has to use 3.6 but it is not in the > repository so he has to download / compile / install it on his own. > There should be some sane Python 3 version supported which is as well present > in Debian repository (or requirement to run with 3.6 should be relaxed) . > (1) > [https://github.com/apache/cassandra/blob/bf9a1d487b9ba469e8d740cf7d1cd419535a7e79/bin/cqlsh#L57-L65] > h2. Summary of work that was done: > I relaxed the requirement of *cqlsh* only working with Python 2.7 & 3.6 by > allowing Python 3.6+. > Note that I left the constraint for Python 3.6 being the minimum Python3 > version. > As [~ptbannister] pointed out, we could remove the Python 3.6 min version > once we remove Python 2.7 support, as otherwise testing with lots of > different Python versions will get costly. > 2 Dockerfiles were added in *pylib* for minimal local testing of *cqlsh* > starting up with Python 3.7 & 3.8 and that both revealed > CASSANDRA-15572 and CASSANDRA-15573. > CASSANDRA-15572 was fixed here as it was a one-liner. And I'm going to > tackle CASSANDRA-15573 later. > Python 3.8 testing was added to the CircleCI config so that we can actually > see what else breaks with newer Python versions. > A new Docker images with Ubuntu 19.10 was required for testing > ([https://github.com/apache/cassandra-builds/pull/17]). This docker image > sets up Python 2.7/3.6/3.7/3.8 with their respective virtual environments, > which are then being used by the CircleCI yaml. > The image *spod/cassandra-testing-ubuntu1810-java11-w-dependencies:20190306* > couldn't be updated unfortunately because it can't be built anymore, due to > Ubuntu 18.10 being EOL. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15706) Fix cqlsh completion test
[
https://issues.apache.org/jira/browse/CASSANDRA-15706?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079101#comment-17079101
]
Benjamin Lerer commented on CASSANDRA-15706:
[~eduard.tudenhoefner] Thanks for the patch.
> Fix cqlsh completion test
> -
>
> Key: CASSANDRA-15706
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15706
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This has been failing for a while now because *system_views* and
> *system_virtual_schema* are occuring in the completion.
> {code}
> cqlshlib.test.test_cqlsh_completion.TestCqlshCompletion.test_complete_in_drop_keyspace
> (from nosetests)
> Failing for the past 1 build (Since Unstable#42 )
> Took 2 sec.
> Error Message
> Items in the second set but not the first:
> 'system_views'
> 'system_virtual_schema'
> """Fail immediately, with the given message."""
> >> raise self.failureException("Items in the second set but not the
> >> first:\n'system_views'\n'system_virtual_schema'")
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15700) Performance regression on internode messaging
[
https://issues.apache.org/jira/browse/CASSANDRA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079102#comment-17079102
]
Sergio Bossa commented on CASSANDRA-15700:
--
Thank you both.
> Performance regression on internode messaging
> -
>
> Key: CASSANDRA-15700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15700
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Oss40patchedvsOss311.png, Oss40vsOss311.png, oss40.gc,
> oss40_nogc.tar.xz, oss40_system.log
>
>
> Me and [~jasonstack] have been investigating a performance regression
> affecting 4.0 during a 3 nodes, RF 3 write throughput test with a timeseries
> like workload, as shown in this plot, where blue is 3.11 and orange is 4.0:
> !Oss40vsOss311.png|width=389,height=214!
> It's been a bit of a long investigation, but two clues ended up standing out:
> 1) An abnormal number of expired messages on 4.0 (as shown in the attached
> system log), while 3.11 has almost none.
> 2) An abnormal GC activity (as shown in the attached gc log).
> Turns out the two are related, as the [on expired
> callback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundConnection.java#L462]
> creates a huge amount of strings in the {{id()}} call. The next question is
> what causes all those message expirations; we thoroughly reviewed the
> internode messaging code and the only issue we could find so far is related
> to the "batch pruning" calls
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L81]
> and
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L188]:
> it _seems_ too much time is spent on those, causing the event loop to fall
> behind in processing the rest of the messages, which will end up being
> expired. This is supported by the analysis of the collapsed stacks (after
> fixing the GC issue):
> {noformat}
> (tprint (top-aggregated-calls oss40nogc "EventLoopDelivery:doRun" 5))
> org/apache/cassandra/net/OutboundConnection$EventLoopDelivery:doRun 3456
> org/apache/cassandra/net/OutboundMessageQueue:access$600 1621
> org/apache/cassandra/net/PrunableArrayQueue:prune 1621
> org/apache/cassandra/net/OutboundMessageQueue$WithLock:close 1621
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1620
> {noformat}
> Those are the top 5 sampled calls from {{EventLoopDelivery#doRun()}} which
> spends half of its time pruning. But only a tiny portion of such pruning time
> is spent actually expiring:
> {noformat}
> (tprint (top-aggregated-calls oss40nogc
> "OutboundMessageQueue:pruneInternalQueueWithLock" 5))
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1900
> org/apache/cassandra/net/PrunableArrayQueue:prune 1894
> org/apache/cassandra/net/OutboundMessageQueue$1Pruner:onPruned 147
> org/apache/cassandra/net/OutboundConnection$$Lambda$444/740904487:accept 147
> org/apache/cassandra/net/OutboundConnection:onExpired 147
> {noformat}
> And indeed, the {{PrunableArrayQueue:prune()}} self time is dominant:
> {noformat}
> (tprint (top-self-calls oss40nogc "PrunableArrayQueue:prune" 5))
> org/apache/cassandra/net/PrunableArrayQueue:prune 1718
> org/apache/cassandra/net/OutboundConnection:releaseCapacity 27
> java/util/concurrent/ConcurrentHashMap:replaceNode 19
> java/util/concurrent/ConcurrentLinkedQueue:offer 16
> java/util/concurrent/LinkedBlockingQueue:offer 15
> {noformat}
> That said, before proceeding with a PR to fix those issues, I'd like to
> understand: what's the reason to prune so often, rather than just when
> polling the message during delivery? If there's a reason I'm missing, let's
> talk about how to optimize pruning, otherwise let's get rid of that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15573) Python 3.8 fails to execute cqlsh
[
https://issues.apache.org/jira/browse/CASSANDRA-15573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17077015#comment-17077015
]
Eduard Tudenhoefner edited comment on CASSANDRA-15573 at 4/9/20, 9:26 AM:
--
||[Patch|https://github.com/apache/cassandra/pull/518]||[CircleCI|https://app.circleci.com/pipelines/github/nastra/cassandra?branch=CASSANDRA-15573]||[Docker
Image
update|https://github.com/apache/cassandra-builds/pull/17]||[Jenkins|https://ci-cassandra.apache.org/job/Cassandra-devbranch/43/]||
was (Author: eduard.tudenhoefner):
||[Patch|https://github.com/apache/cassandra/pull/518]||[CircleCI|https://app.circleci.com/pipelines/github/nastra/cassandra?branch=CASSANDRA-15573]||[Docker
Image update|https://github.com/apache/cassandra-builds/pull/17]||
> Python 3.8 fails to execute cqlsh
> -
>
> Key: CASSANDRA-15573
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15573
> Project: Cassandra
> Issue Type: Sub-task
> Components: Tool/cqlsh
>Reporter: Yuki Morishita
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Python 3.8 renamed sre_parse.Pattern to sre_parse.State (see
> [https://bugs.python.org/issue34681] and corresponding pull request
> [https://github.com/python/cpython/pull/9310])
> So when executing cqlsh with Python 3.8, it throws error:
> {code:java}
> Traceback (most recent call last):
> File ".\bin\cqlsh.py", line 175, in
> from cqlshlib import cql3handling, cqlhandling, pylexotron, sslhandling,
> cqlshhandling
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cql3handling.py", line 19,
> in
> from cqlshlib.cqlhandling import CqlParsingRuleSet, Hint
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\cqlhandling.py", line 23,
> in
> from cqlshlib import pylexotron, util
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 342,
> in
> class ParsingRuleSet:
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\pylexotron.py", line 343,
> in ParsingRuleSet
> RuleSpecScanner = SaferScanner([
> File "C:\Users\Yuki
> Morishita\Projects\cassandra\bin\..\pylib\cqlshlib\saferscanner.py", line 74,
> in __init__
> s = re.sre_parse.Pattern()
> AttributeError: module 'sre_parse' has no attribute 'Pattern'
> {code}
> h2. Summary of Work that was done
> Added a Python 3.8 compatible SaferScanner implementation ([diff
> here|https://github.com/apache/cassandra/pull/518/commits/2e6813f0ef5817e5d8d655052d61ce75a5fc062c]).
> Note that the changes from CASSANDRA-15659 are required in order to verify
> that the issue is fixed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15700) Performance regression on internode messaging
[
https://issues.apache.org/jira/browse/CASSANDRA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Bossa updated CASSANDRA-15700:
-
Authors: Sergio Bossa (was: Sergio Bossa, ZhaoYang)
> Performance regression on internode messaging
> -
>
> Key: CASSANDRA-15700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15700
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Normal
> Fix For: 4.0-beta
>
> Attachments: Oss40patchedvsOss311.png, Oss40vsOss311.png, oss40.gc,
> oss40_nogc.tar.xz, oss40_system.log
>
>
> Me and [~jasonstack] have been investigating a performance regression
> affecting 4.0 during a 3 nodes, RF 3 write throughput test with a timeseries
> like workload, as shown in this plot, where blue is 3.11 and orange is 4.0:
> !Oss40vsOss311.png|width=389,height=214!
> It's been a bit of a long investigation, but two clues ended up standing out:
> 1) An abnormal number of expired messages on 4.0 (as shown in the attached
> system log), while 3.11 has almost none.
> 2) An abnormal GC activity (as shown in the attached gc log).
> Turns out the two are related, as the [on expired
> callback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundConnection.java#L462]
> creates a huge amount of strings in the {{id()}} call. The next question is
> what causes all those message expirations; we thoroughly reviewed the
> internode messaging code and the only issue we could find so far is related
> to the "batch pruning" calls
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L81]
> and
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L188]:
> it _seems_ too much time is spent on those, causing the event loop to fall
> behind in processing the rest of the messages, which will end up being
> expired. This is supported by the analysis of the collapsed stacks (after
> fixing the GC issue):
> {noformat}
> (tprint (top-aggregated-calls oss40nogc "EventLoopDelivery:doRun" 5))
> org/apache/cassandra/net/OutboundConnection$EventLoopDelivery:doRun 3456
> org/apache/cassandra/net/OutboundMessageQueue:access$600 1621
> org/apache/cassandra/net/PrunableArrayQueue:prune 1621
> org/apache/cassandra/net/OutboundMessageQueue$WithLock:close 1621
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1620
> {noformat}
> Those are the top 5 sampled calls from {{EventLoopDelivery#doRun()}} which
> spends half of its time pruning. But only a tiny portion of such pruning time
> is spent actually expiring:
> {noformat}
> (tprint (top-aggregated-calls oss40nogc
> "OutboundMessageQueue:pruneInternalQueueWithLock" 5))
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1900
> org/apache/cassandra/net/PrunableArrayQueue:prune 1894
> org/apache/cassandra/net/OutboundMessageQueue$1Pruner:onPruned 147
> org/apache/cassandra/net/OutboundConnection$$Lambda$444/740904487:accept 147
> org/apache/cassandra/net/OutboundConnection:onExpired 147
> {noformat}
> And indeed, the {{PrunableArrayQueue:prune()}} self time is dominant:
> {noformat}
> (tprint (top-self-calls oss40nogc "PrunableArrayQueue:prune" 5))
> org/apache/cassandra/net/PrunableArrayQueue:prune 1718
> org/apache/cassandra/net/OutboundConnection:releaseCapacity 27
> java/util/concurrent/ConcurrentHashMap:replaceNode 19
> java/util/concurrent/ConcurrentLinkedQueue:offer 16
> java/util/concurrent/LinkedBlockingQueue:offer 15
> {noformat}
> That said, before proceeding with a PR to fix those issues, I'd like to
> understand: what's the reason to prune so often, rather than just when
> polling the message during delivery? If there's a reason I'm missing, let's
> talk about how to optimize pruning, otherwise let's get rid of that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduard Tudenhoefner reassigned CASSANDRA-15711:
---
Assignee: Eduard Tudenhoefner
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Fix For: 4.0
>
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15711:
---
Reviewers: Benjamin Lerer
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Fix For: 4.0
>
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduard Tudenhoefner updated CASSANDRA-15711:
Status: Open (was: Triage Needed)
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15711:
---
Labels: pull-request-available (was: )
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eduard Tudenhoefner updated CASSANDRA-15711:
Test and Documentation Plan: execute *ALTER KEYSPACE* and
*system_virtual_schema* and *system_views* shouldn't show up in the completion
Status: Patch Available (was: In Progress)
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15700) Performance regression on internode messaging
[
https://issues.apache.org/jira/browse/CASSANDRA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-15700:
---
Labels: pull-request-available (was: )
> Performance regression on internode messaging
> -
>
> Key: CASSANDRA-15700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15700
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: Oss40patchedvsOss311.png, Oss40vsOss311.png, oss40.gc,
> oss40_nogc.tar.xz, oss40_system.log
>
>
> Me and [~jasonstack] have been investigating a performance regression
> affecting 4.0 during a 3 nodes, RF 3 write throughput test with a timeseries
> like workload, as shown in this plot, where blue is 3.11 and orange is 4.0:
> !Oss40vsOss311.png|width=389,height=214!
> It's been a bit of a long investigation, but two clues ended up standing out:
> 1) An abnormal number of expired messages on 4.0 (as shown in the attached
> system log), while 3.11 has almost none.
> 2) An abnormal GC activity (as shown in the attached gc log).
> Turns out the two are related, as the [on expired
> callback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundConnection.java#L462]
> creates a huge amount of strings in the {{id()}} call. The next question is
> what causes all those message expirations; we thoroughly reviewed the
> internode messaging code and the only issue we could find so far is related
> to the "batch pruning" calls
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L81]
> and
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L188]:
> it _seems_ too much time is spent on those, causing the event loop to fall
> behind in processing the rest of the messages, which will end up being
> expired. This is supported by the analysis of the collapsed stacks (after
> fixing the GC issue):
> {noformat}
> (tprint (top-aggregated-calls oss40nogc "EventLoopDelivery:doRun" 5))
> org/apache/cassandra/net/OutboundConnection$EventLoopDelivery:doRun 3456
> org/apache/cassandra/net/OutboundMessageQueue:access$600 1621
> org/apache/cassandra/net/PrunableArrayQueue:prune 1621
> org/apache/cassandra/net/OutboundMessageQueue$WithLock:close 1621
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1620
> {noformat}
> Those are the top 5 sampled calls from {{EventLoopDelivery#doRun()}} which
> spends half of its time pruning. But only a tiny portion of such pruning time
> is spent actually expiring:
> {noformat}
> (tprint (top-aggregated-calls oss40nogc
> "OutboundMessageQueue:pruneInternalQueueWithLock" 5))
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1900
> org/apache/cassandra/net/PrunableArrayQueue:prune 1894
> org/apache/cassandra/net/OutboundMessageQueue$1Pruner:onPruned 147
> org/apache/cassandra/net/OutboundConnection$$Lambda$444/740904487:accept 147
> org/apache/cassandra/net/OutboundConnection:onExpired 147
> {noformat}
> And indeed, the {{PrunableArrayQueue:prune()}} self time is dominant:
> {noformat}
> (tprint (top-self-calls oss40nogc "PrunableArrayQueue:prune" 5))
> org/apache/cassandra/net/PrunableArrayQueue:prune 1718
> org/apache/cassandra/net/OutboundConnection:releaseCapacity 27
> java/util/concurrent/ConcurrentHashMap:replaceNode 19
> java/util/concurrent/ConcurrentLinkedQueue:offer 16
> java/util/concurrent/LinkedBlockingQueue:offer 15
> {noformat}
> That said, before proceeding with a PR to fix those issues, I'd like to
> understand: what's the reason to prune so often, rather than just when
> polling the message during delivery? If there's a reason I'm missing, let's
> talk about how to optimize pruning, otherwise let's get rid of that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15700) Performance regression on internode messaging
[
https://issues.apache.org/jira/browse/CASSANDRA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Bossa updated CASSANDRA-15700:
-
Source Control Link: https://github.com/apache/cassandra/pull/531
> Performance regression on internode messaging
> -
>
> Key: CASSANDRA-15700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15700
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: Oss40patchedvsOss311.png, Oss40vsOss311.png, oss40.gc,
> oss40_nogc.tar.xz, oss40_system.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Me and [~jasonstack] have been investigating a performance regression
> affecting 4.0 during a 3 nodes, RF 3 write throughput test with a timeseries
> like workload, as shown in this plot, where blue is 3.11 and orange is 4.0:
> !Oss40vsOss311.png|width=389,height=214!
> It's been a bit of a long investigation, but two clues ended up standing out:
> 1) An abnormal number of expired messages on 4.0 (as shown in the attached
> system log), while 3.11 has almost none.
> 2) An abnormal GC activity (as shown in the attached gc log).
> Turns out the two are related, as the [on expired
> callback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundConnection.java#L462]
> creates a huge amount of strings in the {{id()}} call. The next question is
> what causes all those message expirations; we thoroughly reviewed the
> internode messaging code and the only issue we could find so far is related
> to the "batch pruning" calls
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L81]
> and
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L188]:
> it _seems_ too much time is spent on those, causing the event loop to fall
> behind in processing the rest of the messages, which will end up being
> expired. This is supported by the analysis of the collapsed stacks (after
> fixing the GC issue):
> {noformat}
> (tprint (top-aggregated-calls oss40nogc "EventLoopDelivery:doRun" 5))
> org/apache/cassandra/net/OutboundConnection$EventLoopDelivery:doRun 3456
> org/apache/cassandra/net/OutboundMessageQueue:access$600 1621
> org/apache/cassandra/net/PrunableArrayQueue:prune 1621
> org/apache/cassandra/net/OutboundMessageQueue$WithLock:close 1621
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1620
> {noformat}
> Those are the top 5 sampled calls from {{EventLoopDelivery#doRun()}} which
> spends half of its time pruning. But only a tiny portion of such pruning time
> is spent actually expiring:
> {noformat}
> (tprint (top-aggregated-calls oss40nogc
> "OutboundMessageQueue:pruneInternalQueueWithLock" 5))
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1900
> org/apache/cassandra/net/PrunableArrayQueue:prune 1894
> org/apache/cassandra/net/OutboundMessageQueue$1Pruner:onPruned 147
> org/apache/cassandra/net/OutboundConnection$$Lambda$444/740904487:accept 147
> org/apache/cassandra/net/OutboundConnection:onExpired 147
> {noformat}
> And indeed, the {{PrunableArrayQueue:prune()}} self time is dominant:
> {noformat}
> (tprint (top-self-calls oss40nogc "PrunableArrayQueue:prune" 5))
> org/apache/cassandra/net/PrunableArrayQueue:prune 1718
> org/apache/cassandra/net/OutboundConnection:releaseCapacity 27
> java/util/concurrent/ConcurrentHashMap:replaceNode 19
> java/util/concurrent/ConcurrentLinkedQueue:offer 16
> java/util/concurrent/LinkedBlockingQueue:offer 15
> {noformat}
> That said, before proceeding with a PR to fix those issues, I'd like to
> understand: what's the reason to prune so often, rather than just when
> polling the message during delivery? If there's a reason I'm missing, let's
> talk about how to optimize pruning, otherwise let's get rid of that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15700) Performance regression on internode messaging
[
https://issues.apache.org/jira/browse/CASSANDRA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Bossa updated CASSANDRA-15700:
-
Test and Documentation Plan: See attached performance test plots. Also see
unit tests.
Status: Patch Available (was: In Progress)
> Performance regression on internode messaging
> -
>
> Key: CASSANDRA-15700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15700
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: Oss40patchedvsOss311.png, Oss40vsOss311.png, oss40.gc,
> oss40_nogc.tar.xz, oss40_system.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Me and [~jasonstack] have been investigating a performance regression
> affecting 4.0 during a 3 nodes, RF 3 write throughput test with a timeseries
> like workload, as shown in this plot, where blue is 3.11 and orange is 4.0:
> !Oss40vsOss311.png|width=389,height=214!
> It's been a bit of a long investigation, but two clues ended up standing out:
> 1) An abnormal number of expired messages on 4.0 (as shown in the attached
> system log), while 3.11 has almost none.
> 2) An abnormal GC activity (as shown in the attached gc log).
> Turns out the two are related, as the [on expired
> callback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundConnection.java#L462]
> creates a huge amount of strings in the {{id()}} call. The next question is
> what causes all those message expirations; we thoroughly reviewed the
> internode messaging code and the only issue we could find so far is related
> to the "batch pruning" calls
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L81]
> and
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L188]:
> it _seems_ too much time is spent on those, causing the event loop to fall
> behind in processing the rest of the messages, which will end up being
> expired. This is supported by the analysis of the collapsed stacks (after
> fixing the GC issue):
> {noformat}
> (tprint (top-aggregated-calls oss40nogc "EventLoopDelivery:doRun" 5))
> org/apache/cassandra/net/OutboundConnection$EventLoopDelivery:doRun 3456
> org/apache/cassandra/net/OutboundMessageQueue:access$600 1621
> org/apache/cassandra/net/PrunableArrayQueue:prune 1621
> org/apache/cassandra/net/OutboundMessageQueue$WithLock:close 1621
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1620
> {noformat}
> Those are the top 5 sampled calls from {{EventLoopDelivery#doRun()}} which
> spends half of its time pruning. But only a tiny portion of such pruning time
> is spent actually expiring:
> {noformat}
> (tprint (top-aggregated-calls oss40nogc
> "OutboundMessageQueue:pruneInternalQueueWithLock" 5))
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1900
> org/apache/cassandra/net/PrunableArrayQueue:prune 1894
> org/apache/cassandra/net/OutboundMessageQueue$1Pruner:onPruned 147
> org/apache/cassandra/net/OutboundConnection$$Lambda$444/740904487:accept 147
> org/apache/cassandra/net/OutboundConnection:onExpired 147
> {noformat}
> And indeed, the {{PrunableArrayQueue:prune()}} self time is dominant:
> {noformat}
> (tprint (top-self-calls oss40nogc "PrunableArrayQueue:prune" 5))
> org/apache/cassandra/net/PrunableArrayQueue:prune 1718
> org/apache/cassandra/net/OutboundConnection:releaseCapacity 27
> java/util/concurrent/ConcurrentHashMap:replaceNode 19
> java/util/concurrent/ConcurrentLinkedQueue:offer 16
> java/util/concurrent/LinkedBlockingQueue:offer 15
> {noformat}
> That said, before proceeding with a PR to fix those issues, I'd like to
> understand: what's the reason to prune so often, rather than just when
> polling the message during delivery? If there's a reason I'm missing, let's
> talk about how to optimize pruning, otherwise let's get rid of that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15700) Performance regression on internode messaging
[
https://issues.apache.org/jira/browse/CASSANDRA-15700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079208#comment-17079208
]
Sergio Bossa commented on CASSANDRA-15700:
--
Added some more thorough unit tests in the meantime.
> Performance regression on internode messaging
> -
>
> Key: CASSANDRA-15700
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15700
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Internode
>Reporter: Sergio Bossa
>Assignee: Sergio Bossa
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-beta
>
> Attachments: Oss40patchedvsOss311.png, Oss40vsOss311.png, oss40.gc,
> oss40_nogc.tar.xz, oss40_system.log
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Me and [~jasonstack] have been investigating a performance regression
> affecting 4.0 during a 3 nodes, RF 3 write throughput test with a timeseries
> like workload, as shown in this plot, where blue is 3.11 and orange is 4.0:
> !Oss40vsOss311.png|width=389,height=214!
> It's been a bit of a long investigation, but two clues ended up standing out:
> 1) An abnormal number of expired messages on 4.0 (as shown in the attached
> system log), while 3.11 has almost none.
> 2) An abnormal GC activity (as shown in the attached gc log).
> Turns out the two are related, as the [on expired
> callback|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundConnection.java#L462]
> creates a huge amount of strings in the {{id()}} call. The next question is
> what causes all those message expirations; we thoroughly reviewed the
> internode messaging code and the only issue we could find so far is related
> to the "batch pruning" calls
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L81]
> and
> [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L188]:
> it _seems_ too much time is spent on those, causing the event loop to fall
> behind in processing the rest of the messages, which will end up being
> expired. This is supported by the analysis of the collapsed stacks (after
> fixing the GC issue):
> {noformat}
> (tprint (top-aggregated-calls oss40nogc "EventLoopDelivery:doRun" 5))
> org/apache/cassandra/net/OutboundConnection$EventLoopDelivery:doRun 3456
> org/apache/cassandra/net/OutboundMessageQueue:access$600 1621
> org/apache/cassandra/net/PrunableArrayQueue:prune 1621
> org/apache/cassandra/net/OutboundMessageQueue$WithLock:close 1621
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1620
> {noformat}
> Those are the top 5 sampled calls from {{EventLoopDelivery#doRun()}} which
> spends half of its time pruning. But only a tiny portion of such pruning time
> is spent actually expiring:
> {noformat}
> (tprint (top-aggregated-calls oss40nogc
> "OutboundMessageQueue:pruneInternalQueueWithLock" 5))
> org/apache/cassandra/net/OutboundMessageQueue:pruneInternalQueueWithLock 1900
> org/apache/cassandra/net/PrunableArrayQueue:prune 1894
> org/apache/cassandra/net/OutboundMessageQueue$1Pruner:onPruned 147
> org/apache/cassandra/net/OutboundConnection$$Lambda$444/740904487:accept 147
> org/apache/cassandra/net/OutboundConnection:onExpired 147
> {noformat}
> And indeed, the {{PrunableArrayQueue:prune()}} self time is dominant:
> {noformat}
> (tprint (top-self-calls oss40nogc "PrunableArrayQueue:prune" 5))
> org/apache/cassandra/net/PrunableArrayQueue:prune 1718
> org/apache/cassandra/net/OutboundConnection:releaseCapacity 27
> java/util/concurrent/ConcurrentHashMap:replaceNode 19
> java/util/concurrent/ConcurrentLinkedQueue:offer 16
> java/util/concurrent/LinkedBlockingQueue:offer 15
> {noformat}
> That said, before proceeding with a PR to fix those issues, I'd like to
> understand: what's the reason to prune so often, rather than just when
> polling the message during delivery? If there's a reason I'm missing, let's
> talk about how to optimize pruning, otherwise let's get rid of that.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079224#comment-17079224
]
Eduard Tudenhoefner commented on CASSANDRA-15711:
-
Tests on Jenkins
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch/46/]
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079258#comment-17079258
]
Sergio Bossa commented on CASSANDRA-15667:
--
Thanks [~e.dimitrova] for chiming in.
{quote}From what I recall the bootstrap was sometimes completing too fast
before the streaming is really interrupted from the byteman code and we didn't
really have an instrument to control that.
{quote}
I went through the "resumable bootstrap" test, and unfortunately I don't see
how the bootstrap could ever complete before the byteman script is invoked:
this is because such script [makes node 1 fail before it starts to stream
files|[https://github.com/ekaterinadimitrova2/cassandra-dtest/blob/b56887d67c353d6d69cd60cfd74859405fa37685/byteman/4.0/stream_failure.btm#L10]],
which means there's no way for node 3 to finish bootstrapping before it
received all files from both nodes, which will never happen due to said script
causing node 1 to fail.
So why did the test fail?
I believe that's because of this issue: in other words, node 3 was correctly
seeing its streaming session completed (after node 1 finished streaming with an
error) but *not* failed; this is because the "completed" state is read through
the actual session state, while the "failed" state is read through the
{{SessionInfo}} state, which is what we're fixing here.
That said, I would propose to still re-introduce the original
{{resumable_bootstrap_test}}, because it's an important enough feature to
deserve its own test, and it uses 3 nodes which increases the chances of
detecting errors/races.
Thoughts?
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15708) Fix in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079259#comment-17079259 ] Marcus Eriksson commented on CASSANDRA-15708: - tests for all branches: || unit tests || jvm dtest || jvm upgrade tests|| | [2.2|https://circleci.com/gh/krummas/cassandra/3120] [3.0|https://circleci.com/gh/krummas/cassandra/3117] [3.11|https://circleci.com/gh/krummas/cassandra/3115] [trunk|https://circleci.com/gh/krummas/cassandra/3123] | [2.2|https://circleci.com/gh/krummas/cassandra/3119] [3.0|https://circleci.com/gh/krummas/cassandra/3118] [3.11|https://circleci.com/gh/krummas/cassandra/3116] [trunk|https://circleci.com/gh/krummas/cassandra/3122] | [3.0|https://circleci.com/gh/krummas/cassandra/3131] [3.11|https://circleci.com/gh/krummas/cassandra/3132] [trunk|https://circleci.com/gh/krummas/cassandra/3136] > Fix in-jvm upgrade dtests > - > > Key: CASSANDRA-15708 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15708 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > In-jvm upgrade dtests were broken by CASSANDRA-15539 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Fix cqlsh copy tests on older branches by removing the dependency from the cqlshlib library
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new da8abe3 Fix cqlsh copy tests on older branches by removing the
dependency from the cqlshlib library
da8abe3 is described below
commit da8abe3cab3fc186a6cfb2e3771f647a0dac120e
Author: Stefania Alborghetti
AuthorDate: Fri Apr 3 16:35:46 2020 -0400
Fix cqlsh copy tests on older branches by removing the dependency from the
cqlshlib library
patch by Stefania Alborghetti; reviewed by Aleksandr Sorokoumov for
CASSANDRA-14050
---
cqlsh_tests/cqlsh_test_types.py | 54 +++---
cqlsh_tests/test_cqlsh_copy.py | 368 +++-
2 files changed, 160 insertions(+), 262 deletions(-)
diff --git a/cqlsh_tests/cqlsh_test_types.py b/cqlsh_tests/cqlsh_test_types.py
index d78864e..c58879e 100644
--- a/cqlsh_tests/cqlsh_test_types.py
+++ b/cqlsh_tests/cqlsh_test_types.py
@@ -1,29 +1,12 @@
import datetime
-import sys
+import logging
+import re
from collections import namedtuple
-from contextlib import contextmanager
from cassandra.util import SortedSet
-
-@contextmanager
-def _cqlshlib(cqlshlib_path):
-"""
-Returns the cqlshlib module found at the specified path.
-"""
-# This method accomplishes its goal by manually adding the library to
-# sys.path, returning the module, then restoring the old path once the
-# context manager exits. This isn't great for maintainability and should
-# be replaced if cqlshlib is made easier to interact with.
-saved_path = list(sys.path)
-
-try:
-sys.path = sys.path + [cqlshlib_path]
-import cqlshlib
-yield cqlshlib
-finally:
-sys.path = saved_path
+logger = logging.getLogger(__name__)
def maybe_quote(s):
@@ -52,21 +35,33 @@ class Address(namedtuple('Address', ('name', 'number',
'street', 'phones'))):
phones_str)
+def drop_microseconds(val):
+"""
+For COPY TO, we need to round microsecond to milliseconds because server
side
+TimestampSerializer.dateStringPatterns only parses milliseconds. If we
keep microseconds,
+users may try to import with COPY FROM a file generated with COPY TO and
have problems if
+prepared statements are disabled, see CASSANDRA-11631.
+"""
+def drop_micros(m):
+return m.group(0)[:12] + '+'
+
+# Matches H:MM:SS.00+ and drops the last 3 digits before the +
+ret = re.sub('\\d{2}\\:\\d{2}\\:\\d{2}\\.(\\d{6})\\+', drop_micros, val)
+logger.debug("Rounded microseconds: {} -> {}".format(val, ret))
+return ret
+
+
class Datetime(datetime.datetime):
"""
Extend standard datetime.datetime class with cql formatting.
This could be cleaner if this class was declared inside TestCqlshCopy, but
then pickle
wouldn't have access to the class.
"""
-def __new__(cls, year, month, day, hour=0, minute=0, second=0,
microsecond=0, tzinfo=None, cqlshlib_path=None):
+def __new__(cls, year, month, day, hour=0, minute=0, second=0,
microsecond=0, tzinfo=None,
+time_format='%Y-%m-%d %H:%M:%S%z', round_timestamp=True):
self = datetime.datetime.__new__(cls, year, month, day, hour, minute,
second, microsecond, tzinfo)
-if (cqlshlib_path is not None):
-with _cqlshlib(cqlshlib_path) as cqlshlib:
-from cqlshlib.formatting import DEFAULT_TIMESTAMP_FORMAT,
round_microseconds
-self.default_time_format = DEFAULT_TIMESTAMP_FORMAT
-self.round_microseconds = round_microseconds
-else:
-self.default_time_format = '%Y-%m-%d %H:%M:%S%z'
+self.default_time_format = time_format
+self.round_timestamp = round_timestamp
return self
def __repr__(self):
@@ -77,8 +72,7 @@ class Datetime(datetime.datetime):
def _format_for_csv(self):
ret = self.strftime(self.default_time_format)
-return self.round_microseconds(ret) if self.round_microseconds else ret
-
+return drop_microseconds(ret) if self.round_timestamp else ret
class ImmutableDict(frozenset):
"""
diff --git a/cqlsh_tests/test_cqlsh_copy.py b/cqlsh_tests/test_cqlsh_copy.py
index d25c3f2..76805e0 100644
--- a/cqlsh_tests/test_cqlsh_copy.py
+++ b/cqlsh_tests/test_cqlsh_copy.py
@@ -29,7 +29,7 @@ from cassandra.util import SortedSet
from ccmlib.common import is_win
from .cqlsh_test_types import (Address, Datetime, ImmutableDict,
- ImmutableSet, Name, UTC)
+ ImmutableSet, Name, UTC, drop_microseconds)
from .cqlsh_tools import (DummyColorMap, assert_csvs_items_equal,
csv_rows, monkeypatch_driver, random_list,
unmonkeypat
[jira] [Updated] (CASSANDRA-14050) Many cqlsh_copy_tests are busted
[
https://issues.apache.org/jira/browse/CASSANDRA-14050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania Alborghetti updated CASSANDRA-14050:
-
Status: Ready to Commit (was: Review In Progress)
> Many cqlsh_copy_tests are busted
>
>
> Key: CASSANDRA-14050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14050
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Michael Kjellman
>Assignee: Stefania Alborghetti
>Priority: Normal
>
> Many cqlsh_copy_tests are busted. We should disable the entire suite until
> this is resolved as these tests are currently nothing but a waste of time.
> test_bulk_round_trip_blogposts - cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> test_bulk_round_trip_blogposts_with_max_connections -
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> test_bulk_round_trip_default - cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> Error starting node3.
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-S9NfIH
> dtest: DEBUG: Done setting configuration options:
> { 'initial_token': None,
> 'memtable_allocation_type': 'offheap_objects',
> 'num_tokens': '256',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> - >> end captured logging << -
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/cassandra/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
> line 2546, in test_bulk_round_trip_blogposts
> stress_table='stresscql.blogposts')
> File "/home/cassandra/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
> line 2451, in _test_bulk_round_trip
> self.prepare(nodes=nodes, partitioner=partitioner,
> configuration_options=configuration_options)
> File "/home/cassandra/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
> line 115, in prepare
> self.cluster.populate(nodes,
> tokens=tokens).start(wait_for_binary_proto=True)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/cluster.py",
> line 423, in start
> raise NodeError("Error starting {0}.".format(node.name), p)
> "Error starting node3.\n >> begin captured logging <<
> \ndtest: DEBUG: cluster ccm directory:
> /tmp/dtest-S9NfIH\ndtest: DEBUG: Done setting configuration options:\n{
> 'initial_token': None,\n'memtable_allocation_type': 'offheap_objects',\n
> 'num_tokens': '256',\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms':
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms':
> 1}\n- >> end captured logging <<
> -"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14050) Many cqlsh_copy_tests are busted
[
https://issues.apache.org/jira/browse/CASSANDRA-14050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania Alborghetti updated CASSANDRA-14050:
-
Fix Version/s: 4.0-alpha
Since Version: 4.x
Source Control Link:
https://github.com/apache/cassandra-dtest/commit/da8abe3cab3fc186a6cfb2e3771f647a0dac120e
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Many cqlsh_copy_tests are busted
>
>
> Key: CASSANDRA-14050
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14050
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Testing
>Reporter: Michael Kjellman
>Assignee: Stefania Alborghetti
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Many cqlsh_copy_tests are busted. We should disable the entire suite until
> this is resolved as these tests are currently nothing but a waste of time.
> test_bulk_round_trip_blogposts - cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> test_bulk_round_trip_blogposts_with_max_connections -
> cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> test_bulk_round_trip_default - cqlsh_tests.cqlsh_copy_tests.CqlshCopyTest
> Error starting node3.
> >> begin captured logging <<
> dtest: DEBUG: cluster ccm directory: /tmp/dtest-S9NfIH
> dtest: DEBUG: Done setting configuration options:
> { 'initial_token': None,
> 'memtable_allocation_type': 'offheap_objects',
> 'num_tokens': '256',
> 'phi_convict_threshold': 5,
> 'range_request_timeout_in_ms': 1,
> 'read_request_timeout_in_ms': 1,
> 'request_timeout_in_ms': 1,
> 'truncate_request_timeout_in_ms': 1,
> 'write_request_timeout_in_ms': 1}
> - >> end captured logging << -
> File "/usr/lib/python2.7/unittest/case.py", line 329, in run
> testMethod()
> File "/home/cassandra/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
> line 2546, in test_bulk_round_trip_blogposts
> stress_table='stresscql.blogposts')
> File "/home/cassandra/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
> line 2451, in _test_bulk_round_trip
> self.prepare(nodes=nodes, partitioner=partitioner,
> configuration_options=configuration_options)
> File "/home/cassandra/cassandra-dtest/cqlsh_tests/cqlsh_copy_tests.py",
> line 115, in prepare
> self.cluster.populate(nodes,
> tokens=tokens).start(wait_for_binary_proto=True)
> File
> "/home/cassandra/env/local/lib/python2.7/site-packages/ccmlib/cluster.py",
> line 423, in start
> raise NodeError("Error starting {0}.".format(node.name), p)
> "Error starting node3.\n >> begin captured logging <<
> \ndtest: DEBUG: cluster ccm directory:
> /tmp/dtest-S9NfIH\ndtest: DEBUG: Done setting configuration options:\n{
> 'initial_token': None,\n'memtable_allocation_type': 'offheap_objects',\n
> 'num_tokens': '256',\n'phi_convict_threshold': 5,\n
> 'range_request_timeout_in_ms': 1,\n'read_request_timeout_in_ms':
> 1,\n'request_timeout_in_ms': 1,\n
> 'truncate_request_timeout_in_ms': 1,\n'write_request_timeout_in_ms':
> 1}\n- >> end captured logging <<
> -"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15707) Fix cqlsh output test
[
https://issues.apache.org/jira/browse/CASSANDRA-15707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15707:
---
Status: Ready to Commit (was: Review In Progress)
> Fix cqlsh output test
> -
>
> Key: CASSANDRA-15707
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15707
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> https://ci-cassandra.apache.org/view/branches/job/Cassandra-trunk/45/testReport/cqlshlib.test.test_cqlsh_output/TestCqlshOutput/
> {code}
> Sequences differ: ['CRE[438 chars]nt', ") WITH additional_write_policy =
> '99p'",[710 chars], ''] != ['CRE[438 chars]nt', ') WITH
> bloom_filter_fp_chance = 0.01', "[711 chars], '']
> First differing element 17:
> ") WITH additional_write_policy = '99p'"
> ') WITH bloom_filter_fp_chance = 0.01'
> Diff is 1475 characters long. Set self.maxDiff to None to see it.
> """Fail immediately, with the given message."""
> >> raise self.failureException('Sequences differ: [\'CRE[438 chars]nt\', ")
> >> WITH additional_write_policy = \'99p\'",[710 chars], \'\'] != [\'CRE[438
> >> chars]nt\', \') WITH bloom_filter_fp_chance = 0.01\', "[711 chars],
> >> \'\']\n\nFirst differing element 17:\n") WITH additional_write_policy =
> >> \'99p\'"\n\') WITH bloom_filter_fp_chance = 0.01\'\n\nDiff is 1475
> >> characters long. Set self.maxDiff to None to see it.')
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/02: ninja fix CHANGES.txt
This is an automated email from the ASF dual-hosted git repository. blerer pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 0d7cd7584e366ca11f651bf52f4d2104256fee14 Author: Benjamin Lerer AuthorDate: Thu Apr 9 14:50:59 2020 +0200 ninja fix CHANGES.txt --- CHANGES.txt | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGES.txt b/CHANGES.txt index ac0fe22..6fd60ed 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,4 +1,5 @@ 4.0-alpha4 + * Mark system_views/system_virtual_schema as system keyspaces in cqlsh (CASSANDRA-15706) * Avoid unnecessary collection/iterator allocations during btree construction (CASSANDRA-15390) * Repair history tables should have TTL and TWCS (CASSANDRA-12701) * Fix cqlsh erroring out on Python 3.7 due to webbrowser module being absent (CASSANDRA-15572) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (b3f2738 -> 3db7796)
This is an automated email from the ASF dual-hosted git repository. blerer pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from b3f2738 Mark system_views/system_virtual_schema as system keyspaces in cqlsh new 0d7cd75 ninja fix CHANGES.txt new 3db7796 Fix cqlsh output test The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + pylib/cqlshlib/test/test_cqlsh_output.py | 13 +++-- 2 files changed, 8 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 02/02: Fix cqlsh output test
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 3db7796c1b7aa0e953fab621c0334c576cc7da35
Author: Eduard Tudenhoefner
AuthorDate: Wed Apr 8 12:37:42 2020 +0200
Fix cqlsh output test
patch by Eduard Tudenhoefner; reviewed by Michael Semb Wever for
CASSANDRA-15707
---
pylib/cqlshlib/test/test_cqlsh_output.py | 13 +++--
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git a/pylib/cqlshlib/test/test_cqlsh_output.py
b/pylib/cqlshlib/test/test_cqlsh_output.py
index d2ee800..1182bd6 100644
--- a/pylib/cqlshlib/test/test_cqlsh_output.py
+++ b/pylib/cqlshlib/test/test_cqlsh_output.py
@@ -657,20 +657,20 @@ class TestCqlshOutput(BaseTestCase):
uuidcol uuid,
varcharcol text,
varintcol varint
-) WITH bloom_filter_fp_chance = 0.01
+) WITH additional_write_policy = '99p'
+AND bloom_filter_fp_chance = 0.01
AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
AND cdc = false
AND comment = ''
AND compaction = {'class':
'org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy',
'max_threshold': '32', 'min_threshold': '4'}
AND compression = {'chunk_length_in_kb': '16', 'class':
'org.apache.cassandra.io.compress.LZ4Compressor'}
AND crc_check_chance = 1.0
-AND dclocal_read_repair_chance = 0.0
AND default_time_to_live = 0
AND gc_grace_seconds = 864000
AND max_index_interval = 2048
AND memtable_flush_period_in_ms = 0
AND min_index_interval = 128
-AND read_repair_chance = 0.0
+AND read_repair = 'BLOCKING'
AND speculative_retry = '99p';
""" % quote_name(get_keyspace()))
@@ -762,10 +762,11 @@ class TestCqlshOutput(BaseTestCase):
for semicolon in ('', ';'):
output = c.cmd_and_response('desc full schema' + semicolon)
self.assertNoHasColors(output)
-self.assertRegex(output, '^\nCREATE KEYSPACE')
-self.assertIn("\nCREATE KEYSPACE system WITH replication =
{'class': 'LocalStrategy'} AND durable_writes = true;\n",
+# Since CASSANDRA-7622 'DESC FULL SCHEMA' also shows all
VIRTUAL keyspaces
+self.assertIn('VIRTUAL KEYSPACE system_virtual_schema', output)
+self.assertIn("\nCREATE KEYSPACE system_auth WITH replication
= {'class': 'SimpleStrategy', 'replication_factor': '1'} AND durable_writes =
true;\n",
output)
-self.assertRegex(output, ';\s*$')
+self.assertRegex(output, '.*\s*$')
def test_show_output(self):
with testrun_cqlsh(tty=True, env=self.default_env) as c:
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15707) Fix cqlsh output test
[
https://issues.apache.org/jira/browse/CASSANDRA-15707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15707:
---
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/3db7796c1b7aa0e953fab621c0334c576cc7da35
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed into trunk at 3db7796c1b7aa0e953fab621c0334c576cc7da35
> Fix cqlsh output test
> -
>
> Key: CASSANDRA-15707
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15707
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Eduard Tudenhoefner
>Assignee: Eduard Tudenhoefner
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> https://ci-cassandra.apache.org/view/branches/job/Cassandra-trunk/45/testReport/cqlshlib.test.test_cqlsh_output/TestCqlshOutput/
> {code}
> Sequences differ: ['CRE[438 chars]nt', ") WITH additional_write_policy =
> '99p'",[710 chars], ''] != ['CRE[438 chars]nt', ') WITH
> bloom_filter_fp_chance = 0.01', "[711 chars], '']
> First differing element 17:
> ") WITH additional_write_policy = '99p'"
> ') WITH bloom_filter_fp_chance = 0.01'
> Diff is 1475 characters long. Set self.maxDiff to None to see it.
> """Fail immediately, with the given message."""
> >> raise self.failureException('Sequences differ: [\'CRE[438 chars]nt\', ")
> >> WITH additional_write_policy = \'99p\'",[710 chars], \'\'] != [\'CRE[438
> >> chars]nt\', \') WITH bloom_filter_fp_chance = 0.01\', "[711 chars],
> >> \'\']\n\nFirst differing element 17:\n") WITH additional_write_policy =
> >> \'99p\'"\n\') WITH bloom_filter_fp_chance = 0.01\'\n\nDiff is 1475
> >> characters long. Set self.maxDiff to None to see it.')
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14773) Overflow of 32-bit integer during compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-14773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079309#comment-17079309
]
Benjamin Lerer commented on CASSANDRA-14773:
Jenkins test failures seems unrelated
[Jenkins|https://ci-cassandra.apache.org/job/Cassandra-devbranch/31/]
and the circleCI test looks good.
> Overflow of 32-bit integer during compaction.
> -
>
> Key: CASSANDRA-14773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14773
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Vladimir Bukhtoyarov
>Assignee: Francisco Fernandez
>Priority: Urgent
> Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In scope of CASSANDRA-13444 the compaction was significantly improved from
> CPU and memory perspective. Hovewer this improvement introduces the bug in
> rounding. When rounding the expriration time which is close to
> *Cell.MAX_DELETION_TIME*(it is just *Integer.MAX_VALUE*) the math overflow
> happens(because in scope of -CASSANDRA-13444-) data type for point was
> changed from Long to Integer in order to reduce memory footprint), as result
> point became negative and acts as silent poison for internal structures of
> StreamingTombstoneHistogramBuilder like *DistanceHolder* and *DataHolder*.
> Then depending of point intervals:
> * The TombstoneHistogram produces wrong values when interval of points is
> less then binSize, it is not critical.
> * Compaction crashes with ArrayIndexOutOfBoundsException if amount of point
> intervals is great then binSize, this case is very critical.
>
> This is pull request [https://github.com/apache/cassandra/pull/273] that
> reproduces the issue and provides the fix.
>
> The stacktrace when running(on codebase without fix)
> *testMathOverflowDuringRoundingOfLargeTimestamp* without -ea JVM flag
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$DistanceHolder.add(StreamingTombstoneHistogramBuilder.java:208)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushValue(StreamingTombstoneHistogramBuilder.java:140)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$$Lambda$1/1967205423.consume(Unknown
> Source)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$Spool.forEach(StreamingTombstoneHistogramBuilder.java:574)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushHistogram(StreamingTombstoneHistogramBuilder.java:124)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.build(StreamingTombstoneHistogramBuilder.java:184)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilderTest.testMathOverflowDuringRoundingOfLargeTimestamp(StreamingTombstoneHistogramBuilderTest.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij
[cassandra] branch cassandra-3.11 updated (912c6c4 -> 57b0bb9)
This is an automated email from the ASF dual-hosted git repository. stefania pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 912c6c4 Merge branch 'cassandra-3.0' into cassandra-3.11 new f3568c0 Fix parse error in cqlsh COPY FROM and formatting for map of blobs new c8081c2 Merge branch 'cassandra-2.1' into cassandra-2.2 new fe263ad Merge branch 'cassandra-2.2' into cassandra-3.0 new 57b0bb9 Merge branch 'cassandra-3.0' into cassandra-3.11 The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 ++ pylib/cqlshlib/copyutil.py | 4 ++-- pylib/cqlshlib/formatting.py | 9 - 3 files changed, 12 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (e3f54d4 -> fe263ad)
This is an automated email from the ASF dual-hosted git repository. stefania pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from e3f54d4 Memtable memory allocations may deadlock new f3568c0 Fix parse error in cqlsh COPY FROM and formatting for map of blobs new c8081c2 Merge branch 'cassandra-2.1' into cassandra-2.2 new fe263ad Merge branch 'cassandra-2.2' into cassandra-3.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 ++ pylib/cqlshlib/copyutil.py | 4 ++-- pylib/cqlshlib/formatting.py | 9 - 3 files changed, 12 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. stefania pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit fe263ad9e98c1d43eef74b88fcb111b697f619ef Merge: e3f54d4 c8081c2 Author: Stefania Alborghetti AuthorDate: Thu Apr 9 08:58:55 2020 -0400 Merge branch 'cassandra-2.2' into cassandra-3.0 CHANGES.txt | 2 ++ pylib/cqlshlib/copyutil.py | 4 ++-- pylib/cqlshlib/formatting.py | 9 - 3 files changed, 12 insertions(+), 3 deletions(-) diff --cc CHANGES.txt index 2d14e34,56dd315..eb0b66e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,8 -1,9 +1,10 @@@ -2.2.18 +3.0.21 + * Memtable memory allocations may deadlock (CASSANDRA-15367) + * Run evictFromMembership in GossipStage (CASSANDRA-15592) +Merged from 2.2: * Disable JMX rebinding (CASSANDRA-15653) + Merged from 2.1: + * Fix parse error in cqlsh COPY FROM and formatting for map of blobs (CASSANDRA-15679) - -2.2.17 * Fix Commit log replays when static column clustering keys are collections (CASSANDRA-14365) * Fix Red Hat init script on newer systemd versions (CASSANDRA-15273) * Allow EXTRA_CLASSPATH to work on tar/source installations (CASSANDRA-15567) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (3db7796 -> 2f78181)
This is an automated email from the ASF dual-hosted git repository. stefania pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 3db7796 Fix cqlsh output test new f3568c0 Fix parse error in cqlsh COPY FROM and formatting for map of blobs new c8081c2 Merge branch 'cassandra-2.1' into cassandra-2.2 new fe263ad Merge branch 'cassandra-2.2' into cassandra-3.0 new 57b0bb9 Merge branch 'cassandra-3.0' into cassandra-3.11 new 2f78181 Merge branch 'cassandra-3.11' into trunk The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 ++ pylib/cqlshlib/copyutil.py | 7 +-- pylib/cqlshlib/formatting.py | 9 - 3 files changed, 15 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated (3940a9e -> c8081c2)
This is an automated email from the ASF dual-hosted git repository. stefania pushed a change to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 3940a9e Jenkins 'Cassandra' label applied to the declarative pipeline new f3568c0 Fix parse error in cqlsh COPY FROM and formatting for map of blobs new c8081c2 Merge branch 'cassandra-2.1' into cassandra-2.2 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 2 ++ pylib/cqlshlib/copyutil.py | 4 ++-- pylib/cqlshlib/formatting.py | 9 - 3 files changed, 12 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.1' into cassandra-2.2
This is an automated email from the ASF dual-hosted git repository.
stefania pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit c8081c2467cfa34efec5e44c7dd31b7a99db5945
Merge: 3940a9e f3568c0
Author: Stefania Alborghetti
AuthorDate: Thu Apr 9 08:56:38 2020 -0400
Merge branch 'cassandra-2.1' into cassandra-2.2
CHANGES.txt | 2 ++
pylib/cqlshlib/copyutil.py | 4 ++--
pylib/cqlshlib/formatting.py | 9 -
3 files changed, 12 insertions(+), 3 deletions(-)
diff --cc CHANGES.txt
index 885d33e,d0d0993..56dd315
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,55 -1,7 +1,57 @@@
-2.1.21
- * Fix parse error in cqlsh COPY FROM and formatting for map of blobs
(CASSANDRA-15679)
+2.2.18
+ * Disable JMX rebinding (CASSANDRA-15653)
++Merged from 2.1:
++ * Fix parse error in cqlsh COPY FROM and formatting for map of blobs
(CASSANDRA-15679)
+
+2.2.17
+ * Fix Commit log replays when static column clustering keys are collections
(CASSANDRA-14365)
+ * Fix Red Hat init script on newer systemd versions (CASSANDRA-15273)
+ * Allow EXTRA_CLASSPATH to work on tar/source installations (CASSANDRA-15567)
+
+2.2.16
+ * Fix SELECT JSON output for empty blobs (CASSANDRA-15435)
+ * In-JVM DTest: Set correct internode message version for upgrade test
(CASSANDRA-15371)
+ * In-JVM DTest: Support NodeTool in dtest
+
+2.2.15
+ * Catch non-IOException in FileUtils.close to make sure that all resources
are closed (CASSANDRA-15225)
+ * Handle exceptions during authentication/authorization (CASSANDRA-15041)
+ * Fix JDK7 compatibility broken in cassandra-2.2 (CASSANDRA-15050)
+ * Support cross version messaging in in-jvm upgrade dtests (CASSANDRA-15078)
+ * Fix index summary redistribution cancellation (CASSANDRA-15045)
+ * Refactor Circle CI configuration (CASSANDRA-14806)
+ * Fixing invalid CQL in security documentation (CASSANDRA-15020)
+ * Make tools/bin/token-generator py2/3 compatible (CASSANDRA-15012)
+ * Multi-version in-JVM dtests (CASSANDRA-14937)
+ * Allow instance class loaders to be garbage collected for inJVM dtest
(CASSANDRA-15170)
+ * Add support for network topology and query tracing for inJVM dtest
(CASSANDRA-15319)
+
+
+2.2.14
+ * CircleCI docker image should bake in more dependencies (CASSANDRA-14985)
+ * Don't enable client transports when bootstrap is pending (CASSANDRA-14525)
+ * MigrationManager attempts to pull schema from different major version
nodes (CASSANDRA-14928)
+ * Don't skip entire sstables when reading backwards with mixed clustering
column order
+ (CASSANDRA-14910)
+ * Cannot perform slice reads in reverse direction against tables with
clustering columns
+ in mixed order (CASSANDRA-14899)
+ * Fix incorrect cqlsh results when selecting same columns multiple times
(CASSANDRA-13262)
+ * Returns null instead of NaN or Infinity in JSON strings (CASSANDRA-14377)
+Merged from 2.1:
* Paged Range Slice queries with DISTINCT can drop rows from results
(CASSANDRA-14956)
* Update release checksum algorithms to SHA-256, SHA-512 (CASSANDRA-14970)
+
+
+2.2.13
+ * Fix bug that prevented compaction of SSTables after full repairs
(CASSANDRA-14423)
+ * Incorrect counting of pending messages in OutboundTcpConnection
(CASSANDRA-11551)
+ * Fix compaction failure caused by reading un-flushed data (CASSANDRA-12743)
+ * Use Bounds instead of Range for sstables in anticompaction
(CASSANDRA-14411)
+ * Fix JSON queries with IN restrictions and ORDER BY clause (CASSANDRA-14286)
+ * CQL fromJson(null) throws NullPointerException (CASSANDRA-13891)
+ * Fix query pager DEBUG log leak causing hit in paged reads throughput
(CASSANDRA-14318)
+ * Backport circleci yaml (CASSANDRA-14240)
+Merged from 2.1:
* Check checksum before decompressing data (CASSANDRA-14284)
* CVE-2017-5929 Security vulnerability in Logback warning in NEWS.txt
(CASSANDRA-14183)
diff --cc pylib/cqlshlib/copyutil.py
index c9c5829,a8bd0a2..b28d96d
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@@ -52,7 -52,7 +52,7 @@@ from cassandra.util import Date, Tim
from cql3handling import CqlRuleSet
from displaying import NO_COLOR_MAP
- from formatting import format_value_default, DateTimeFormat, EMPTY,
get_formatter
-from formatting import format_value_default, EMPTY, get_formatter, BlobType
++from formatting import format_value_default, DateTimeFormat, EMPTY,
get_formatter, BlobType
from sslhandling import ssl_settings
PROFILE_ON = False
@@@ -1810,7 -1726,7 +1810,7 @@@ class ImportConversion(object)
return converters.get(t.typename, convert_unknown)(v, ct=t)
def convert_blob(v, **_):
- return bytearray.fromhex(v[2:])
-return BlobType(unicode(v[2:]).decode("hex"))
++return BlobType(v[2:].decode("hex"))
def convert_text(v, **_):
return v
diff --cc pylib/cqlshlib/formatting.py
index f88fc5
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository.
stefania pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 2f781819b184ba306ed0cc4412441a3c444365d6
Merge: 3db7796 57b0bb9
Author: Stefania Alborghetti
AuthorDate: Thu Apr 9 09:05:14 2020 -0400
Merge branch 'cassandra-3.11' into trunk
CHANGES.txt | 2 ++
pylib/cqlshlib/copyutil.py | 7 +--
pylib/cqlshlib/formatting.py | 9 -
3 files changed, 15 insertions(+), 3 deletions(-)
diff --cc pylib/cqlshlib/copyutil.py
index 3bed2ad,b91bb76..853893c
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@@ -56,12 -50,11 +56,12 @@@ from cassandra.metadata import protect_
from cassandra.policies import RetryPolicy, WhiteListRoundRobinPolicy,
DCAwareRoundRobinPolicy, FallthroughRetryPolicy
from cassandra.query import BatchStatement, BatchType, SimpleStatement,
tuple_factory
from cassandra.util import Date, Time
+from cqlshlib.util import profile_on, profile_off
-from cql3handling import CqlRuleSet
-from displaying import NO_COLOR_MAP
-from formatting import format_value_default, CqlType, DateTimeFormat, EMPTY,
get_formatter, BlobType
-from sslhandling import ssl_settings
+from cqlshlib.cql3handling import CqlRuleSet
+from cqlshlib.displaying import NO_COLOR_MAP
- from cqlshlib.formatting import format_value_default, CqlType,
DateTimeFormat, EMPTY, get_formatter
++from cqlshlib.formatting import format_value_default, CqlType,
DateTimeFormat, EMPTY, get_formatter, BlobType
+from cqlshlib.sslhandling import ssl_settings
PROFILE_ON = False
STRACE_ON = False
@@@ -1933,10 -1868,10 +1933,13 @@@ class ImportConversion(object)
return converters.get(t.typename, convert_unknown)(v, ct=t)
def convert_blob(v, **_):
- return bytearray.fromhex(v[2:])
-return BlobType(v[2:].decode("hex"))
++if sys.version_info.major >= 3:
++return bytes.fromhex(v[2:])
++else:
++return BlobType(v[2:].decode("hex"))
def convert_text(v, **_):
-return v
+return ensure_str(v)
def convert_uuid(v, **_):
return UUID(v)
diff --cc pylib/cqlshlib/formatting.py
index 7ed6735,9927aa1..e00d91f
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@@ -239,10 -236,16 +239,16 @@@ def formatter_for(typname)
return f
return registrator
+ class BlobType(object):
+ def __init__(self, val):
+ self.val = val
- @formatter_for('bytearray')
+ def __str__(self):
+ return str(self.val)
+
+ @formatter_for('BlobType')
def format_value_blob(val, colormap, **_):
-bval = '0x' + binascii.hexlify(val)
+bval = ensure_text('0x') + ensure_text(binascii.hexlify(val))
return colorme(bval, colormap, 'blob')
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.1 updated: Fix parse error in cqlsh COPY FROM and formatting for map of blobs
This is an automated email from the ASF dual-hosted git repository.
stefania pushed a commit to branch cassandra-2.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.1 by this push:
new f3568c0 Fix parse error in cqlsh COPY FROM and formatting for map of
blobs
f3568c0 is described below
commit f3568c0d50ac7573b53f0043b2567bde3b39bee8
Author: Aleksandr Sorokoumov
AuthorDate: Wed Apr 1 16:32:52 2020 +0200
Fix parse error in cqlsh COPY FROM and formatting for map of blobs
patch by Aleksandr Sorokoumov; reviewed by Stefania Alborghetti for
CASSANDRA-15679
---
CHANGES.txt | 1 +
pylib/cqlshlib/copyutil.py | 8 ++--
pylib/cqlshlib/formatting.py | 9 -
3 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 3582d4f..d0d0993 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
2.1.21
+ * Fix parse error in cqlsh COPY FROM and formatting for map of blobs
(CASSANDRA-15679)
* Paged Range Slice queries with DISTINCT can drop rows from results
(CASSANDRA-14956)
* Update release checksum algorithms to SHA-256, SHA-512 (CASSANDRA-14970)
* Check checksum before decompressing data (CASSANDRA-14284)
diff --git a/pylib/cqlshlib/copyutil.py b/pylib/cqlshlib/copyutil.py
index 85e2678..a8bd0a2 100644
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@ -52,7 +52,7 @@ from cassandra.util import Date, Time
from cql3handling import CqlRuleSet
from displaying import NO_COLOR_MAP
-from formatting import format_value_default, EMPTY, get_formatter
+from formatting import format_value_default, EMPTY, get_formatter, BlobType
from sslhandling import ssl_settings
PROFILE_ON = False
@@ -1726,11 +1726,7 @@ class ImportConversion(object):
return converters.get(t.typename, convert_unknown)(v, ct=t)
def convert_blob(v, **_):
-try:
-return bytearray.fromhex(v[2:])
-except TypeError:
-# Work-around for Python 2.6 bug
-return bytearray.fromhex(unicode(v[2:]))
+return BlobType(unicode(v[2:]).decode("hex"))
def convert_text(v, **_):
return v
diff --git a/pylib/cqlshlib/formatting.py b/pylib/cqlshlib/formatting.py
index eac1810..436404a 100644
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@ -126,13 +126,20 @@ def formatter_for(typname):
return f
return registrator
+class BlobType(object):
+def __init__(self, val):
+self.val = val
-@formatter_for('bytearray')
+def __str__(self):
+return str(self.val)
+
+@formatter_for('BlobType')
def format_value_blob(val, colormap, **_):
bval = '0x' + binascii.hexlify(str(val))
return colorme(bval, colormap, 'blob')
+formatter_for('bytearray')(format_value_blob)
formatter_for('buffer')(format_value_blob)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
stefania pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 57b0bb93ab4811529918300c41fbb08433f4f707
Merge: 912c6c4 fe263ad
Author: Stefania Alborghetti
AuthorDate: Thu Apr 9 09:00:58 2020 -0400
Merge branch 'cassandra-3.0' into cassandra-3.11
CHANGES.txt | 2 ++
pylib/cqlshlib/copyutil.py | 4 ++--
pylib/cqlshlib/formatting.py | 9 -
3 files changed, 12 insertions(+), 3 deletions(-)
diff --cc CHANGES.txt
index 9342825,eb0b66e..ce4775b
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -7,6 -3,8 +7,8 @@@ Merged from 3.0
* Run evictFromMembership in GossipStage (CASSANDRA-15592)
Merged from 2.2:
* Disable JMX rebinding (CASSANDRA-15653)
+ Merged from 2.1:
- * Fix parse error in cqlsh COPY FROM and formatting for map of blobs
(CASSANDRA-15679)
++ * Fix parse error in cqlsh COPY FROM and formatting for map of blobs
(CASSANDRA-15679)
* Fix Commit log replays when static column clustering keys are collections
(CASSANDRA-14365)
* Fix Red Hat init script on newer systemd versions (CASSANDRA-15273)
* Allow EXTRA_CLASSPATH to work on tar/source installations (CASSANDRA-15567)
diff --cc pylib/cqlshlib/copyutil.py
index 7f97b49,6a45153..b91bb76
--- a/pylib/cqlshlib/copyutil.py
+++ b/pylib/cqlshlib/copyutil.py
@@@ -53,7 -53,7 +53,7 @@@ from cassandra.util import Date, Tim
from cql3handling import CqlRuleSet
from displaying import NO_COLOR_MAP
- from formatting import format_value_default, CqlType, DateTimeFormat, EMPTY,
get_formatter
-from formatting import format_value_default, DateTimeFormat, EMPTY,
get_formatter, BlobType
++from formatting import format_value_default, CqlType, DateTimeFormat, EMPTY,
get_formatter, BlobType
from sslhandling import ssl_settings
PROFILE_ON = False
diff --cc pylib/cqlshlib/formatting.py
index 803ea63,53ba478..9927aa1
--- a/pylib/cqlshlib/formatting.py
+++ b/pylib/cqlshlib/formatting.py
@@@ -243,8 -160,8 +249,9 @@@ def format_value_blob(val, colormap, **
return colorme(bval, colormap, 'blob')
+ formatter_for('bytearray')(format_value_blob)
formatter_for('buffer')(format_value_blob)
+formatter_for('blob')(format_value_blob)
def format_python_formatted_type(val, colormap, color, quote=False):
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15612) Mutating sstable metadata during anticompaction fails silently on legacy sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-15612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079325#comment-17079325 ] Marcus Eriksson commented on CASSANDRA-15612: - [unit tests|https://circleci.com/gh/krummas/cassandra/3129] [jvm dtests|https://circleci.com/gh/krummas/cassandra/3130] [dtests vnodes|https://circleci.com/gh/krummas/cassandra/3133] [dtests novnodes|https://circleci.com/gh/krummas/cassandra/3134] > Mutating sstable metadata during anticompaction fails silently on legacy > sstables > - > > Key: CASSANDRA-15612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15612 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > If there is an sstable with a version without pending repair information > included in an incremental repair we silently don't set the pending repair > correctly. We should fail the IR instead. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Add cqlsh copy tests for blobs in collections
This is an automated email from the ASF dual-hosted git repository.
stefania pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new 88cc70b Add cqlsh copy tests for blobs in collections
88cc70b is described below
commit 88cc70b4d6b598c60d4b567cdceff49a76140312
Author: Aleksandr Sorokoumov
AuthorDate: Thu Apr 2 11:20:06 2020 +0200
Add cqlsh copy tests for blobs in collections
patch by Aleksandr Sorokoumov; reviewed by Stefania Alborghetti for
CASSANDRA-15679
---
cqlsh_tests/test_cqlsh_copy.py | 28
1 file changed, 20 insertions(+), 8 deletions(-)
diff --git a/cqlsh_tests/test_cqlsh_copy.py b/cqlsh_tests/test_cqlsh_copy.py
index 76805e0..286b488 100644
--- a/cqlsh_tests/test_cqlsh_copy.py
+++ b/cqlsh_tests/test_cqlsh_copy.py
@@ -260,7 +260,10 @@ class TestCqlshCopy(Tester):
u frozen>>,
v frozen,set>>,
w frozen>>,
-x map>>
+x map>>,
+y map,
+z list,
+za set
)''')
self.session.cluster.register_user_type('ks', 'name_type', Name)
@@ -302,7 +305,10 @@ class TestCqlshCopy(Tester):
# and this will cause comparison problems when comparing
with csv strings therefore failing
# some tests
ImmutableSet([ImmutableSet(['127.0.0.1']),
ImmutableSet(['127.0.0.1', '127.0.0.2'])]),
- {'key1': ['value1', 'value2']} # map>>
+ {'key1': ['value1', 'value2']}, # map>>
+ {3: bytes.fromhex('74776f')}, # y
+ [bytes.fromhex('74776f')], # z
+ {bytes.fromhex('74776f')} # za
)
def assertCsvResultEqual(self, csv_filename, results, table_name=None,
@@ -1649,12 +1655,11 @@ class TestCqlshCopy(Tester):
@jira_ticket CASSANDRA-9302
"""
-
self.all_datatypes_prepare()
insert_statement = self.session.prepare(
-"""INSERT INTO testdatatype (a, b, c, d, e, f, g, h, i, j, k, l,
m, n, o, p, q, r, s, t, u, v, w, x)
-VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?)""")
+"""INSERT INTO testdatatype (a, b, c, d, e, f, g, h, i, j, k, l,
m, n, o, p, q, r, s, t, u, v, w, x, y, z, za)
+VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?)""")
self.session.execute(insert_statement, self.data)
def _test(prepared_statements):
@@ -1691,13 +1696,20 @@ class TestCqlshCopy(Tester):
writer = csv.writer(csvfile)
# serializing blob bytearray in friendly format
data_set = list(self.data)
+
data_set[2] = self.format_blob(self.data[2])
+# Here we convert containers of blobs to strings that match
exactly the output of the SELECT *
+# because otherwise the comparison fails due to extra quotes added
by the csv writer around the blobs
+# that were converted to strings. White spaces do matter
+data_set[24] = '{3: ' + self.format_blob(self.data[24][3]) + '}'
+data_set[25] = '[' + ', '.join(self.format_blob(b) for b in
self.data[25]) + ']'
+data_set[26] = '{' + ', '.join(self.format_blob(b) for b in
self.data[26]) + '}'
writer.writerow(data_set)
def _test(prepared_statements):
logger.debug('Importing from csv file:
{name}'.format(name=tempfile.name))
out, err, _ = self.run_cqlsh(cmds="COPY ks.testdatatype FROM '{}'
WITH PREPAREDSTATEMENTS = {}"
- .format(tempfile.name, prepared_statements))
+ .format(tempfile.name, prepared_statements))
out, err, _ = self.run_cqlsh(cmds="SELECT * FROM ks.testdatatype")
results = self.parse_cqlsh_query(out=out, num_cols=len(self.data),
timestamps_to_be_rounded=[10, 17])
@@ -1725,8 +1737,8 @@ class TestCqlshCopy(Tester):
self.all_datatypes_prepare()
insert_statement = self.session.prepare(
-"""INSERT INTO testdatatype (a, b, c, d, e, f, g, h, i, j, k, l,
m, n, o, p, q, r, s, t, u, v, w, x)
-VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?)""")
+"""INSERT INTO testdatatype (a, b, c, d, e, f, g, h, i, j, k, l,
m, n, o, p, q, r, s, t, u, v, w, x, y, z, za)
+VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?, ?, ?, ?, ?)""")
self.session.execute(insert_statement, self.data)
tempfile = self.get_temp_file()
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For addit
[cassandra] branch trunk updated: Fix overflows on StreamingTombstoneHistogramBuilder produced by large deletion times.
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 00fb6d7 Fix overflows on StreamingTombstoneHistogramBuilder produced
by large deletion times.
00fb6d7 is described below
commit 00fb6d76d0a97af06ba27c1180d6dcddfa337fea
Author: Francisco Fernandez Castano
AuthorDate: Wed Mar 25 12:15:20 2020 +0100
Fix overflows on StreamingTombstoneHistogramBuilder produced by large
deletion times.
patch by Francisco Fernandez; reviewed by Benjamin Lerer and Robert Stupp
for CASSANDRA-14773
This patch:
* prevents int32-bit integer overflow
* simplifies the underlying structures of StreamingTombstoneHistogramBuilder
* avoid humongous allocations by maintaining separate arrays for tombstone
timestamps and number of
tombstone occurrences (these two were kept in the same array before)
* introduces more test coverage.
---
CHANGES.txt| 1 +
.../StreamingTombstoneHistogramBuilder.java| 437 +
.../utils/streamhist/TombstoneHistogram.java | 7 +-
.../StreamingTombstoneHistogramBuilderTest.java| 232 ++-
4 files changed, 423 insertions(+), 254 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 86a8813..46790c9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Fix overflows on StreamingTombstoneHistogramBuilder produced by large
deletion times (CASSANDRA-14773)
* Mark system_views/system_virtual_schema as system keyspaces in cqlsh
(CASSANDRA-15706)
* Avoid unnecessary collection/iterator allocations during btree construction
(CASSANDRA-15390)
* Repair history tables should have TTL and TWCS (CASSANDRA-12701)
diff --git
a/src/java/org/apache/cassandra/utils/streamhist/StreamingTombstoneHistogramBuilder.java
b/src/java/org/apache/cassandra/utils/streamhist/StreamingTombstoneHistogramBuilder.java
index 9856253..eda88bc 100755
---
a/src/java/org/apache/cassandra/utils/streamhist/StreamingTombstoneHistogramBuilder.java
+++
b/src/java/org/apache/cassandra/utils/streamhist/StreamingTombstoneHistogramBuilder.java
@@ -22,28 +22,32 @@ import java.math.RoundingMode;
import java.util.Arrays;
import java.util.stream.Collectors;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.math.IntMath;
-import static
org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.AddResult.ACCUMULATED;
-import static
org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.AddResult.INSERTED;
+import org.apache.cassandra.db.rows.Cell;
/**
* Histogram that can be constructed from streaming of data.
+ *
+ * Histogram used to retrieve the number of droppable tombstones for example
via
+ * {@link
org.apache.cassandra.io.sstable.format.SSTableReader#getDroppableTombstonesBefore(int)}.
*
- * The original algorithm is taken from following paper:
- * Yael Ben-Haim and Elad Tom-Tov, "A Streaming Parallel Decision Tree
Algorithm" (2010)
- * http://jmlr.csail.mit.edu/papers/volume11/ben-haim10a/ben-haim10a.pdf
+ * When an sstable is written (or streamed), this histogram-builder receives
the "local deletion timestamp"
+ * as an {@code int} via {@link #update(int)}. Negative values are not
supported.
*
* Algorithm: Histogram is represented as collection of {point, weight} pairs.
When new point p with weight m is added:
*
- * If point p is already exists in collection, add m to
recorded value of point p
- * If there is no point p in the collection, add point p
with weight m
- * If point was added and collection size became lorger than
maxBinSize:
- *
- * Find nearest points p1 and p2 in the collection
- * Replace theese two points with one weighted point p3 =
(p1*m1+p2*m2)/(p1+p2)
+ * If point p is already exists in collection, add m to
recorded value of point p
+ * If there is no point p in the collection, add point p
with weight m
+ * If point was added and collection size became lorger than
maxBinSize:
*
+ *
+ *
+ * Find nearest points p1 and p2 in the collection
+ * Replace theese two points with one weighted point p3 =
(p1*m1+p2*m2)/(p1+p2)
*
+ *
*
* There are some optimization to make histogram builder faster:
*
@@ -51,19 +55,19 @@ import static
org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramB
* For example, if spoolSize=100, binSize=10 and there are only 50
different points. it will be only 40 merges regardless how many points will be
added.
* Spool is organized as open-addressing primitive hash map where odd
elements are points and event elements are values.
* Spool can not resize => when number of collisions became bigger than
threashold or size became large that array_size/2 Spool is drained to
bin
-
[jira] [Updated] (CASSANDRA-15679) cqlsh COPY FROM of map of blobs fails with parse error "unhashable type: 'bytearray'"
[
https://issues.apache.org/jira/browse/CASSANDRA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania Alborghetti updated CASSANDRA-15679:
-
Status: Ready to Commit (was: Review In Progress)
> cqlsh COPY FROM of map of blobs fails with parse error "unhashable type:
> 'bytearray'"
> -
>
> Key: CASSANDRA-15679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15679
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Erick Ramirez
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 2.1.21, 2.2.17, 3.0.21, 3.11.7, 4.x
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> h2. Background
> A user was having issues loading CSV data with the {{COPY FROM}} command into
> a {{map}} column with {{blob}} values.
> h2. Replication steps
> I can easily replicate the problem with this simple table:
> {noformat}
> CREATE TABLE community.blobmaptable (
> id text PRIMARY KEY,
> blobmapcol map
> )
> {noformat}
> I have this CSV file that contains just 1 row:
> {noformat}
> $ cat blobmap.csv
> c3,{3: 0x74776f}
> {noformat}
> And here's the error when I try to load it:
> {noformat}
> cqlsh:community> COPY blobmaptable (id, blobmapcol) FROM '~/blobmap.csv' ;
> Using 1 child processes
> Starting copy of community.blobmaptable with columns [id, blobmapcol].
> Failed to import 1 rows: ParseError - Failed to parse {3: 0x74776f} :
> unhashable type: 'bytearray', given up without retries
> Failed to process 1 rows; failed rows written to
> import_community_blobmaptable.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
> 1 rows imported from 1 files in 0.389 seconds (0 skipped).
> {noformat}
> I've also logged
> [PYTHON-1234|https://datastax-oss.atlassian.net/browse/PYTHON-1234] because I
> wasn't sure if it was a Python driver issue. Cheers!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14773) Overflow of 32-bit integer during compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-14773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-14773:
---
Status: Ready to Commit (was: Review In Progress)
> Overflow of 32-bit integer during compaction.
> -
>
> Key: CASSANDRA-14773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14773
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Vladimir Bukhtoyarov
>Assignee: Francisco Fernandez
>Priority: Urgent
> Labels: pull-request-available
> Fix For: 4.0, 4.0-beta
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In scope of CASSANDRA-13444 the compaction was significantly improved from
> CPU and memory perspective. Hovewer this improvement introduces the bug in
> rounding. When rounding the expriration time which is close to
> *Cell.MAX_DELETION_TIME*(it is just *Integer.MAX_VALUE*) the math overflow
> happens(because in scope of -CASSANDRA-13444-) data type for point was
> changed from Long to Integer in order to reduce memory footprint), as result
> point became negative and acts as silent poison for internal structures of
> StreamingTombstoneHistogramBuilder like *DistanceHolder* and *DataHolder*.
> Then depending of point intervals:
> * The TombstoneHistogram produces wrong values when interval of points is
> less then binSize, it is not critical.
> * Compaction crashes with ArrayIndexOutOfBoundsException if amount of point
> intervals is great then binSize, this case is very critical.
>
> This is pull request [https://github.com/apache/cassandra/pull/273] that
> reproduces the issue and provides the fix.
>
> The stacktrace when running(on codebase without fix)
> *testMathOverflowDuringRoundingOfLargeTimestamp* without -ea JVM flag
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$DistanceHolder.add(StreamingTombstoneHistogramBuilder.java:208)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushValue(StreamingTombstoneHistogramBuilder.java:140)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$$Lambda$1/1967205423.consume(Unknown
> Source)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$Spool.forEach(StreamingTombstoneHistogramBuilder.java:574)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushHistogram(StreamingTombstoneHistogramBuilder.java:124)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.build(StreamingTombstoneHistogramBuilder.java:184)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilderTest.testMathOverflowDuringRoundingOfLargeTimestamp(StreamingTombstoneHistogramBuilderTest.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter
[jira] [Updated] (CASSANDRA-11703) To extend Cassandra CI for Linux on z(s390x).
[ https://issues.apache.org/jira/browse/CASSANDRA-11703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nayana Thorat updated CASSANDRA-11703: -- Resolution: Abandoned Status: Resolved (was: Open) > To extend Cassandra CI for Linux on z(s390x). > - > > Key: CASSANDRA-11703 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11703 > Project: Cassandra > Issue Type: New Feature > Components: Build, Packaging > Environment: Linux on z >Reporter: Meerabo Shah >Priority: Low > > Hi Team, I am not sure if this is right channel to discuss this topic but we > would like to know if Cassandra CI can be extended to support z system. > Please let us know what support will be required from IBM side (we are aware > on the H/w support). > Let me know if you want to start this discussion on some other channel. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14773) Overflow of 32-bit integer during compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-14773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-14773:
---
Fix Version/s: (was: 4.0-beta)
(was: 4.0)
4.0-alpha
Since Version: 4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/00fb6d76d0a97af06ba27c1180d6dcddfa337fea
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed into trunk at 00fb6d76d0a97af06ba27c1180d6dcddfa337fea
> Overflow of 32-bit integer during compaction.
> -
>
> Key: CASSANDRA-14773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14773
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Vladimir Bukhtoyarov
>Assignee: Francisco Fernandez
>Priority: Urgent
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In scope of CASSANDRA-13444 the compaction was significantly improved from
> CPU and memory perspective. Hovewer this improvement introduces the bug in
> rounding. When rounding the expriration time which is close to
> *Cell.MAX_DELETION_TIME*(it is just *Integer.MAX_VALUE*) the math overflow
> happens(because in scope of -CASSANDRA-13444-) data type for point was
> changed from Long to Integer in order to reduce memory footprint), as result
> point became negative and acts as silent poison for internal structures of
> StreamingTombstoneHistogramBuilder like *DistanceHolder* and *DataHolder*.
> Then depending of point intervals:
> * The TombstoneHistogram produces wrong values when interval of points is
> less then binSize, it is not critical.
> * Compaction crashes with ArrayIndexOutOfBoundsException if amount of point
> intervals is great then binSize, this case is very critical.
>
> This is pull request [https://github.com/apache/cassandra/pull/273] that
> reproduces the issue and provides the fix.
>
> The stacktrace when running(on codebase without fix)
> *testMathOverflowDuringRoundingOfLargeTimestamp* without -ea JVM flag
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$DistanceHolder.add(StreamingTombstoneHistogramBuilder.java:208)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushValue(StreamingTombstoneHistogramBuilder.java:140)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$$Lambda$1/1967205423.consume(Unknown
> Source)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$Spool.forEach(StreamingTombstoneHistogramBuilder.java:574)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushHistogram(StreamingTombstoneHistogramBuilder.java:124)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.build(StreamingTombstoneHistogramBuilder.java:184)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilderTest.testMathOverflowDuringRoundingOfLargeTimestamp(StreamingTombstoneHistogramBuilderTest.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> at
> com.
[jira] [Commented] (CASSANDRA-14773) Overflow of 32-bit integer during compaction.
[
https://issues.apache.org/jira/browse/CASSANDRA-14773?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079334#comment-17079334
]
Benjamin Lerer commented on CASSANDRA-14773:
Thanks for the patch [~fcofdezc]
> Overflow of 32-bit integer during compaction.
> -
>
> Key: CASSANDRA-14773
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14773
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction
>Reporter: Vladimir Bukhtoyarov
>Assignee: Francisco Fernandez
>Priority: Urgent
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In scope of CASSANDRA-13444 the compaction was significantly improved from
> CPU and memory perspective. Hovewer this improvement introduces the bug in
> rounding. When rounding the expriration time which is close to
> *Cell.MAX_DELETION_TIME*(it is just *Integer.MAX_VALUE*) the math overflow
> happens(because in scope of -CASSANDRA-13444-) data type for point was
> changed from Long to Integer in order to reduce memory footprint), as result
> point became negative and acts as silent poison for internal structures of
> StreamingTombstoneHistogramBuilder like *DistanceHolder* and *DataHolder*.
> Then depending of point intervals:
> * The TombstoneHistogram produces wrong values when interval of points is
> less then binSize, it is not critical.
> * Compaction crashes with ArrayIndexOutOfBoundsException if amount of point
> intervals is great then binSize, this case is very critical.
>
> This is pull request [https://github.com/apache/cassandra/pull/273] that
> reproduces the issue and provides the fix.
>
> The stacktrace when running(on codebase without fix)
> *testMathOverflowDuringRoundingOfLargeTimestamp* without -ea JVM flag
> {noformat}
> java.lang.ArrayIndexOutOfBoundsException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$DistanceHolder.add(StreamingTombstoneHistogramBuilder.java:208)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushValue(StreamingTombstoneHistogramBuilder.java:140)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$$Lambda$1/1967205423.consume(Unknown
> Source)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder$Spool.forEach(StreamingTombstoneHistogramBuilder.java:574)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.flushHistogram(StreamingTombstoneHistogramBuilder.java:124)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilder.build(StreamingTombstoneHistogramBuilder.java:184)
> at
> org.apache.cassandra.utils.streamhist.StreamingTombstoneHistogramBuilderTest.testMathOverflowDuringRoundingOfLargeTimestamp(StreamingTombstoneHistogramBuilderTest.java:183)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
> at
> com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
> at
> com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
> at com.intellij.rt.execution.juni
[jira] [Updated] (CASSANDRA-15679) cqlsh COPY FROM of map of blobs fails with parse error "unhashable type: 'bytearray'"
[
https://issues.apache.org/jira/browse/CASSANDRA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefania Alborghetti updated CASSANDRA-15679:
-
Fix Version/s: (was: 3.0.21)
(was: 2.1.21)
(was: 4.x)
4.0-alpha
2.1.x
Since Version: 2.1.21
Source Control Link:
https://gitbox.apache.org/repos/asf?p=cassandra.git;a=commit;h=f3568c0d50ac7573b53f0043b2567bde3b39bee8
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
CI runs:
[28|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/28/]
[30|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/30/]
[31|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/31/]
[34|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/34/]
[[35|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/35/|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/35/]]
Dtest changes:
[88cc70b4d6b598c60d4b567cdceff49a76140312|https://github.com/apache/cassandra-dtest/commit/88cc70b4d6b598c60d4b567cdceff49a76140312]
> cqlsh COPY FROM of map of blobs fails with parse error "unhashable type:
> 'bytearray'"
> -
>
> Key: CASSANDRA-15679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15679
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Erick Ramirez
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 2.2.17, 3.11.7, 2.1.x, 4.0-alpha
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> h2. Background
> A user was having issues loading CSV data with the {{COPY FROM}} command into
> a {{map}} column with {{blob}} values.
> h2. Replication steps
> I can easily replicate the problem with this simple table:
> {noformat}
> CREATE TABLE community.blobmaptable (
> id text PRIMARY KEY,
> blobmapcol map
> )
> {noformat}
> I have this CSV file that contains just 1 row:
> {noformat}
> $ cat blobmap.csv
> c3,{3: 0x74776f}
> {noformat}
> And here's the error when I try to load it:
> {noformat}
> cqlsh:community> COPY blobmaptable (id, blobmapcol) FROM '~/blobmap.csv' ;
> Using 1 child processes
> Starting copy of community.blobmaptable with columns [id, blobmapcol].
> Failed to import 1 rows: ParseError - Failed to parse {3: 0x74776f} :
> unhashable type: 'bytearray', given up without retries
> Failed to process 1 rows; failed rows written to
> import_community_blobmaptable.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
> 1 rows imported from 1 files in 0.389 seconds (0 skipped).
> {noformat}
> I've also logged
> [PYTHON-1234|https://datastax-oss.atlassian.net/browse/PYTHON-1234] because I
> wasn't sure if it was a Python driver issue. Cheers!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15679) cqlsh COPY FROM of map of blobs fails with parse error "unhashable type: 'bytearray'"
[
https://issues.apache.org/jira/browse/CASSANDRA-15679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079338#comment-17079338
]
Stefania Alborghetti edited comment on CASSANDRA-15679 at 4/9/20, 1:32 PM:
---
CI runs:
[28|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/28/]
[30|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/30/]
[31|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/31/]
[34|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/34/]
[35|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/35/]
Dtest changes:
[88cc70b4d6b598c60d4b567cdceff49a76140312|https://github.com/apache/cassandra-dtest/commit/88cc70b4d6b598c60d4b567cdceff49a76140312]
was (Author: stefania):
CI runs:
[28|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/28/]
[30|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/30/]
[31|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/31/]
[34|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/34/]
[[35|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/35/|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/35/]]
Dtest changes:
[88cc70b4d6b598c60d4b567cdceff49a76140312|https://github.com/apache/cassandra-dtest/commit/88cc70b4d6b598c60d4b567cdceff49a76140312]
> cqlsh COPY FROM of map of blobs fails with parse error "unhashable type:
> 'bytearray'"
> -
>
> Key: CASSANDRA-15679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15679
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Erick Ramirez
>Assignee: Aleksandr Sorokoumov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 2.2.17, 3.11.7, 2.1.x, 4.0-alpha
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> h2. Background
> A user was having issues loading CSV data with the {{COPY FROM}} command into
> a {{map}} column with {{blob}} values.
> h2. Replication steps
> I can easily replicate the problem with this simple table:
> {noformat}
> CREATE TABLE community.blobmaptable (
> id text PRIMARY KEY,
> blobmapcol map
> )
> {noformat}
> I have this CSV file that contains just 1 row:
> {noformat}
> $ cat blobmap.csv
> c3,{3: 0x74776f}
> {noformat}
> And here's the error when I try to load it:
> {noformat}
> cqlsh:community> COPY blobmaptable (id, blobmapcol) FROM '~/blobmap.csv' ;
> Using 1 child processes
> Starting copy of community.blobmaptable with columns [id, blobmapcol].
> Failed to import 1 rows: ParseError - Failed to parse {3: 0x74776f} :
> unhashable type: 'bytearray', given up without retries
> Failed to process 1 rows; failed rows written to
> import_community_blobmaptable.err
> Processed: 1 rows; Rate: 2 rows/s; Avg. rate: 3 rows/s
> 1 rows imported from 1 files in 0.389 seconds (0 skipped).
> {noformat}
> I've also logged
> [PYTHON-1234|https://datastax-oss.atlassian.net/browse/PYTHON-1234] because I
> wasn't sure if it was a Python driver issue. Cheers!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15708) Fix in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15708: Status: Ready to Commit (was: Review In Progress) > Fix in-jvm upgrade dtests > - > > Key: CASSANDRA-15708 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15708 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > In-jvm upgrade dtests were broken by CASSANDRA-15539 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15708) Fix in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079349#comment-17079349 ] Alex Petrov commented on CASSANDRA-15708: - Thank you for the patch! +1 from my side. The only (minor) suggestion is to maybe warn/log on the missing field. If we want to bring previous behaviour of failing/non failing on config fields, we shouold change `Config` API. But I'm not sure if it's worth doing. If you like suggestion, you can just do it on commit (or ignore otherwise). > Fix in-jvm upgrade dtests > - > > Key: CASSANDRA-15708 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15708 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > > In-jvm upgrade dtests were broken by CASSANDRA-15539 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079380#comment-17079380
]
Ekaterina Dimitrova commented on CASSANDRA-15667:
-
Hi Sergio,
Thanks for the patch and the broad explanation.
I just returned the test
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15667]
Unfortunately, I am not a committer so anyone else should do it. Thanks again!
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15667) StreamResultFuture check for completeness is inconsistent, leading to races
[
https://issues.apache.org/jira/browse/CASSANDRA-15667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079380#comment-17079380
]
Ekaterina Dimitrova edited comment on CASSANDRA-15667 at 4/9/20, 2:06 PM:
--
Hi Sergio,
Thanks for the patch and the broad explanation.
I just returned the test
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15667]
Unfortunately, I am not a committer so someone else should do it. Thanks again!
was (Author: e.dimitrova):
Hi Sergio,
Thanks for the patch and the broad explanation.
I just returned the test
[here|https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15667]
Unfortunately, I am not a committer so anyone else should do it. Thanks again!
> StreamResultFuture check for completeness is inconsistent, leading to races
> ---
>
> Key: CASSANDRA-15667
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15667
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Streaming and Messaging
>Reporter: Sergio Bossa
>Assignee: Massimiliano Tomassi
>Priority: Normal
> Fix For: 4.0
>
>
> {{StreamResultFuture#maybeComplete()}} uses
> {{StreamCoordinator#hasActiveSessions()}} to determine if all sessions are
> completed, but then accesses each session state via
> {{StreamCoordinator#getAllSessionInfo()}}: this is inconsistent, as the
> former relies on the actual {{StreamSession}} state, while the latter on the
> {{SessionInfo}} state, and the two are concurrently updated with no
> coordination whatsoever.
> This leads to races, i.e. apparent in some dtest spurious failures, such as
> {{TestBootstrap.resumable_bootstrap_test}} in CASSANDRA-15614 cc
> [~e.dimitrova].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15702) copyutil.py uses Exception.message which doesn't exist in Python 3
[ https://issues.apache.org/jira/browse/CASSANDRA-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefania Alborghetti updated CASSANDRA-15702: - Reviewers: Stefania Alborghetti, Stefania Alborghetti (was: Stefania Alborghetti) Stefania Alborghetti, Stefania Alborghetti Status: Review In Progress (was: Patch Available) > copyutil.py uses Exception.message which doesn't exist in Python 3 > -- > > Key: CASSANDRA-15702 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15702 > Project: Cassandra > Issue Type: Bug > Components: Tool/cqlsh >Reporter: Eduard Tudenhoefner >Assignee: Eduard Tudenhoefner >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 10m > Remaining Estimate: 0h > > Python 3 deprecated and removed the use of Exception.message. The purpose of > this ticket is to convert *Exception.message* to something that is Python 3 > compatible -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit f83706176ac767e9959aeca49c16598e5293cb43 Merge: 57b0bb9 7d201cd Author: Marcus Eriksson AuthorDate: Thu Apr 9 16:30:40 2020 +0200 Merge branch 'cassandra-3.0' into cassandra-3.11 .../cassandra/distributed/UpgradeableCluster.java | 9 - .../cassandra/distributed/impl/InstanceConfig.java | 2 +- .../cassandra/distributed/upgrade/UpgradeTestBase.java | 17 + 3 files changed, 18 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (00fb6d7 -> 23e6e6a)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 00fb6d7 Fix overflows on StreamingTombstoneHistogramBuilder produced by large deletion times. new 479a76d Fix upgrade dtests new 7d201cd Merge branch 'cassandra-2.2' into cassandra-3.0 new f837061 Merge branch 'cassandra-3.0' into cassandra-3.11 new 23e6e6a Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../cassandra/distributed/UpgradeableCluster.java | 9 - .../cassandra/distributed/impl/InstanceConfig.java | 2 +- .../cassandra/distributed/upgrade/UpgradeTestBase.java | 17 + 3 files changed, 18 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7d201cd0e32260d80ada3a0de0829d129632b1c9 Merge: fe263ad 479a76d Author: Marcus Eriksson AuthorDate: Thu Apr 9 16:30:27 2020 +0200 Merge branch 'cassandra-2.2' into cassandra-3.0 .../cassandra/distributed/UpgradeableCluster.java | 9 - .../cassandra/distributed/impl/InstanceConfig.java | 2 +- .../cassandra/distributed/upgrade/UpgradeTestBase.java | 17 + 3 files changed, 18 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: Fix upgrade dtests
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 479a76d Fix upgrade dtests
479a76d is described below
commit 479a76df2320856a0bcc5c202d40050c3b3ab14d
Author: Marcus Eriksson
AuthorDate: Wed Apr 8 14:54:07 2020 +0200
Fix upgrade dtests
Patch by marcuse; reviewed by Alex Petrov and David Capwell for
CASSANDRA-15708
---
.../cassandra/distributed/UpgradeableCluster.java | 9 -
.../cassandra/distributed/impl/InstanceConfig.java | 2 +-
.../cassandra/distributed/upgrade/UpgradeTestBase.java | 17 +
3 files changed, 18 insertions(+), 10 deletions(-)
diff --git
a/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
b/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
index a7899fe..73a3c8a 100644
--- a/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
+++ b/test/distributed/org/apache/cassandra/distributed/UpgradeableCluster.java
@@ -52,15 +52,6 @@ public class UpgradeableCluster extends
AbstractCluster im
{
return new Builder(UpgradeableCluster::new)
{
-{
-withVersion(CURRENT_VERSION);
-}
-
-protected void setupLogging(File file)
-{
-setupLogging(file);
-}
-
protected IInstanceConfig generateConfig(int nodeNum, String
ipAddress, NetworkTopology networkTopology, File root, String token, String
seedIp)
{
return InstanceConfig.generate(nodeNum, ipAddress,
networkTopology, root, token, seedIp);
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
b/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
index 0aafbca..55b0dae 100644
--- a/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
@@ -220,7 +220,7 @@ public class InstanceConfig implements IInstanceConfig
}
catch (NoSuchFieldException e)
{
-throw new IllegalStateException(e);
+return;
}
if (valueField.getType().isEnum() && value instanceof String)
diff --git
a/test/distributed/org/apache/cassandra/distributed/upgrade/UpgradeTestBase.java
b/test/distributed/org/apache/cassandra/distributed/upgrade/UpgradeTestBase.java
index 6ae1d7b..effec83 100644
---
a/test/distributed/org/apache/cassandra/distributed/upgrade/UpgradeTestBase.java
+++
b/test/distributed/org/apache/cassandra/distributed/upgrade/UpgradeTestBase.java
@@ -24,6 +24,9 @@ import java.util.HashSet;
import java.util.List;
import java.util.Set;
+import org.junit.After;
+import org.junit.BeforeClass;
+
import org.apache.cassandra.distributed.UpgradeableCluster;
import org.apache.cassandra.distributed.api.ICluster;
import org.apache.cassandra.distributed.api.IInstance;
@@ -38,6 +41,20 @@ import static
org.apache.cassandra.distributed.shared.Versions.find;
public class UpgradeTestBase extends DistributedTestBase
{
+@After
+public void afterEach()
+{
+System.runFinalization();
+System.gc();
+}
+
+@BeforeClass
+public static void beforeClass() throws Throwable
+{
+ICluster.setup();
+}
+
+
public Builder builder()
{
return (Builder) UpgradeableCluster.build();
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 23e6e6ab4664f3e4db0ab3d2f0d0243341a47a66 Merge: 00fb6d7 f837061 Author: Marcus Eriksson AuthorDate: Thu Apr 9 16:31:04 2020 +0200 Merge branch 'cassandra-3.11' into trunk .../cassandra/distributed/UpgradeableCluster.java | 9 - .../cassandra/distributed/impl/InstanceConfig.java | 2 +- .../cassandra/distributed/upgrade/UpgradeTestBase.java | 17 + 3 files changed, 18 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (fe263ad -> 7d201cd)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from fe263ad Merge branch 'cassandra-2.2' into cassandra-3.0 new 479a76d Fix upgrade dtests new 7d201cd Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../cassandra/distributed/UpgradeableCluster.java | 9 - .../cassandra/distributed/impl/InstanceConfig.java | 2 +- .../cassandra/distributed/upgrade/UpgradeTestBase.java | 17 + 3 files changed, 18 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (57b0bb9 -> f837061)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 57b0bb9 Merge branch 'cassandra-3.0' into cassandra-3.11 new 479a76d Fix upgrade dtests new 7d201cd Merge branch 'cassandra-2.2' into cassandra-3.0 new f837061 Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../cassandra/distributed/UpgradeableCluster.java | 9 - .../cassandra/distributed/impl/InstanceConfig.java | 2 +- .../cassandra/distributed/upgrade/UpgradeTestBase.java | 17 + 3 files changed, 18 insertions(+), 10 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15708) Fix in-jvm upgrade dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15708: Fix Version/s: 4.0-alpha 3.11.7 3.0.21 2.2.17 Since Version: 2.2.x Source Control Link: https://github.com/apache/cassandra/commit/479a76df2320856a0bcc5c202d40050c3b3ab14d Resolution: Fixed Status: Resolved (was: Ready to Commit) committed but forgot adding the log message, will ninja that in > Fix in-jvm upgrade dtests > - > > Key: CASSANDRA-15708 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15708 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 2.2.17, 3.0.21, 3.11.7, 4.0-alpha > > > In-jvm upgrade dtests were broken by CASSANDRA-15539 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15702) copyutil.py uses Exception.message which doesn't exist in Python 3
[ https://issues.apache.org/jira/browse/CASSANDRA-15702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079420#comment-17079420 ] Stefania Alborghetti commented on CASSANDRA-15702: -- Patch looks good, running cqlsh tests here: [53|https://ci-cassandra.apache.org/job/Cassandra-devbranch-dtest/53/] > copyutil.py uses Exception.message which doesn't exist in Python 3 > -- > > Key: CASSANDRA-15702 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15702 > Project: Cassandra > Issue Type: Bug > Components: Tool/cqlsh >Reporter: Eduard Tudenhoefner >Assignee: Eduard Tudenhoefner >Priority: Normal > Labels: pull-request-available > Fix For: 4.0-alpha > > Time Spent: 10m > Remaining Estimate: 0h > > Python 3 deprecated and removed the use of Exception.message. The purpose of > this ticket is to convert *Exception.message* to something that is Python 3 > compatible -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15614) Flakey test - TestBootstrap.test_resumable_bootstrap
[
https://issues.apache.org/jira/browse/CASSANDRA-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079426#comment-17079426
]
Ekaterina Dimitrova commented on CASSANDRA-15614:
-
CASSANDRA-15667 solves the issue, the test is deterministic already.
> Flakey test - TestBootstrap.test_resumable_bootstrap
> -
>
> Key: CASSANDRA-15614
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15614
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
> Attachments: test_resumable_bootstrap-CASSANDRA-15614.txt
>
>
> Fails 2 out of 10 times on Mac/Java 8
> {noformat}
> ==
> FAILURES
> ==
> *___
> TestBootstrap.test_resumable_bootstrap
> ___*
>
> self =
>
> *@since('2.2')*
> *def test_resumable_bootstrap(self):*
> *"""*
> *Test resuming bootstrap after data streaming failure*
> *"""*
> *cluster = self.cluster*
> *cluster.populate(2)*
> **
> *node1 = cluster.nodes['node1']*
> *# set up byteman*
> *node1.byteman_port = '8100'*
> *node1.import_config_files()*
> **
> *cluster.start(wait_other_notice=True)*
> *# kill stream to node3 in the middle of streaming to let it fail*
> *if cluster.version() < '4.0':*
> *node1.byteman_submit([self.byteman_submit_path_pre_4_0])*
> *else:*
> *node1.byteman_submit([self.byteman_submit_path_4_0])*
> *node1.stress(['write', 'n=1K', 'no-warmup', 'cl=TWO', '-schema',
> 'replication(factor=2)', '-rate', 'threads=50'])*
> *cluster.flush()*
> **
> *# start bootstrapping node3 and wait for streaming*
> *node3 = new_node(cluster)*
> *node3.start(wait_other_notice=False)*
> **
> *# let streaming fail as we expect*
> *> node3.watch_log_for('Some data streaming failed')*
>
> *bootstrap_test.py*:365:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _
>
> self = , exprs = 'Some data streaming
> failed', from_mark = None, timeout = 600, process = None, verbose = False,
> filename = 'system.log'
>
> *def watch_log_for(self, exprs, from_mark=None, timeout=600,
> process=None, verbose=False, filename='system.log'):*
> *"""*
> *Watch the log until one or more (regular) expression are found.*
> *This methods when all the expressions have been found or the
> method*
> *timeouts (a TimeoutError is then raised). On successful
> completion,*
> *a list of pair (line matched, match object) is returned.*
> *"""*
> *start = time.time()*
> *tofind = [exprs] if isinstance(exprs, string_types) else exprs*
> *tofind = [re.compile(e) for e in tofind]*
> *matchings = []*
> *reads = ""*
> *if len(tofind) == 0:*
> *return None*
> **
> *log_file = os.path.join(self.get_path(), 'logs', filename)*
> *output_read = False*
> *while not os.path.exists(log_file):*
> *time.sleep(.5)*
> *if start + timeout < time.time():*
> *raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S",
> time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be
> created.".format(log_file))*
> *if process and not output_read:*
> *process.poll()*
> *if process.returncode is not None:*
> *self.print_process_output(self.name, process, verbose)*
> *output_read = True*
> *if process.returncode != 0:*
> *raise RuntimeError() # Shouldn't reuse RuntimeError
> but I'm lazy*
> **
> *with open(log_file) as f:*
> *if from_mark:*
> *f.seek(from_mark)*
> **
> *while True:*
> *# First, if we have a process to check, then check it.*
> *# Skip on Windows - stdout/stderr is cassandra.bat*
>
[jira] [Updated] (CASSANDRA-15632) Create a deterministic in-jvm equivalent of dtest TestBootstrap.test_resumable_bootstrap
[ https://issues.apache.org/jira/browse/CASSANDRA-15632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-15632: Resolution: Fixed Status: Resolved (was: Open) CASSANDRA-15667 solves the issue, the test is deterministic already. > Create a deterministic in-jvm equivalent of dtest > TestBootstrap.test_resumable_bootstrap > > > Key: CASSANDRA-15632 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15632 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Priority: Normal > > This ticket is a followup on CASSANDRA-15614 - fix flakey test > TestBootstrap.test_resumable_bootstrap. > The dtest is not really fully deterministic. In-jvm test might give a way > for better control on the streaming failures. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 6193b7c55621edf2fe3447cfd7075e0c49b59553 Merge: 23e6e6a 9e891cb Author: Marcus Eriksson AuthorDate: Thu Apr 9 16:39:56 2020 +0200 Merge branch 'cassandra-3.11' into trunk .../org/apache/cassandra/distributed/impl/InstanceConfig.java| 5 + 1 file changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (23e6e6a -> 6193b7c)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 23e6e6a Merge branch 'cassandra-3.11' into trunk new 91ce047 CASSANDRA-15708 followup - log warning if there is a missing field new 29c8624 Merge branch 'cassandra-2.2' into cassandra-3.0 new 9e891cb Merge branch 'cassandra-3.0' into cassandra-3.11 new 6193b7c Merge branch 'cassandra-3.11' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/distributed/impl/InstanceConfig.java| 5 + 1 file changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 9e891cbf4eb2e4a47f0e88e647b9fbb6369b4e35 Merge: f837061 29c8624 Author: Marcus Eriksson AuthorDate: Thu Apr 9 16:39:49 2020 +0200 Merge branch 'cassandra-3.0' into cassandra-3.11 .../org/apache/cassandra/distributed/impl/InstanceConfig.java| 5 + 1 file changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated: CASSANDRA-15708 followup - log warning if there is a missing field
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch cassandra-2.2
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.2 by this push:
new 91ce047 CASSANDRA-15708 followup - log warning if there is a missing
field
91ce047 is described below
commit 91ce0473833f87328e858001806b81c0686efe2a
Author: Marcus Eriksson
AuthorDate: Thu Apr 9 16:39:22 2020 +0200
CASSANDRA-15708 followup - log warning if there is a missing field
---
.../org/apache/cassandra/distributed/impl/InstanceConfig.java| 5 +
1 file changed, 5 insertions(+)
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
b/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
index 55b0dae..d13a0b6 100644
--- a/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
+++ b/test/distributed/org/apache/cassandra/distributed/impl/InstanceConfig.java
@@ -30,6 +30,9 @@ import java.util.TreeMap;
import java.util.UUID;
import java.util.function.Function;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
import org.apache.cassandra.distributed.api.Feature;
import org.apache.cassandra.distributed.api.IInstanceConfig;
import org.apache.cassandra.distributed.shared.NetworkTopology;
@@ -40,6 +43,7 @@ import org.apache.cassandra.locator.SimpleSeedProvider;
public class InstanceConfig implements IInstanceConfig
{
private static final Object NULL = new Object();
+private static final Logger logger =
LoggerFactory.getLogger(InstanceConfig.class);
public final int num;
public int num() { return num; }
@@ -220,6 +224,7 @@ public class InstanceConfig implements IInstanceConfig
}
catch (NoSuchFieldException e)
{
+logger.warn("No such field: {} in config class {}", fieldName,
configClass);
return;
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (f837061 -> 9e891cb)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from f837061 Merge branch 'cassandra-3.0' into cassandra-3.11 new 91ce047 CASSANDRA-15708 followup - log warning if there is a missing field new 29c8624 Merge branch 'cassandra-2.2' into cassandra-3.0 new 9e891cb Merge branch 'cassandra-3.0' into cassandra-3.11 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/distributed/impl/InstanceConfig.java| 5 + 1 file changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079224#comment-17079224
]
Eduard Tudenhoefner edited comment on CASSANDRA-15711 at 4/9/20, 2:41 PM:
--
Tests on Jenkins
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch/46/], more
specifically
[this|https://ci-cassandra.apache.org/job/Cassandra-devbranch-cqlsh-tests/36/cython=no,jdk=JDK%201.8%20(latest),label=cassandra/testReport/cqlshlib.test.test_cqlsh_completion/TestCqlshCompletion/test_complete_in_alter_keyspace/]
is the newly added test.
was (Author: eduard.tudenhoefner):
Tests on Jenkins
[here|https://ci-cassandra.apache.org/job/Cassandra-devbranch/46/]
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 29c862462a4271affc7627d5ebb00e00df5ffb37 Merge: 7d201cd 91ce047 Author: Marcus Eriksson AuthorDate: Thu Apr 9 16:39:38 2020 +0200 Merge branch 'cassandra-2.2' into cassandra-3.0 .../org/apache/cassandra/distributed/impl/InstanceConfig.java| 5 + 1 file changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (7d201cd -> 29c8624)
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 7d201cd Merge branch 'cassandra-2.2' into cassandra-3.0 new 91ce047 CASSANDRA-15708 followup - log warning if there is a missing field new 29c8624 Merge branch 'cassandra-2.2' into cassandra-3.0 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../org/apache/cassandra/distributed/impl/InstanceConfig.java| 5 + 1 file changed, 5 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15612) Mutating sstable metadata during anticompaction fails silently on legacy sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-15612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079436#comment-17079436 ] Benjamin Lerer commented on CASSANDRA-15612: Thanks [~marcuse] +1 > Mutating sstable metadata during anticompaction fails silently on legacy > sstables > - > > Key: CASSANDRA-15612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15612 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > If there is an sstable with a version without pending repair information > included in an incremental repair we silently don't set the pending repair > correctly. We should fail the IR instead. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15612) Mutating sstable metadata during anticompaction fails silently on legacy sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-15612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-15612: --- Status: Ready to Commit (was: Review In Progress) > Mutating sstable metadata during anticompaction fails silently on legacy > sstables > - > > Key: CASSANDRA-15612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15612 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > If there is an sstable with a version without pending repair information > included in an incremental repair we silently don't set the pending repair > correctly. We should fail the IR instead. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Fail incremental repair if there is an old version sstable involved
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 579c9ab Fail incremental repair if there is an old version sstable
involved
579c9ab is described below
commit 579c9ab311e9efcbc3329599232a8f9992b2694e
Author: Marcus Eriksson
AuthorDate: Mon Mar 2 11:07:57 2020 +0100
Fail incremental repair if there is an old version sstable involved
Patch by marcuse; reviewed by Benjamin Lerer for CASSANDRA-15612
---
CHANGES.txt| 1 +
.../db/compaction/CompactionStrategyManager.java | 12
.../cassandra/db/repair/PendingAntiCompaction.java | 18 --
.../cassandra/io/sstable/LegacySSTableTest.java| 71 ++
4 files changed, 98 insertions(+), 4 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 46790c9..cda7585 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Fail incremental repair if an old version sstable is involved
(CASSANDRA-15612)
* Fix overflows on StreamingTombstoneHistogramBuilder produced by large
deletion times (CASSANDRA-14773)
* Mark system_views/system_virtual_schema as system keyspaces in cqlsh
(CASSANDRA-15706)
* Avoid unnecessary collection/iterator allocations during btree construction
(CASSANDRA-15390)
diff --git
a/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
index fd4dbeb..546f61b 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionStrategyManager.java
@@ -27,6 +27,7 @@ import java.util.Collections;
import java.util.ConcurrentModificationException;
import java.util.HashSet;
import java.util.List;
+import java.util.Objects;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.locks.ReentrantReadWriteLock;
@@ -1201,6 +1202,7 @@ public class CompactionStrategyManager implements
INotificationConsumer
{
sstable.descriptor.getMetadataSerializer().mutateRepairMetadata(sstable.descriptor,
repairedAt, pendingRepair, isTransient);
sstable.reloadSSTableMetadata();
+verifyMetadata(sstable, repairedAt, pendingRepair,
isTransient);
changed.add(sstable);
}
}
@@ -1218,4 +1220,14 @@ public class CompactionStrategyManager implements
INotificationConsumer
}
}
}
+
+private static void verifyMetadata(SSTableReader sstable, long repairedAt,
UUID pendingRepair, boolean isTransient)
+{
+if (!Objects.equals(pendingRepair, sstable.getPendingRepair()))
+throw new IllegalStateException(String.format("Failed setting
pending repair to %s on %s (pending repair is %s)", pendingRepair, sstable,
sstable.getPendingRepair()));
+if (repairedAt != sstable.getRepairedAt())
+throw new IllegalStateException(String.format("Failed setting
repairedAt to %d on %s (repairedAt is %d)", repairedAt, sstable,
sstable.getRepairedAt()));
+if (isTransient != sstable.isTransient())
+throw new IllegalStateException(String.format("Failed setting
isTransient to %b on %s (isTransient is %b)", isTransient, sstable,
sstable.isTransient()));
+}
}
diff --git a/src/java/org/apache/cassandra/db/repair/PendingAntiCompaction.java
b/src/java/org/apache/cassandra/db/repair/PendingAntiCompaction.java
index fac164d..e49e76e 100644
--- a/src/java/org/apache/cassandra/db/repair/PendingAntiCompaction.java
+++ b/src/java/org/apache/cassandra/db/repair/PendingAntiCompaction.java
@@ -69,7 +69,7 @@ public class PendingAntiCompaction
private static final int ACQUIRE_SLEEP_MS =
Integer.getInteger("cassandra.acquire_sleep_ms", 1000);
private static final int ACQUIRE_RETRY_SECONDS =
Integer.getInteger("cassandra.acquire_retry_seconds", 60);
-static class AcquireResult
+public static class AcquireResult
{
final ColumnFamilyStore cfs;
final Refs refs;
@@ -82,7 +82,8 @@ public class PendingAntiCompaction
this.txn = txn;
}
-void abort()
+@VisibleForTesting
+public void abort()
{
if (txn != null)
txn.abort();
@@ -122,6 +123,14 @@ public class PendingAntiCompaction
if (metadata.repairedAt != UNREPAIRED_SSTABLE)
return false;
+if (!sstable.descriptor.version.hasPendingRepair())
+{
+String message = String.format("Prepare phase failed because
it encountered legacy sstables that don't " +
+ "support pending repair, run
upgradesstables before starting incremental " +
+
[jira] [Updated] (CASSANDRA-15612) Mutating sstable metadata during anticompaction fails silently on legacy sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-15612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-15612: Source Control Link: https://github.com/apache/cassandra/commit/579c9ab311e9efcbc3329599232a8f9992b2694e Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed, thanks! > Mutating sstable metadata during anticompaction fails silently on legacy > sstables > - > > Key: CASSANDRA-15612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15612 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-alpha > > > If there is an sstable with a version without pending repair information > included in an incremental repair we silently don't set the pending repair > correctly. We should fail the IR instead. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Mark system_views/system_virtual_schema as non-alterable keyspaces in cqlsh
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 39a0703 Mark system_views/system_virtual_schema as non-alterable
keyspaces in cqlsh
39a0703 is described below
commit 39a0703b65c985c4b99eed1c045d0f00bbbcc9db
Author: Eduard Tudenhoefner
AuthorDate: Thu Apr 9 12:01:21 2020 +0200
Mark system_views/system_virtual_schema as non-alterable keyspaces in cqlsh
patch by Eduard Tudenhoefner; reviewed by Benjamin Lerer for
CASSANDRA-15711
---
CHANGES.txt | 3 +--
pylib/cqlshlib/cql3handling.py | 2 +-
pylib/cqlshlib/test/test_cqlsh_completion.py | 8
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index cda7585..2ad2aab 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0-alpha4
+ * Mark system_views/system_virtual_schema as non-alterable keyspaces in cqlsh
(CASSANDRA-15711)
* Fail incremental repair if an old version sstable is involved
(CASSANDRA-15612)
* Fix overflows on StreamingTombstoneHistogramBuilder produced by large
deletion times (CASSANDRA-14773)
* Mark system_views/system_virtual_schema as system keyspaces in cqlsh
(CASSANDRA-15706)
@@ -56,8 +57,6 @@
Merged from 3.11:
* Allow sstableloader to use SSL on the native port (CASSANDRA-14904)
Merged from 3.0:
-===
-3.0.21
* Memtable memory allocations may deadlock (CASSANDRA-15367)
* Run evictFromMembership in GossipStage (CASSANDRA-15592)
Merged from 2.2:
diff --git a/pylib/cqlshlib/cql3handling.py b/pylib/cqlshlib/cql3handling.py
index 8738f10..b2403a7 100644
--- a/pylib/cqlshlib/cql3handling.py
+++ b/pylib/cqlshlib/cql3handling.py
@@ -35,7 +35,7 @@ class UnexpectedTableStructure(UserWarning):
SYSTEM_KEYSPACES = ('system', 'system_schema', 'system_traces', 'system_auth',
'system_distributed', 'system_views', 'system_virtual_schema')
-NONALTERBALE_KEYSPACES = ('system', 'system_schema')
+NONALTERBALE_KEYSPACES = ('system', 'system_schema', 'system_views',
'system_virtual_schema')
class Cql3ParsingRuleSet(CqlParsingRuleSet):
diff --git a/pylib/cqlshlib/test/test_cqlsh_completion.py
b/pylib/cqlshlib/test/test_cqlsh_completion.py
index a8d6719..d432f22 100644
--- a/pylib/cqlshlib/test/test_cqlsh_completion.py
+++ b/pylib/cqlshlib/test/test_cqlsh_completion.py
@@ -820,3 +820,11 @@ class TestCqlshCompletion(CqlshCompletionCase):
def test_complete_in_drop_index(self):
pass
+
+def test_complete_in_alter_keyspace(self):
+self.trycompletions('ALTER KEY', 'SPACE ')
+self.trycompletions('ALTER KEYSPACE ', '',
choices=[self.cqlsh.keyspace, 'system_auth',
+
'system_distributed', 'system_traces'])
+self.trycompletions('ALTER KEYSPACE system_trac', "es WITH replication
= {'class': '")
+self.trycompletions("ALTER KEYSPACE system_traces WITH replication =
{'class': '", '',
+choices=['NetworkTopologyStrategy',
'SimpleStrategy'])
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15711:
---
Reviewers: Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer)
Status: Review In Progress (was: Patch Available)
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15711:
---
Status: Ready to Commit (was: Review In Progress)
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079447#comment-17079447
]
Benjamin Lerer commented on CASSANDRA-15711:
Thanks. The patch look good to me.
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15711) cqlsh ALTER KEYSPACE completion show virtual table keyspaces
[
https://issues.apache.org/jira/browse/CASSANDRA-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15711:
---
Fix Version/s: (was: 4.0)
4.0-alpha
Source Control Link:
https://github.com/apache/cassandra/commit/39a0703b65c985c4b99eed1c045d0f00bbbcc9db
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed into trunk at 39a0703b65c985c4b99eed1c045d0f00bbbcc9db
> cqlsh ALTER KEYSPACE completion show virtual table keyspaces
>
>
> Key: CASSANDRA-15711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15711
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/cqlsh
>Reporter: Benjamin Lerer
>Assignee: Eduard Tudenhoefner
>Priority: Low
> Labels: pull-request-available
> Fix For: 4.0-alpha
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code}
> cqlsh> ALTER KEYSPACE
> system_traces system_virtual_schema system_auth
> system_views system_distributed
> {code}
> The {{system_virtual_schema}} and {{system_views}} keyspaces are virtual
> tables keyspaces that cannot be altered and by consequence they should not
> appear in this list.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-13666) Secondary index query on partition key columns might not return partitions with only static data
[
https://issues.apache.org/jira/browse/CASSANDRA-13666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer reassigned CASSANDRA-13666:
--
Assignee: Berenguer Blasi (was: Sam Tunnicliffe)
> Secondary index query on partition key columns might not return partitions
> with only static data
>
>
> Key: CASSANDRA-13666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13666
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Benjamin Lerer
>Assignee: Berenguer Blasi
>Priority: Normal
>
> The problem can be reproduced with the following test in {{3.0}}:
> {code}
>@Test
> public void testIndexOnPartitionKeyWithPartitionWithoutRows() throws
> Throwable
> {
> createTable("CREATE TABLE %s (pk1 int, pk2 int, c int, s int static,
> v int, PRIMARY KEY((pk1, pk2), c))");
> createIndex("CREATE INDEX ON %s (pk2)");
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 1, 1, 1, 9, 1);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 1, 1, 2, 9, 2);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 3, 1, 1, 9, 1);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 4, 1, 1, 9, 1);
> flush();
> assertRows(execute("SELECT * FROM %s WHERE pk2 = ?", 1),
>row(1, 1, 1, 9, 1),
>row(1, 1, 2, 9, 2),
>row(3, 1, 1, 9, 1),
>row(4, 1, 1, 9, 1));
> execute("DELETE FROM %s WHERE pk1 = ? AND pk2 = ? AND c = ?", 3, 1,
> 1);
> assertRows(execute("SELECT * FROM %s WHERE pk2 = ?", 1),
>row(1, 1, 1, 9, 1),
>row(1, 1, 2, 9, 2),
>row(3, 1, null, 9, null), // This row will not be returned
>row(4, 1, 1, 9, 1));
> }
> {code}
> The problem seems to be that the index entries for the static data are
> inserted with an empty clustering key. When the first {{SELECT}} is executed
> those entries are removed by {{CompositesSearcher::filterStaleEntries}} which
> consider that those entries are stales. When the second {{SELECT}} is
> executed the index ignore the (3, 1) partition as there is not entry for it
> anymore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13666) Secondary index query on partition key columns might not return partitions with only static data
[
https://issues.apache.org/jira/browse/CASSANDRA-13666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079467#comment-17079467
]
Benjamin Lerer commented on CASSANDRA-13666:
[~beobal] I hope you do not mind. I reassigned the ticket to [~Bereng].
My guess was that you were busy with some other tickets.
> Secondary index query on partition key columns might not return partitions
> with only static data
>
>
> Key: CASSANDRA-13666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13666
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Benjamin Lerer
>Assignee: Berenguer Blasi
>Priority: Normal
>
> The problem can be reproduced with the following test in {{3.0}}:
> {code}
>@Test
> public void testIndexOnPartitionKeyWithPartitionWithoutRows() throws
> Throwable
> {
> createTable("CREATE TABLE %s (pk1 int, pk2 int, c int, s int static,
> v int, PRIMARY KEY((pk1, pk2), c))");
> createIndex("CREATE INDEX ON %s (pk2)");
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 1, 1, 1, 9, 1);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 1, 1, 2, 9, 2);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 3, 1, 1, 9, 1);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 4, 1, 1, 9, 1);
> flush();
> assertRows(execute("SELECT * FROM %s WHERE pk2 = ?", 1),
>row(1, 1, 1, 9, 1),
>row(1, 1, 2, 9, 2),
>row(3, 1, 1, 9, 1),
>row(4, 1, 1, 9, 1));
> execute("DELETE FROM %s WHERE pk1 = ? AND pk2 = ? AND c = ?", 3, 1,
> 1);
> assertRows(execute("SELECT * FROM %s WHERE pk2 = ?", 1),
>row(1, 1, 1, 9, 1),
>row(1, 1, 2, 9, 2),
>row(3, 1, null, 9, null), // This row will not be returned
>row(4, 1, 1, 9, 1));
> }
> {code}
> The problem seems to be that the index entries for the static data are
> inserted with an empty clustering key. When the first {{SELECT}} is executed
> those entries are removed by {{CompositesSearcher::filterStaleEntries}} which
> consider that those entries are stales. When the second {{SELECT}} is
> executed the index ignore the (3, 1) partition as there is not entry for it
> anymore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13666) Secondary index query on partition key columns might not return partitions with only static data
[
https://issues.apache.org/jira/browse/CASSANDRA-13666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17079467#comment-17079467
]
Benjamin Lerer edited comment on CASSANDRA-13666 at 4/9/20, 3:17 PM:
-
[~beobal] I reassigned the ticket to [~Bereng]. I hope you do not mind.
My guess was that you were busy with some other tickets.
was (Author: blerer):
[~beobal] I hope you do not mind. I reassigned the ticket to [~Bereng].
My guess was that you were busy with some other tickets.
> Secondary index query on partition key columns might not return partitions
> with only static data
>
>
> Key: CASSANDRA-13666
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13666
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Benjamin Lerer
>Assignee: Berenguer Blasi
>Priority: Normal
>
> The problem can be reproduced with the following test in {{3.0}}:
> {code}
>@Test
> public void testIndexOnPartitionKeyWithPartitionWithoutRows() throws
> Throwable
> {
> createTable("CREATE TABLE %s (pk1 int, pk2 int, c int, s int static,
> v int, PRIMARY KEY((pk1, pk2), c))");
> createIndex("CREATE INDEX ON %s (pk2)");
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 1, 1, 1, 9, 1);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 1, 1, 2, 9, 2);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 3, 1, 1, 9, 1);
> execute("INSERT INTO %s (pk1, pk2, c, s, v) VALUES (?, ?, ?, ?, ?)",
> 4, 1, 1, 9, 1);
> flush();
> assertRows(execute("SELECT * FROM %s WHERE pk2 = ?", 1),
>row(1, 1, 1, 9, 1),
>row(1, 1, 2, 9, 2),
>row(3, 1, 1, 9, 1),
>row(4, 1, 1, 9, 1));
> execute("DELETE FROM %s WHERE pk1 = ? AND pk2 = ? AND c = ?", 3, 1,
> 1);
> assertRows(execute("SELECT * FROM %s WHERE pk2 = ?", 1),
>row(1, 1, 1, 9, 1),
>row(1, 1, 2, 9, 2),
>row(3, 1, null, 9, null), // This row will not be returned
>row(4, 1, 1, 9, 1));
> }
> {code}
> The problem seems to be that the index entries for the static data are
> inserted with an empty clustering key. When the first {{SELECT}} is executed
> those entries are removed by {{CompositesSearcher::filterStaleEntries}} which
> consider that those entries are stales. When the second {{SELECT}} is
> executed the index ignore the (3, 1) partition as there is not entry for it
> anymore.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
