[jira] [Commented] (CASSANDRA-19007) Queries with multi-column replica-side filtering can miss rows
[
https://issues.apache.org/jira/browse/CASSANDRA-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829393#comment-17829393
]
Berenguer Blasi commented on CASSANDRA-19007:
-
I lack the context to create that ticket atm. If you could create it that'd be
awesome so we can close 5.0 loose ends.
> Queries with multi-column replica-side filtering can miss rows
> --
>
> Key: CASSANDRA-19007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19007
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{SELECT}} queries with multi-column replica-side filtering can miss rows if
> the filtered columns are spread across out-of-sync replicas. This dtest
> reproduces the issue:
> {code:java}
> @Test
> public void testMultiColumnReplicaSideFiltering() throws IOException
> {
> try (Cluster cluster = init(Cluster.build().withNodes(2).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int PRIMARY
> KEY, a int, b int)"));
> // insert a split row
> cluster.get(1).executeInternal(withKeyspace("INSERT INTO %s.t(k, a)
> VALUES (0, 1)"));
> cluster.get(2).executeInternal(withKeyspace("INSERT INTO %s.t(k, b)
> VALUES (0, 2)"));
> String select = withKeyspace("SELECT * FROM %s.t WHERE a = 1 AND b =
> 2 ALLOW FILTERING");
> Object[][] initialRows = cluster.coordinator(1).execute(select, ALL);
> assertRows(initialRows, row(0, 1, 2)); // not found!!
> }
> }
> {code}
> This edge case affects queries using {{ALLOW FILTERING}} or any index
> implementation.
> It affects all branches since multi-column replica-side filtering queries
> were introduced, long before 3.0.
> The protection mechanism added by CASSANDRA-8272/8273 won't deal with this
> case, since it only solves single-column conflicts where stale rows could
> resurrect. This bug however doesn't resurrect data, it can only miss rows
> while the replicas are out-of-sync.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[
https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829370#comment-17829370
]
Caleb Rackliffe edited comment on CASSANDRA-18325 at 3/21/24 3:56 AM:
--
I saw this again while testing the 5.0 patch for CASSANDRA-19461. All my logs
are attached to that ticket...
was (Author: maedhroz):
I saw this again while testing CASSANDRA-19461. All my logs are attached to
that ticket...
> Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
> -
>
> Key: CASSANDRA-18325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18325
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Maxwell Guo
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
>
> {code:java}
> assert not True
> + where True = 0x7f5b16fda070>>()
> + where 0x7f5b16fda070>> = .is_set
> Stacktrace
> self =
> def test_cleanup(self):
> """
> @jira_ticket CASSANDRA-11179
> Make sure we remove processed files during cleanup
> """
> cluster = self.cluster
> cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED',
> 'True')
> cluster.set_configuration_options(values=
> {'concurrent_compactors': 4}
> )
> cluster.populate(1)
> cluster.start()
> node1, = cluster.nodelist()
> for x in range(0, 5):
> node1.stress(['write', 'n=100k', 'no-warmup', '-schema',
> 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)',
> 'replication(factor=1)', '-rate', 'threads=10'])
> node1.flush()
> node2 = new_node(cluster)
> node2.start(wait_for_binary_proto=True)
> event = threading.Event()
> failed = threading.Event()
> jobs = 1
> thread = threading.Thread(target=self._monitor_datadir, args=(node1, event,
> len(node1.get_sstables("keyspace1", "standard1")), jobs, failed))
> thread.setDaemon(True)
> thread.start()
> node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs))
> event.set()
> thread.join()
> > assert not failed.is_set()
> E assert not True
> E + where True = 0x7f5b16fda070>>()
> E + where 0x7f5b16fda070>> = .is_set
> bootstrap_test.py:912: AssertionError
> {code}
>
> failed twice
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18325) Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
[
https://issues.apache.org/jira/browse/CASSANDRA-18325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829370#comment-17829370
]
Caleb Rackliffe commented on CASSANDRA-18325:
-
I saw this again while testing CASSANDRA-19461. All my logs are attached to
that ticket...
> Test failure: dtest.bootstrap_test.TestBootstrap.test_cleanup
> -
>
> Key: CASSANDRA-18325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18325
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Maxwell Guo
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
>
> {code:java}
> assert not True
> + where True = 0x7f5b16fda070>>()
> + where 0x7f5b16fda070>> = .is_set
> Stacktrace
> self =
> def test_cleanup(self):
> """
> @jira_ticket CASSANDRA-11179
> Make sure we remove processed files during cleanup
> """
> cluster = self.cluster
> cluster.set_environment_variable('CASSANDRA_TOKEN_PREGENERATION_DISABLED',
> 'True')
> cluster.set_configuration_options(values=
> {'concurrent_compactors': 4}
> )
> cluster.populate(1)
> cluster.start()
> node1, = cluster.nodelist()
> for x in range(0, 5):
> node1.stress(['write', 'n=100k', 'no-warmup', '-schema',
> 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)',
> 'replication(factor=1)', '-rate', 'threads=10'])
> node1.flush()
> node2 = new_node(cluster)
> node2.start(wait_for_binary_proto=True)
> event = threading.Event()
> failed = threading.Event()
> jobs = 1
> thread = threading.Thread(target=self._monitor_datadir, args=(node1, event,
> len(node1.get_sstables("keyspace1", "standard1")), jobs, failed))
> thread.setDaemon(True)
> thread.start()
> node1.nodetool("cleanup -j {} keyspace1 standard1".format(jobs))
> event.set()
> thread.join()
> > assert not failed.is_set()
> E assert not True
> E + where True = 0x7f5b16fda070>>()
> E + where 0x7f5b16fda070>> = .is_set
> bootstrap_test.py:912: AssertionError
> {code}
>
> failed twice
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Reviewers: David Capwell, Ekaterina Dimitrova (was: Caleb Rackliffe, David
Capwell, Ekaterina Dimitrova)
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Reviewers: David Capwell, Ekaterina Dimitrova, Caleb Rackliffe (was:
Ekaterina Dimitrova)
David Capwell, Ekaterina Dimitrova, Caleb Rackliffe (was: Caleb
Rackliffe, David Capwell, Ekaterina Dimitrova)
Status: Review In Progress (was: Patch Available)
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Attachment: result_details.tar.gz
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html, result_details.tar.gz
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829301#comment-17829301
]
Caleb Rackliffe edited comment on CASSANDRA-19461 at 3/21/24 3:54 AM:
--
5.0 patch: https://github.com/apache/cassandra/pull/3185
CI [looks good|^ci_summary.html]. Just a unit test issue I've already fixed
and a couple unrelated Python dtest failures, which look like CASSANDRA-18325.
was (Author: maedhroz):
5.0 patch: https://github.com/apache/cassandra/pull/3185
(CI run in progress...)
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Attachment: ci_summary.html
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829337#comment-17829337
]
Ekaterina Dimitrova commented on CASSANDRA-19461:
-
I'll review it tomorrow when we have CI also ready. Thanks
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-19461:
Reviewers: Ekaterina Dimitrova
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Source Control Link: (was: https://github.com/apache/cassandra/pull/3189,
https://github.com/apache/cassandra/pull/3187)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Source Control Link: https://github.com/apache/cassandra/pull/3189,
https://github.com/apache/cassandra/pull/3187 (was:
https://github.com/apache/cassandra/pull/3189)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Reviewers: Joshua McKenzie
Source Control Link: https://github.com/apache/cassandra/pull/3189
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Test and Documentation Plan: Run `ant dependency-check` on each branch
Status: Patch Available (was: Open)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19477:
--
Test and Documentation Plan: ci
Status: Patch Available (was: In Progress)
> Do not go to disk to get HintsStore.getTotalFileSize
>
>
> Key: CASSANDRA-19477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19477
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Jon Haddad
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0-rc, 5.x
>
> Attachments: flamegraph.cpu.html
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> When testing a cluster with more requests than it could handle, I noticed
> significant CPU time (25%) spent in HintsStore.getTotalFileSize. Here's what
> I'm seeing from profiling:
> 10% of CPU time spent in HintsDescriptor.fileName which only does this:
>
> {noformat}
> return String.format("%s-%s-%s.hints", hostId, timestamp, version);{noformat}
> At a bare minimum here we should create this string up front with the host
> and version and eliminate 2 of the 3 substitutions, but I think it's probably
> faster to use a StringBuilder and avoid the underlying regular expression
> altogether.
> 12% of the time is spent in org.apache.cassandra.io.util.File.length. It
> looks like this is called once for each hint file on disk for each host we're
> hinting to. In the case of an overloaded cluster, this is significant. It
> would be better if we were to track the file size in memory for each hint
> file and reference that rather than go to the filesystem.
> These fairly small changes should make Cassandra more reliable when under
> load spikes.
> CPU Flame graph attached.
> I only tested this in 4.1 but it looks like this is present up to trunk.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829325#comment-17829325
]
Brandon Williams commented on CASSANDRA-19484:
--
This sounds great.
bq. conditionally supply the property without having a dedicated invocation of
the dependency-check task with/without the parameter nvdDataFeedUrl specified.
Without seeing it, I think I'm ok with a dedicated task for it, it seems worth
the gains.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-19477:
--
Fix Version/s: 4.0.x
4.1.x
5.0-rc
> Do not go to disk to get HintsStore.getTotalFileSize
>
>
> Key: CASSANDRA-19477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19477
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Jon Haddad
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0-rc, 5.x
>
> Attachments: flamegraph.cpu.html
>
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> When testing a cluster with more requests than it could handle, I noticed
> significant CPU time (25%) spent in HintsStore.getTotalFileSize. Here's what
> I'm seeing from profiling:
> 10% of CPU time spent in HintsDescriptor.fileName which only does this:
>
> {noformat}
> return String.format("%s-%s-%s.hints", hostId, timestamp, version);{noformat}
> At a bare minimum here we should create this string up front with the host
> and version and eliminate 2 of the 3 substitutions, but I think it's probably
> faster to use a StringBuilder and avoid the underlying regular expression
> altogether.
> 12% of the time is spent in org.apache.cassandra.io.util.File.length. It
> looks like this is called once for each hint file on disk for each host we're
> hinting to. In the case of an overloaded cluster, this is significant. It
> would be better if we were to track the file size in memory for each hint
> file and reference that rather than go to the filesystem.
> These fairly small changes should make Cassandra more reliable when under
> load spikes.
> CPU Flame graph attached.
> I only tested this in 4.1 but it looks like this is present up to trunk.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829318#comment-17829318
]
Ariel Weisberg commented on CASSANDRA-19484:
[~jlewandowski] [~brandon.williams] FYI
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829313#comment-17829313 ] Francisco Guerrero commented on CASSANDRASC-106: +1 thanks for the patch. CI is green > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Description:
This allows you to point to a mirror that is faster and doesn’t require an API
key.
This is kind of painful to make work in {{ant}} because you can't specify the
property at all if you want to use the API and I couldn't find a way to get
{{ant}} to conditionally supply the property without having a dedicated
invocation of the {{dependency-check}} task with/without the parameter
{{nvdDataFeedUrl}} specified.
was:
This allows you to point to a mirror that is faster and doesn’t require an API
key.
This is kind of painful to make work in `ant` because you can't specify the
property at all if you want to use the API and I couldn't find a way to get
`ant` to conditionally supply the property without having a dedicated
invocation of the `dependency-check` task with/without the parameter
{noformat}
nvdDataFeedUrl
{noformat}
specified.
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in {{ant}} because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> {{ant}} to conditionally supply the property without having a dedicated
> invocation of the {{dependency-check}} task with/without the parameter
> {{nvdDataFeedUrl}} specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
[
https://issues.apache.org/jira/browse/CASSANDRA-19484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ariel Weisberg updated CASSANDRA-19484:
---
Change Category: Quality Assurance
Complexity: Low Hanging Fruit
Fix Version/s: 3.0.x
3.11.x
4.0.x
4.1.x
5.0.x
5.x
Status: Open (was: Triage Needed)
> Add support for providing nvdDatafeedUrl to OWASP
> -
>
> Key: CASSANDRA-19484
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Ariel Weisberg
>Assignee: Ariel Weisberg
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.0.x, 5.x
>
>
> This allows you to point to a mirror that is faster and doesn’t require an
> API key.
> This is kind of painful to make work in `ant` because you can't specify the
> property at all if you want to use the API and I couldn't find a way to get
> `ant` to conditionally supply the property without having a dedicated
> invocation of the `dependency-check` task with/without the parameter
> {noformat}
> nvdDataFeedUrl
> {noformat}
> specified.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19484) Add support for providing nvdDatafeedUrl to OWASP
Ariel Weisberg created CASSANDRA-19484:
--
Summary: Add support for providing nvdDatafeedUrl to OWASP
Key: CASSANDRA-19484
URL: https://issues.apache.org/jira/browse/CASSANDRA-19484
Project: Cassandra
Issue Type: Improvement
Components: Build
Reporter: Ariel Weisberg
Assignee: Ariel Weisberg
This allows you to point to a mirror that is faster and doesn’t require an API
key.
This is kind of painful to make work in `ant` because you can't specify the
property at all if you want to use the API and I couldn't find a way to get
`ant` to conditionally supply the property without having a dedicated
invocation of the `dependency-check` task with/without the parameter
{noformat}
nvdDataFeedUrl
{noformat}
specified.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRASC-106) Add restore task watcher to report long running tasks
[ https://issues.apache.org/jira/browse/CASSANDRASC-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829305#comment-17829305 ] Doug Rohrer commented on CASSANDRASC-106: - Addressed review comments, new Circle run started at [https://app.circleci.com/pipelines/github/JeetKunDoug/cassandra-sidecar/122/workflows/fcf9a5e0-4a70-4895-bd84-463ac91fffd5] > Add restore task watcher to report long running tasks > - > > Key: CASSANDRASC-106 > URL: https://issues.apache.org/jira/browse/CASSANDRASC-106 > Project: Sidecar for Apache Cassandra > Issue Type: Improvement > Components: Rest API >Reporter: Yifan Cai >Assignee: Doug Rohrer >Priority: Normal > Labels: pull-request-available > > Having a watcher to report the long running restore slice task can provide > better insights. > The watcher can live inside the RestoreProcessor and periodically examine the > futures of the running tasks. > Ideally, it signals the task to log the current stack trace. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19461) SAI does not index empty bytes even for types that allow empty bytes as a valid input
[
https://issues.apache.org/jira/browse/CASSANDRA-19461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19461:
Test and Documentation Plan:
- new tests around empty text values for literal indexes through the write
lifecycle
- new general randomized tests for supported index data types
Status: Patch Available (was: In Progress)
5.0 patch: https://github.com/apache/cassandra/pull/3185
(CI run in progress...)
> SAI does not index empty bytes even for types that allow empty bytes as a
> valid input
> -
>
> Key: CASSANDRA-19461
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19461
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/SAI
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> This is easy to reproduce with a test that looks something like this:
> {noformat}
> @Test
> public void testEmptyString()
> {
> createTable("CREATE TABLE %s (k TEXT PRIMARY KEY, v text)");
> createIndex(String.format(CREATE_INDEX_TEMPLATE, 'v'));
> execute("INSERT INTO %s (k, v) VALUES ('0', '')");
> execute("INSERT INTO %s (k) VALUES ('1')");
>
> // flush(); < there is not always a memtable index involved, a fix
> will have to pay attention to this
> List rows = executeNet("SELECT * FROM %s WHERE v = ''").all();
> assertEquals(1, rows.size()); <— FAILS! No matches...
> }
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
JeetKunDoug commented on code in PR #41:
URL:
https://github.com/apache/cassandra-analytics/pull/41#discussion_r1532764805
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/CassandraBulkSourceRelation.java:
##
@@ -129,6 +143,64 @@ private void persist(@NotNull JavaPairRDD sortedRDD, Str
}
}
+private void recordSuccessfulJobStats(List writeResults)
+{
+List streamResults = writeResults.stream()
+
.map(WriteResult::streamResults)
+
.flatMap(Collection::stream)
+
.collect(Collectors.toList());
+
+long rowCount = streamResults.stream().mapToLong(res ->
res.rowCount).sum();
+long totalBytesWritten = streamResults.stream().mapToLong(res ->
res.bytesWritten).sum();
+boolean hasClusterTopologyChanged = writeResults.stream()
+
.map(WriteResult::isClusterResizeDetected)
+.anyMatch(b -> b);
+LOGGER.info("Bulk writer has written {} rows and {} bytes with
cluster-resize status: {}",
+rowCount,
+totalBytesWritten,
+hasClusterTopologyChanged);
+writerContext.recordJobStats(new HashMap<>()
+{
+{
+put("rowsWritten", Long.toString(rowCount));
+put("bytesWritten", Long.toString(totalBytesWritten));
+put("jobStatus", "Succeeded");
+put("clusterResizeDetected",
String.valueOf(hasClusterTopologyChanged));
+put("jobElapsedTimeMillis",
Long.toString(getElapsedTimeMillis()));
+}
+});
+}
+
+private void recordFailureStats(String reason)
+{
+writerContext.recordJobStats(new HashMap<>()
+{
+{
+put("jobStatus", "Failed");
+put("failureReason", reason);
+put("jobElapsedTimeMillis",
Long.toString(getElapsedTimeMillis()));
+}
+});
+}
+
+private long getElapsedTimeMillis()
+{
+long now = System.nanoTime();
+return TimeUnit.NANOSECONDS.toMillis(now - this.startTimeNanos);
+}
+
+/**
+ * Get a ref copy of BulkWriterContext broadcast variable and compose a
function to transform a partition into StreamResult
+ *
+ * @param ctx BulkWriterContext broadcast variable
+ * @return FlatMapFunction
+ */
+private static FlatMapFunction>,
WriteResult>
+partitionsFlatMapFunc(Broadcast ctx, String[]
columnNames)
Review Comment:
Why rename this function from `writeRowsInPartition` to
`partitionsFlatMapFunc` - the first is a descriptive name for what the function
does, the second is a descriptive name for what interface the returned value
implements... one is "business domain" and the other is "implementation
detail", and I think the former has much more value than the latter.
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/common/JobStats.java:
##
@@ -0,0 +1,41 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.cassandra.spark.common;
+
+import java.util.Map;
+
+/**
+ * Interface to provide functionality to report Spark Job Statistics and/or
properties
+ * that can optionally be instrumented. The default implementation merely logs
these
+ * stats at the end of the job.
+ */
+public interface JobStats
Review Comment:
See my other comments about how to implement this in BulkWriterContext - may
not need to be Serializable if we move it to an instance variable like we've
done with other parts of the BulkWriterContext interface.
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/BulkWriterContext.java:
##
@@ -21,7 +21,9 @@
import java.io.Serializable;
-public interface BulkWriterContext extends Serializable
+import org.apache.cassandra.spark.common.JobStats;
+
+public interface BulkWriterContext extends Serializable, JobSta
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
adutra commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532813593
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java:
##
@@ -100,7 +100,10 @@ protected void initializeGauge(
m -> {
MetricId id = getMetricId(m);
Iterable tags = MicrometerTags.toMicrometerTags(id.getTags());
-return Gauge.builder(id.getName(),
supplier).tags(tags).register(registry);
+return Gauge.builder(id.getName(), supplier)
+.strongReference(false)
Review Comment:
Well I think we can keep both approaches, the weak ref trick can still be
useful.
Also, what would be really awesome is to have a test for this. Maybe
adapting the code that was used to detect the leak in the first place? FYI we
have a `SessionLeakIT` class that runs in isolation, so we can get crazy there
and create 100 sessions if it's necessary. (Although, it's probably enough to
create just one session, close it, then assert that the registry becomes empty
again within some configured timeframe.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
adutra commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532801560
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdater.java:
##
@@ -31,13 +31,18 @@
import io.micrometer.core.instrument.Timer;
import java.time.Duration;
import java.util.Set;
+import java.util.function.Supplier;
import net.jcip.annotations.ThreadSafe;
@ThreadSafe
public class MicrometerNodeMetricUpdater extends
MicrometerMetricUpdater
implements NodeMetricUpdater {
private final Node node;
+ private final Supplier openConnectionsSupplier;
Review Comment:
Oooh I see, we are keeping a strong ref here to compensate for the weak ref
there. Smart :-)
But I agree with you that we need to better document the intent. Not only
IDEs will complain (mine is complaining), but it won't be long until someone
refactors this and removes the final fields, thus silently re-introducing the
memory leak.
A simple trick to calm down IDEs is to move all the calls to
`initializeGauge` to a separate method:
```java
public MicrometerNodeMetricUpdater(
Node node,
InternalDriverContext context,
Set enabledMetrics,
MeterRegistry registry) {
super(context, enabledMetrics, registry);
this.node = node;
// Keep a strong reference to all gauge suppliers, otherwise they will
be GC'ed too soon
openConnectionsSupplier = node::getOpenConnections;
streamSupplier = () -> availableStreamIds(node);
inFlightRequestsSupplier = () -> inFlightRequests(node);
orphanedStreamIdsSupplier = () -> orphanedStreamIds(node);
initMetrics();
}
private void initMetrics() {
DriverExecutionProfile profile = context.getConfig().getDefaultProfile();
initializeGauge(DefaultNodeMetric.OPEN_CONNECTIONS, profile,
openConnectionsSupplier);
initializeGauge(DefaultNodeMetric.AVAILABLE_STREAMS, profile,
streamSupplier);
initializeGauge(DefaultNodeMetric.IN_FLIGHT, profile,
inFlightRequestsSupplier);
initializeGauge(DefaultNodeMetric.ORPHANED_STREAMS, profile,
orphanedStreamIdsSupplier);
// ...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18753) Add an optimized default configuration to tests and make it available for new users
[
https://issues.apache.org/jira/browse/CASSANDRA-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829289#comment-17829289
]
Ekaterina Dimitrova edited comment on CASSANDRA-18753 at 3/20/24 7:54 PM:
--
{quote}The fault is from this ticket, so leaving it open is my preference (so
to maintain focus on the breakage, which is serious, but forgivable considering
agents are regularly hanging/thrashing atm).
{quote}
Agreed, I did not mean to suggest to close it before all outstanding issues are
fixed (here or in another blocker ticket). Just clarifying the status.
{quote}18594 solves it. It is only waiting on other small final changes in the
patch, eta is this week.
{quote}
Marking then 18594 as a blocker here, for clarity. Thanks!
was (Author: e.dimitrova):
{quote}The fault is from this ticket, so leaving it open is my preference (so
to maintain focus on the breakage, which is serious, but forgivable considering
agents are regularly hanging/thrashing atm).
{quote}
Agreed, I did not mean to suggest to close it before all outstanding issues are
fixed (here or in another blocker ticket). Just clarifying the status.
{quote}18594 solves it. It is only waiting on other small final changes in the
patch, eta is this week.
{quote}
Marking them 18594 as a blocker here, for clarity. Thanks!
> Add an optimized default configuration to tests and make it available for new
> users
> ---
>
> Key: CASSANDRA-18753
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18753
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Urgent
> Fix For: 5.0-rc, 5.x
>
> Attachments:
> CCM_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch,
>
> DTEST_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> We currently offer only one sample configuration file with Cassandra, and
> that file is deliberately configured to disable all new functionality and
> incompatible improvements. This works well for legacy users that want to have
> a painless upgrade, but is a very bad choice for new users, or anyone wanting
> to make comparisons between Cassandra versions or between Cassandra and other
> databases.
> We offer very little indication, in the database packaging itself, that there
> are well-tested configuration choices that can solve known problems and
> dramatically improve performance. This is guaranteed to paint the database in
> a worse light than it deserves, and will very likely hurt adoption.
> We should find a way to offer a very easy way of choosing between "optimized"
> and "compatible" defaults. At minimal, we could provide alternate yaml files.
> Alternatively, we could build on the {{storage_compatibility_mode}} concept
> to grow it into a setting that not only enables/disables certain settings,
> but also changes their default values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18753) Add an optimized default configuration to tests and make it available for new users
[
https://issues.apache.org/jira/browse/CASSANDRA-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829289#comment-17829289
]
Ekaterina Dimitrova edited comment on CASSANDRA-18753 at 3/20/24 7:53 PM:
--
{quote}The fault is from this ticket, so leaving it open is my preference (so
to maintain focus on the breakage, which is serious, but forgivable considering
agents are regularly hanging/thrashing atm).
{quote}
Agreed, I did not mean to suggest to close it before all outstanding issues are
fixed (here or in another blocker ticket). Just clarifying the status.
{quote}18594 solves it. It is only waiting on other small final changes in the
patch, eta is this week.
{quote}
Marking them 18594 as a blocker here, for clarity. Thanks!
was (Author: e.dimitrova):
{quote}The fault is from this ticket, so leaving it open is my preference (so
to maintain focus on the breakage, which is serious, but forgivable considering
agents are regularly hanging/thrashing atm).
{quote}
Agreed, I did not mean to suggest to close it before all outstanding issues are
fixed (here or in another blocker ticket).
{quote}18594 solves it. It is only waiting on other small final changes in the
patch, eta is this week.
{quote}
Marking them 18594 as a blocker here, for clarity. Thanks!
> Add an optimized default configuration to tests and make it available for new
> users
> ---
>
> Key: CASSANDRA-18753
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18753
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Urgent
> Fix For: 5.0-rc, 5.x
>
> Attachments:
> CCM_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch,
>
> DTEST_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> We currently offer only one sample configuration file with Cassandra, and
> that file is deliberately configured to disable all new functionality and
> incompatible improvements. This works well for legacy users that want to have
> a painless upgrade, but is a very bad choice for new users, or anyone wanting
> to make comparisons between Cassandra versions or between Cassandra and other
> databases.
> We offer very little indication, in the database packaging itself, that there
> are well-tested configuration choices that can solve known problems and
> dramatically improve performance. This is guaranteed to paint the database in
> a worse light than it deserves, and will very likely hurt adoption.
> We should find a way to offer a very easy way of choosing between "optimized"
> and "compatible" defaults. At minimal, we could provide alternate yaml files.
> Alternatively, we could build on the {{storage_compatibility_mode}} concept
> to grow it into a setting that not only enables/disables certain settings,
> but also changes their default values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18753) Add an optimized default configuration to tests and make it available for new users
[
https://issues.apache.org/jira/browse/CASSANDRA-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829289#comment-17829289
]
Ekaterina Dimitrova commented on CASSANDRA-18753:
-
{quote}The fault is from this ticket, so leaving it open is my preference (so
to maintain focus on the breakage, which is serious, but forgivable considering
agents are regularly hanging/thrashing atm).
{quote}
Agreed, I did not mean to suggest to close it before all outstanding issues are
fixed (here or in another blocker ticket).
{quote}18594 solves it. It is only waiting on other small final changes in the
patch, eta is this week.
{quote}
Marking them 18594 as a blocker here, for clarity. Thanks!
> Add an optimized default configuration to tests and make it available for new
> users
> ---
>
> Key: CASSANDRA-18753
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18753
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Urgent
> Fix For: 5.0-rc, 5.x
>
> Attachments:
> CCM_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch,
>
> DTEST_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> We currently offer only one sample configuration file with Cassandra, and
> that file is deliberately configured to disable all new functionality and
> incompatible improvements. This works well for legacy users that want to have
> a painless upgrade, but is a very bad choice for new users, or anyone wanting
> to make comparisons between Cassandra versions or between Cassandra and other
> databases.
> We offer very little indication, in the database packaging itself, that there
> are well-tested configuration choices that can solve known problems and
> dramatically improve performance. This is guaranteed to paint the database in
> a worse light than it deserves, and will very likely hurt adoption.
> We should find a way to offer a very easy way of choosing between "optimized"
> and "compatible" defaults. At minimal, we could provide alternate yaml files.
> Alternatively, we could build on the {{storage_compatibility_mode}} concept
> to grow it into a setting that not only enables/disables certain settings,
> but also changes their default values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18753) Add an optimized default configuration to tests and make it available for new users
[
https://issues.apache.org/jira/browse/CASSANDRA-18753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829286#comment-17829286
]
Michael Semb Wever commented on CASSANDRA-18753:
The fault is from this ticket, so leaving it open is my preference (so to
maintain focus on the breakage, which is serious, but forgivable considering
agents are regularly hanging/thrashing atm).
18594 solves it. It is only waiting on other small final changes in the patch,
eta is this week.
> Add an optimized default configuration to tests and make it available for new
> users
> ---
>
> Key: CASSANDRA-18753
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18753
> Project: Cassandra
> Issue Type: Improvement
> Components: Packaging
>Reporter: Branimir Lambov
>Assignee: Branimir Lambov
>Priority: Urgent
> Fix For: 5.0-rc, 5.x
>
> Attachments:
> CCM_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch,
>
> DTEST_Add_support_for_specifying_the_name_of_the_file_to_use_as_cassandra_YAML_.patch
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> We currently offer only one sample configuration file with Cassandra, and
> that file is deliberately configured to disable all new functionality and
> incompatible improvements. This works well for legacy users that want to have
> a painless upgrade, but is a very bad choice for new users, or anyone wanting
> to make comparisons between Cassandra versions or between Cassandra and other
> databases.
> We offer very little indication, in the database packaging itself, that there
> are well-tested configuration choices that can solve known problems and
> dramatically improve performance. This is guaranteed to paint the database in
> a worse light than it deserves, and will very likely hurt adoption.
> We should find a way to offer a very easy way of choosing between "optimized"
> and "compatible" defaults. At minimal, we could provide alternate yaml files.
> Alternatively, we could build on the {{storage_compatibility_mode}} concept
> to grow it into a setting that not only enables/disables certain settings,
> but also changes their default values.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829282#comment-17829282
]
Brandon Williams commented on CASSANDRA-19471:
--
bq. I doubt this is a home run
I guess I wasn't wrong, this [fails
horribly|https://app.circleci.com/pipelines/github/driftx/cassandra/1533/workflows/b81cb765-6f9f-44c0-ae36-83158bc208d2/jobs/78867]
in circle, though oddly I can't reproduce that anywhere else.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
absurdfarce commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532705838
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java:
##
@@ -100,7 +100,10 @@ protected void initializeGauge(
m -> {
MetricId id = getMetricId(m);
Iterable tags = MicrometerTags.toMicrometerTags(id.getTags());
-return Gauge.builder(id.getName(),
supplier).tags(tags).register(registry);
+return Gauge.builder(id.getName(), supplier)
+.strongReference(false)
Review Comment:
Ooh, I _very_ much like this! I _vastly_ prefer this to the current
proposed fix; it's _far_ more explicit about what's going on and doesn't
require complicated comments to explain what's going on.
FWIW I did
[mention](https://issues.apache.org/jira/browse/CASSANDRA-19457?focusedCommentId=17824523&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17824523)
some initial testing I'd done to try to determine the scope of this issue.
That testing seemed to indicate that the other metrics impls might be subject
to similar issues but I'm now somewhat suspicious of those results, especially
since the cause was so clearly tracked down to an issue with Micrometer.
Regardless, we should (a) re-confirm the current state of MicroProfile +
DropWizard behaviour and (b) confirm that your proposed fix addresses any
issues there (which it absolutely will).
@SiyaoIsHiding I very much agree with @adutra that something in this vein is
a better way to implement metric lifetimes than what we have now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
absurdfarce commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532689998
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdater.java:
##
@@ -30,22 +30,28 @@
import io.micrometer.core.instrument.Timer;
import java.time.Duration;
import java.util.Set;
+import java.util.function.Supplier;
import net.jcip.annotations.ThreadSafe;
@ThreadSafe
public class MicrometerSessionMetricUpdater extends
MicrometerMetricUpdater
implements SessionMetricUpdater {
+ private final Supplier connectedNodesSupplier = this::connectedNodes;
Review Comment:
Same as the similar question asked above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
absurdfarce commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532687869
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdater.java:
##
@@ -31,13 +31,18 @@
import io.micrometer.core.instrument.Timer;
import java.time.Duration;
import java.util.Set;
+import java.util.function.Supplier;
import net.jcip.annotations.ThreadSafe;
@ThreadSafe
public class MicrometerNodeMetricUpdater extends
MicrometerMetricUpdater
implements NodeMetricUpdater {
private final Node node;
+ private final Supplier openConnectionsSupplier;
Review Comment:
This is really the heart of the fix. In the original implementation the
supplier was passed inline to Micrometer; there was no ref kept anywhere else.
The unexpected (at least to me) [strong reference used in the Gauge
impl](https://github.com/micrometer-metrics/micrometer/blob/v1.6.5/micrometer-core/src/main/java/io/micrometer/core/instrument/Gauge.java#L46-L61)
was the only thing preventing the Supplier from getting GC'd. And that was
entirely a side-effect of the constructor we were using. So if we then change
the Gauge implementation to not using strong references the only ref for the
Supplier is now the weak ref in the Gauge... which means it's eligible for GC
earlier than we'd like.
Creating a strong ref on the class avoids that issue; the key question is
where. It seemed reasonable to define it on the updater itself so that it
won't persist beyond the lifetime of this class.
I very much grant you this is all very indirect and non-intuitive. As a
result the impl is brittle and (at a bare minimum) needs some additional
commentary to explain what's going on.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19483) Catch InvalidLegacyProtocolMagic exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad Schoening updated CASSANDRA-19483:
---
Description:
Similar to CASSANDRA-18839, we frequently see the exception
InvalidLegacyProtocolMagic with full stack traces flooding our logs. These are
due to Qualys vulnerability scans.
It seem to be a simple check in Message.java and would be better handled by:
a) returning a boolean from validateLegacyProtocolMagic() ->
hasValidLegacyProtocolMagic() instead of creating a custom exception class or
b) adding a catch block in HandshakeProtocol.java and return null as is done
for messagingVersion
{code:java}
static Initiate maybeDecode(ByteBuf buf) throws IOException
{
...
try (DataInputBuffer in = new DataInputBuffer(nio, false))
{
validateLegacyProtocolMagic(in.readInt()); // throws exception
int flags = in.readInt();
// legacy pre40 messagingVersion flag
if (getBits(flags, 8, 8) < VERSION_40)
return null;
int minMessagingVersion = getBits(flags, 16, 8);
int maxMessagingVersion = getBits(flags, 24, 8);
// 5.0+ does not support pre40
if (maxMessagingVersion < MessagingService.VERSION_40)
return null;
}
catch (EOFException e)
{
return null;
}
}
{code}
{code:java}
static void validateLegacyProtocolMagic(int magic) throws
InvalidLegacyProtocolMagic
{
if (magic != PROTOCOL_MAGIC)
throw new InvalidLegacyProtocolMagic(magic);
}
{code}
{code:java}
{{2024-03-20 03:47:27,380 [ERROR] [Messaging-EventLoop-3-2] cluster_id=9
ip_address=10.0.0.1 InboundConnectionInitiator.java:360 - Failed to properly
handshake with peer /10.0.2:33356. Closing the channel.}}
{{io.netty.handler.codec.DecoderException:
org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic: Read 1431520594,
Expected -900387334}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:478)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)}}
{{ at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)}}
{{ at
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)}}
{{ at
io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)}}
{{ at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)}}
{{ at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)}}
{{ at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)}}
{{ at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)}}
{{ at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)}}
{{ at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)}}
{{ at java.base/java.lang.Thread.run(Thread.java:834)}}
{{Caused by: org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic: Read
1431520594, Expected -900387334}}
{{ at
org.apache.cassandra.net.Message.validateLegacyProtocolMagic(Message.java:343)}}
{{ at
org.apache.cassandra.net.HandshakeProtocol$Initiate.maybeDecode(HandshakeProtocol.java:167)}}
{{ at
org.apache.cassandra.net.InboundConnectionInitiator$Handler.initiate(InboundConnectionInitiator.java:242)}}
{{ at
org.apache.cassandra.net.InboundConnectionInitiator$Handler.decode(InboundConnectionInitiator.java:235)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447)}}
{{ ... 18 common frames omitted}}
{code}
was:
Similar to CASSANDRA-18839, we frequently see the
[jira] [Updated] (CASSANDRA-19483) Catch InvalidLegacyProtocolMagic exceptions
[
https://issues.apache.org/jira/browse/CASSANDRA-19483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brad Schoening updated CASSANDRA-19483:
---
Change Category: Operability
Complexity: Low Hanging Fruit
Priority: Low (was: Normal)
Status: Open (was: Triage Needed)
> Catch InvalidLegacyProtocolMagic exceptions
> ---
>
> Key: CASSANDRA-19483
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19483
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Logging
>Reporter: Brad Schoening
>Priority: Low
>
> Similar to CASSANDRA-18839, we frequently see the exception
> InvalidLegacyProtocolMagic with full stack traces flooding our logs.
> It seem to be a simple check in Message.java and would be better handled by:
> a) return a boolean from validateLegacyProtocolMagic() instead of creating a
> custom exception class or
> b) add a catch block in HandshakeProtocol.java and return null as is done for
> messagingVersion
> {code:java}
> static Initiate maybeDecode(ByteBuf buf) throws IOException
> {
> ...
> try (DataInputBuffer in = new DataInputBuffer(nio, false))
> {
> validateLegacyProtocolMagic(in.readInt()); // throws
> exception
> int flags = in.readInt();
> // legacy pre40 messagingVersion flag
> if (getBits(flags, 8, 8) < VERSION_40)
> return null;
> int minMessagingVersion = getBits(flags, 16, 8);
> int maxMessagingVersion = getBits(flags, 24, 8);
> // 5.0+ does not support pre40
> if (maxMessagingVersion < MessagingService.VERSION_40)
> return null;
>
> }
> catch (EOFException e)
> {
> return null;
> }
> }
> {code}
> {code:java}
> static void validateLegacyProtocolMagic(int magic) throws
> InvalidLegacyProtocolMagic
> {
> if (magic != PROTOCOL_MAGIC)
> throw new InvalidLegacyProtocolMagic(magic);
> }
> {code}
> {code:java}
> {{2024-03-20 03:47:27,380 [ERROR] [Messaging-EventLoop-3-2] cluster_id=9
> ip_address=10.0.0.1 InboundConnectionInitiator.java:360 - Failed to properly
> handshake with peer /10.0.2:33356. Closing the channel.}}
> {{io.netty.handler.codec.DecoderException:
> org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic: Read 1431520594,
> Expected -900387334}}
> {{ at
> io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:478)}}
> {{ at
> io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404)}}
> {{ at
> io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371)}}
> {{ at
> io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354)}}
> {{ at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)}}
> {{ at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)}}
> {{ at
> io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)}}
> {{ at
> io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)}}
> {{ at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)}}
> {{ at
> io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)}}
> {{ at
> io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)}}
> {{ at
> io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)}}
> {{ at
> io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)}}
> {{ at
> io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)}}
> {{ at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)}}
> {{ at
> io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)}}
> {{ at
> io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)}}
> {{ at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)}}
> {{ at java.base/java.lang.Thread.run(Thread.java:834)}}
> {{Caused by: org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic:
> Read 1431520594, Expected -900387334}}
> {{ at
> org.apache.cassandra.net.Message.validateLegacyProtocolMagic(Message.java:343)}}
> {{ at
> org.apache.cassandra.net.H
[jira] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[ https://issues.apache.org/jira/browse/CASSANDRA-19471 ]
Brandon Williams deleted comment on CASSANDRA-19471:
--
was (Author: brandon.williams):
I did that, but the problem is there are no repeat jobs generated. It looks
like it's not just latest but all of them.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829278#comment-17829278
]
Brandon Williams commented on CASSANDRA-19471:
--
I did that, but the problem is there are no repeat jobs generated. It looks
like it's not just latest but all of them.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829257#comment-17829257
]
Dipietro Salvatore commented on CASSANDRA-19429:
[~rustyrazorblade] I understand. Would it help if I can provide a r7g.16xlarge
instance for a couple of days to you to run your test?
If so, I would just need your public SSH key that I can add to the instance to
allow you to access to it
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, Screenshot 2024-03-19 at 15.22.50.png,
> asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Profiling Cassandra 4.1.3 on large AWS instances, a high number of lock
> acquires is measured in the `getCapacity` function from
> `org/apache/cassandra/cache/InstrumentingCache` (1.9M lock acquires per 60
> seconds). Based on our tests on r8g.24xlarge instances (using Ubuntu 22.04),
> this limits the CPU utilization of the system to under 50% when testing at
> full load and therefore limits the achieved throughput.
> Removing the lock contention from the SSTableReader.java file by replacing
> the call to `getCapacity` with `size` achieves up to 2.95x increase in
> throughput on r8g.24xlarge and 2x on r7i.24xlarge:
> |Instance type|Cass 4.1.3|Cass 4.1.3 patched|
> |r8g.24xlarge|168k ops|496k ops (2.95x)|
> |r7i.24xlarge|153k ops|304k ops (1.98x)|
>
> Instructions to reproduce:
> {code:java}
> ## Requirements for Ubuntu 22.04
> sudo apt install -y ant git openjdk-11-jdk
> ## Build and run
> CASSANDRA_USE_JDK11=true ant realclean && CASSANDRA_USE_JDK11=true ant jar &&
> CASSANDRA_USE_JDK11=true ant stress-build && rm -rf data && bin/cassandra -f
> -R
> # Run
> bin/cqlsh -e 'drop table if exists keyspace1.standard1;' && \
> bin/cqlsh -e 'drop keyspace if exists keyspace1;' && \
> bin/nodetool clearsnapshot --all && tools/bin/cassandra-stress write
> n=1000 cl=ONE -rate threads=384 -node 127.0.0.1 -log file=cload.log
> -graph file=cload.html && \
> bin/nodetool compact keyspace1 && sleep 30s && \
> tools/bin/cassandra-stress mixed ratio\(write=10,read=90\) duration=10m
> cl=ONE -rate threads=406 -node localhost -log file=result.log -graph
> file=graph.html
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19483) Catch InvalidLegacyProtocolMagic exceptions
Brad Schoening created CASSANDRA-19483:
--
Summary: Catch InvalidLegacyProtocolMagic exceptions
Key: CASSANDRA-19483
URL: https://issues.apache.org/jira/browse/CASSANDRA-19483
Project: Cassandra
Issue Type: Improvement
Components: Observability/Logging
Reporter: Brad Schoening
Similar to CASSANDRA-18839, we frequently see the exception
InvalidLegacyProtocolMagic with full stack traces flooding our logs.
It seem to be a simple check in Message.java and would be better handled by:
a) return a boolean from validateLegacyProtocolMagic() instead of creating a
custom exception class or
b) add a catch block in HandshakeProtocol.java and return null as is done for
messagingVersion
{code:java}
static Initiate maybeDecode(ByteBuf buf) throws IOException
{
...
try (DataInputBuffer in = new DataInputBuffer(nio, false))
{
validateLegacyProtocolMagic(in.readInt()); // throws exception
int flags = in.readInt();
// legacy pre40 messagingVersion flag
if (getBits(flags, 8, 8) < VERSION_40)
return null;
int minMessagingVersion = getBits(flags, 16, 8);
int maxMessagingVersion = getBits(flags, 24, 8);
// 5.0+ does not support pre40
if (maxMessagingVersion < MessagingService.VERSION_40)
return null;
}
catch (EOFException e)
{
return null;
}
}
{code}
{code:java}
static void validateLegacyProtocolMagic(int magic) throws
InvalidLegacyProtocolMagic
{
if (magic != PROTOCOL_MAGIC)
throw new InvalidLegacyProtocolMagic(magic);
}
{code}
{code:java}
{{2024-03-20 03:47:27,380 [ERROR] [Messaging-EventLoop-3-2] cluster_id=9
ip_address=10.0.0.1 InboundConnectionInitiator.java:360 - Failed to properly
handshake with peer /10.0.2:33356. Closing the channel.}}
{{io.netty.handler.codec.DecoderException:
org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic: Read 1431520594,
Expected -900387334}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:478)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:404)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.channelInputClosed(ByteToMessageDecoder.java:371)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.channelInactive(ByteToMessageDecoder.java:354)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241)}}
{{ at
io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262)}}
{{ at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248)}}
{{ at
io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901)}}
{{ at
io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:819)}}
{{ at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)}}
{{ at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)}}
{{ at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)}}
{{ at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)}}
{{ at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)}}
{{ at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)}}
{{ at java.base/java.lang.Thread.run(Thread.java:834)}}
{{Caused by: org.apache.cassandra.net.Message$InvalidLegacyProtocolMagic: Read
1431520594, Expected -900387334}}
{{ at
org.apache.cassandra.net.Message.validateLegacyProtocolMagic(Message.java:343)}}
{{ at
org.apache.cassandra.net.HandshakeProtocol$Initiate.maybeDecode(HandshakeProtocol.java:167)}}
{{ at
org.apache.cassandra.net.InboundConnectionInitiator$Handler.initiate(InboundConnectionInitiator.java:242)}}
{{ at
org.apache.cassandra.net.InboundConnectionInitiator$Handler.decode(InboundConnectionInitiator.java:235)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508)}}
{{ at
io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447)}}
{{ .
[jira] [Comment Edited] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829250#comment-17829250
]
Ekaterina Dimitrova edited comment on CASSANDRA-19471 at 3/20/24 5:15 PM:
--
{quote}it looks like the 'latest repeat' is now gone and I don't see a way to
trigger it in generate.sh
{quote}
It replaces the offheap which never had a dedicated environment variable
either. We use the REPEATED_DTESTS one for generating repeated jobs for vnode,
no_vnode, latest(former offheap), etc.
{code:java}
.circleci/generate.sh -ps -e
REPEATED_DTESTS=configuration_test.py::TestConfiguration::test_change_durable_writes{code}
was (Author: e.dimitrova):
{quote}it looks like the 'latest repeat' is now gone and I don't see a way to
trigger it in generate.sh
{quote}
It replaces the offheap which never had a dedicated environment variable
either. We use the REPEATED_DTESTS one for vnode, no_vnode, latest(former
offheap), etc.
{code:java}
.circleci/generate.sh -ps -e
REPEATED_DTESTS=configuration_test.py::TestConfiguration::test_change_durable_writes{code}
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829250#comment-17829250
]
Ekaterina Dimitrova commented on CASSANDRA-19471:
-
{quote}it looks like the 'latest repeat' is now gone and I don't see a way to
trigger it in generate.sh
{quote}
It replaces the offheap which never had a dedicated environment variable
either. We use the REPEATED_DTESTS one for vnode, no_vnode, latest(former
offheap), etc.
{code:java}
.circleci/generate.sh -ps -e
REPEATED_DTESTS=configuration_test.py::TestConfiguration::test_change_durable_writes{code}
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19418 - Changes to report additional bulk analytics job stats for instrumentation [cassandra-analytics]
frankgh commented on code in PR #41:
URL:
https://github.com/apache/cassandra-analytics/pull/41#discussion_r1532396813
##
cassandra-analytics-core/src/test/java/org/apache/cassandra/spark/bulkwriter/MockBulkWriterContext.java:
##
@@ -77,6 +78,23 @@ public class MockBulkWriterContext implements
BulkWriterContext, ClusterInfo, Jo
private int sstableDataSizeInMB = 128;
private int sstableWriteBatchSize = 2;
+private final Map jobStats = new HashMap<>();
+
+public Map jobStats()
+{
+return jobStats;
+}
Review Comment:
NIT: missing line break
```suggestion
}
```
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/WriteResult.java:
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.cassandra.spark.bulkwriter;
+
+import java.util.List;
+
+import scala.Serializable;
Review Comment:
shouldn't this be a `java.io.Serializable` instead?
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/SSTableWriter.java:
##
@@ -59,6 +59,9 @@ public class SSTableWriter
private final Map fileDigestMap = new HashMap<>();
private final DigestAlgorithm digestAlgorithm;
+private long rowCount = 0;
+
Review Comment:
NIT: remove extra line break
```suggestion
```
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/SSTableWriter.java:
##
@@ -100,6 +103,15 @@ public void addRow(BigInteger token, Map
boundValues) throws IOE
}
maxToken = token;
cqlSSTableWriter.addRow(boundValues);
+rowCount += 1;
+}
+
+/**
+ * @return the total number of rows added
Review Comment:
```suggestion
* @return the total number of rows written
```
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/RingInstance.java:
##
@@ -125,40 +126,28 @@ private void writeObject(ObjectOutputStream out) throws
IOException
out.writeUTF(ringEntry.address());
out.writeInt(ringEntry.port());
out.writeUTF(ringEntry.datacenter());
-out.writeUTF(ringEntry.load());
Review Comment:
shouldn't we handle nulls during serialization/deserialization instead?
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/WriteResult.java:
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.cassandra.spark.bulkwriter;
+
+import java.util.List;
+
+import scala.Serializable;
+
+public class WriteResult implements Serializable
Review Comment:
let's add javadocs here
##
cassandra-analytics-core/src/main/java/org/apache/cassandra/spark/bulkwriter/WriteResult.java:
##
@@ -0,0 +1,54 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distribute
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
adutra commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532412536
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java:
##
@@ -100,7 +100,10 @@ protected void initializeGauge(
m -> {
MetricId id = getMetricId(m);
Iterable tags = MicrometerTags.toMicrometerTags(id.getTags());
-return Gauge.builder(id.getName(),
supplier).tags(tags).register(registry);
+return Gauge.builder(id.getName(), supplier)
+.strongReference(false)
Review Comment:
This is really clever, but I wonder if we aren't masking a broader problem:
When a session is closed, ideally, I'd like all metrics (session- and
node-level ones) that were registered by it to be explicitly de-registered.
This is not the case currently, and that's imho the real cause why the memory
leak is happening in the first place.
Explicitly de-registering metrics is especially welcome if the sessions are
sharing the same registry, which is the case by default when using Micrometer.
If the registry is not shared, I bet there is no memory leak, but still, it'd
be more correct to de-register everything.
My problem with the fix proposed here is that it will fix the issue for
Micrometer gauges, but what if MicroProfile or Dropwizard change something in
the future that would cause yet another strong reference to be retained by a
metric?
I wonder if we shouldn't investigate introducing a new method `void
deregisterMetrics()` in `MetricUpdater`.
Then we could modify `DefaultSession` and add this to
`DefaultSession.close()` and `forceClose()`:
```java
metricUpdater.deregisterMetrics();
for (Node n : metadataManager.getMetadata().getNodes().values()) {
((DefaultNode) n).getMetricUpdater().deregisterMetrics();
}
```
The implementation for `MicrometerMetricUpdater` should be trivial (and for
other impls too, I guess):
```java
@Override
public void deregisterMetrics() {
metrics.values().forEach(registry::remove);
}
```
But obviously, the devil is likely hiding in the details :-)
I'd be curious to hear what others think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] CASSANDRA-19457: Object reference in Micrometer metrics prevent GC from reclaiming Session instances [cassandra-java-driver]
adutra commented on code in PR #1916:
URL:
https://github.com/apache/cassandra-java-driver/pull/1916#discussion_r1532412536
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java:
##
@@ -100,7 +100,10 @@ protected void initializeGauge(
m -> {
MetricId id = getMetricId(m);
Iterable tags = MicrometerTags.toMicrometerTags(id.getTags());
-return Gauge.builder(id.getName(),
supplier).tags(tags).register(registry);
+return Gauge.builder(id.getName(), supplier)
+.strongReference(false)
Review Comment:
This is really clever, but I wonder if we aren't masking a broader problem:
When a session is closed, ideally, I'd like all metrics (session- and
node-level ones) that were registered by it to be explicitly de-registered.
This is not the case currently, and that's imho the real cause why the memory
leak is happening in the first place.
Explicitly de-registering metrics is especially welcome if the sessions are
sharing the same registry, which is the case by default when using Micrometer.
If the registry is not shared, I bet there is no memory leak, but still, it'd
be more correct to de-register everything.
My problem with the fix proposed here is that it will fix the issue for
Micrometer gauges, but what if MicroProfile or Dropwizard change something in
the future that would cause yet another strong reference to be retained by a
metric?
I wonder if we shouldn't investigate introducing a new method `void
deregisterMetrics()` in `MetricUpdater`.
Then we could modify `DefaultSession` and add this to
`DefaultSession.close()` and `forceClose()`:
```java
metricUpdater.deregisterMetrics();
metadataManager
.getMetadata()
.getNodes()
.values()
.forEach(n -> ((DefaultNode) n).getMetricUpdater().deregisterMetrics());
```
The implementation for `MicrometerMetricUpdater` should be trivial (and for
other impls too, I guess):
```java
@Override
public void deregisterMetrics() {
metrics.values().forEach(registry::remove);
}
```
But obviously, the devil is likely hiding in the details :-)
I'd be curious to hear what others think.
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdater.java:
##
@@ -31,13 +31,18 @@
import io.micrometer.core.instrument.Timer;
import java.time.Duration;
import java.util.Set;
+import java.util.function.Supplier;
import net.jcip.annotations.ThreadSafe;
@ThreadSafe
public class MicrometerNodeMetricUpdater extends
MicrometerMetricUpdater
implements NodeMetricUpdater {
private final Node node;
+ private final Supplier openConnectionsSupplier;
Review Comment:
Why create fields for these? They are not referenced anywhere. The original
class was fine imho.
##
metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdater.java:
##
@@ -30,22 +30,28 @@
import io.micrometer.core.instrument.Timer;
import java.time.Duration;
import java.util.Set;
+import java.util.function.Supplier;
import net.jcip.annotations.ThreadSafe;
@ThreadSafe
public class MicrometerSessionMetricUpdater extends
MicrometerMetricUpdater
implements SessionMetricUpdater {
+ private final Supplier connectedNodesSupplier = this::connectedNodes;
Review Comment:
Same here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829242#comment-17829242
]
Brandon Williams commented on CASSANDRA-19471:
--
I doubt this is a home run since I am not very familiar with this code at all,
but the stab I took
[here|https://github.com/driftx/cassandra/commits/CASSANDRA-19471-5.0/] does
fix the test. Which I would prove on circle except it looks like the 'latest
repeat' is now gone and I don't see a way to trigger it in generate.sh.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829239#comment-17829239
]
Stefan Miklosovic commented on CASSANDRA-19471:
---
I am not planning to be involved. Feel free to take it. Thank you very much.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19042) Repair fuzz tests fail with paxos_variant: v2
[
https://issues.apache.org/jira/browse/CASSANDRA-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829219#comment-17829219
]
Ekaterina Dimitrova commented on CASSANDRA-19042:
-
CI ended up green for 5.0. As paxos_variant was not changed in the latest
configuration to v2 as part of CASSANDRA-18753, I changed it locally and reran
all the mentioned failing tests.
They all finished successfully, too.
I will start reviewing it in the afternoon.
> Repair fuzz tests fail with paxos_variant: v2
> -
>
> Key: CASSANDRA-19042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19042
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Feature/Lightweight Transactions,
> Test/fuzz
>Reporter: Branimir Lambov
>Assignee: David Capwell
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Adding {{paxos_variant: v2}} to the test yaml causes all fuzz repair tests to
> fail with
> {code}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:337)
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:300)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:176)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:147)
> at
> org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1067)
> at org.apache.cassandra.net.Message.payloadSize(Message.java:1114)
> at
> org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:750)
> at org.apache.cassandra.net.Message.serializedSize(Message.java:1094)
> ...
> {code}
> This happens for all three options of {{paxos_state_purging}} and both with
> and without {{storage_compatibility_mode: NONE}}.
> Tests still fail if {{PaxosStartPrepareCleanup}} is changed to use
> {{EndpointState.nullableSerializer}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19042) Repair fuzz tests fail with paxos_variant: v2
[
https://issues.apache.org/jira/browse/CASSANDRA-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829219#comment-17829219
]
Ekaterina Dimitrova edited comment on CASSANDRA-19042 at 3/20/24 3:56 PM:
--
CI ended up green for 5.0. As paxos_variant was not changed in the latest
configuration to v2 as part of CASSANDRA-18753, I changed it locally and reran
all the mentioned failing tests with the applied patch.
They all finished successfully, too.
I will start reviewing it in the afternoon.
was (Author: e.dimitrova):
CI ended up green for 5.0. As paxos_variant was not changed in the latest
configuration to v2 as part of CASSANDRA-18753, I changed it locally and reran
all the mentioned failing tests.
They all finished successfully, too.
I will start reviewing it in the afternoon.
> Repair fuzz tests fail with paxos_variant: v2
> -
>
> Key: CASSANDRA-19042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19042
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Feature/Lightweight Transactions,
> Test/fuzz
>Reporter: Branimir Lambov
>Assignee: David Capwell
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Adding {{paxos_variant: v2}} to the test yaml causes all fuzz repair tests to
> fail with
> {code}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:337)
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:300)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:176)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:147)
> at
> org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1067)
> at org.apache.cassandra.net.Message.payloadSize(Message.java:1114)
> at
> org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:750)
> at org.apache.cassandra.net.Message.serializedSize(Message.java:1094)
> ...
> {code}
> This happens for all three options of {{paxos_state_purging}} and both with
> and without {{storage_compatibility_mode: NONE}}.
> Tests still fail if {{PaxosStartPrepareCleanup}} is changed to use
> {{EndpointState.nullableSerializer}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19007) Queries with multi-column replica-side filtering can miss rows
[
https://issues.apache.org/jira/browse/CASSANDRA-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829211#comment-17829211
]
Caleb Rackliffe edited comment on CASSANDRA-19007 at 3/20/24 3:26 PM:
--
[~Bereng] I wouldn't say that an actual solution (which I had [started to
explore|https://github.com/apache/cassandra/pull/3155] along w/
CASSANDRA-19018) should block 5.0, as this has been broken since the beginning
of time for normal filtering queries. What I think we should do at this point
is just spin off a separate Jira to emit a client warning/put a guardrail in
place if a user attempts to do a read that involves filtering (without SAI,
which is fixed, or at least without an index) on multiple mutable/regular
columns at a consistency level that requires coordinator resolution.
We can keep this Jira to track an actual fix, which honestly might not be that
bad, given how much of the dirty work CASSANDRA-19018 has already done, in
terms of testing infrastructure, all the fixes to RFP, and replica awareness of
read CL.
If that all makes sense, feel free to throw up that Jira, or let me know if
you'd like me to, and I can.
was (Author: maedhroz):
[~Bereng] I wouldn't say that an actual solution (which I had [started to
explore|https://github.com/apache/cassandra/pull/3155] along w/
CASSANDRA-19018) should block 5.0, as this has been broken since the beginning
of time for normal filtering queries. What I think we should do at this point
is just spin off a separate Jira to emit a client warning/put a guardrail in
place if a user attempts to do a read that involves filtering (without SAI,
which is fixed, or at least without an index) on multiple mutable/regular
columns at a consistency level that requires coordinator resolution.
We can keep this Jira to track an actual fix, which honestly might not be that
bad, given how much of the dirty work CASSANDRA-19018 has already done, in
terms of testing infrastructure and all the fixes to RFP.
If that all makes sense, feel free to throw up that Jira, or let me know if
you'd like me to, and I can.
> Queries with multi-column replica-side filtering can miss rows
> --
>
> Key: CASSANDRA-19007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19007
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{SELECT}} queries with multi-column replica-side filtering can miss rows if
> the filtered columns are spread across out-of-sync replicas. This dtest
> reproduces the issue:
> {code:java}
> @Test
> public void testMultiColumnReplicaSideFiltering() throws IOException
> {
> try (Cluster cluster = init(Cluster.build().withNodes(2).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int PRIMARY
> KEY, a int, b int)"));
> // insert a split row
> cluster.get(1).executeInternal(withKeyspace("INSERT INTO %s.t(k, a)
> VALUES (0, 1)"));
> cluster.get(2).executeInternal(withKeyspace("INSERT INTO %s.t(k, b)
> VALUES (0, 2)"));
> String select = withKeyspace("SELECT * FROM %s.t WHERE a = 1 AND b =
> 2 ALLOW FILTERING");
> Object[][] initialRows = cluster.coordinator(1).execute(select, ALL);
> assertRows(initialRows, row(0, 1, 2)); // not found!!
> }
> }
> {code}
> This edge case affects queries using {{ALLOW FILTERING}} or any index
> implementation.
> It affects all branches since multi-column replica-side filtering queries
> were introduced, long before 3.0.
> The protection mechanism added by CASSANDRA-8272/8273 won't deal with this
> case, since it only solves single-column conflicts where stale rows could
> resurrect. This bug however doesn't resurrect data, it can only miss rows
> while the replicas are out-of-sync.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19007) Queries with multi-column replica-side filtering can miss rows
[
https://issues.apache.org/jira/browse/CASSANDRA-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829211#comment-17829211
]
Caleb Rackliffe commented on CASSANDRA-19007:
-
[~Bereng] I wouldn't say that an actual solution (which I had [started to
explore|https://github.com/apache/cassandra/pull/3155] along w/
CASSANDRA-19018) should block 5.0, as this has been broken since the beginning
of time for normal filtering queries. What I think we should do at this point
is just spin off a separate Jira to emit a client warning/put a guardrail in
place if a user attempts to do a read that involves filtering (without SAI,
which is fixed, or at least without an index) on multiple mutable/regular
columns at a consistency level that requires coordinator resolution.
We can keep this Jira to track an actual fix, which honestly might not be that
bad, given how much of the dirty work CASSANDRA-19018 has already done, in
terms of testing infrastructure and all the fixes to RFP.
If that all makes sense, feel free to throw up that Jira, or let me know if
you'd like me to, and I can.
> Queries with multi-column replica-side filtering can miss rows
> --
>
> Key: CASSANDRA-19007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19007
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{SELECT}} queries with multi-column replica-side filtering can miss rows if
> the filtered columns are spread across out-of-sync replicas. This dtest
> reproduces the issue:
> {code:java}
> @Test
> public void testMultiColumnReplicaSideFiltering() throws IOException
> {
> try (Cluster cluster = init(Cluster.build().withNodes(2).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int PRIMARY
> KEY, a int, b int)"));
> // insert a split row
> cluster.get(1).executeInternal(withKeyspace("INSERT INTO %s.t(k, a)
> VALUES (0, 1)"));
> cluster.get(2).executeInternal(withKeyspace("INSERT INTO %s.t(k, b)
> VALUES (0, 2)"));
> String select = withKeyspace("SELECT * FROM %s.t WHERE a = 1 AND b =
> 2 ALLOW FILTERING");
> Object[][] initialRows = cluster.coordinator(1).execute(select, ALL);
> assertRows(initialRows, row(0, 1, 2)); // not found!!
> }
> }
> {code}
> This edge case affects queries using {{ALLOW FILTERING}} or any index
> implementation.
> It affects all branches since multi-column replica-side filtering queries
> were introduced, long before 3.0.
> The protection mechanism added by CASSANDRA-8272/8273 won't deal with this
> case, since it only solves single-column conflicts where stale rows could
> resurrect. This bug however doesn't resurrect data, it can only miss rows
> while the replicas are out-of-sync.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19471) Commitlog with direct io fails test_change_durable_writes
[
https://issues.apache.org/jira/browse/CASSANDRA-19471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829201#comment-17829201
]
Ekaterina Dimitrova commented on CASSANDRA-19471:
-
[~blambov] , thank you for looking into this and for providing pointers.
[~smiklosovic] , [~Bereng] , [~brandon.williams] , does anyone of you plan to
look into it?
I can take it tomorrow if no one assigns it, but I wanted to confirm no one is
on top of it yet, as I see several people commenting, etc.
> Commitlog with direct io fails test_change_durable_writes
> -
>
> Key: CASSANDRA-19471
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19471
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log
>Reporter: Brandon Williams
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
>
> With the commitlog_disk_access_mode set to direct, and the improved
> configuration_test.py::TestConfiguration::test_change_durable_writes from
> CASSANDRA-19465, this fails with either:
> {noformat}
> AssertionError: Commitlog was written with durable writes disabled
> {noformat}
> Or what appears to be the original exception reported in CASSANDRA-19465:
> {noformat}
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,465
> StorageService.java:631 - Stopping native transport
> node1: ERROR [MutationStage-5] 2024-03-14 17:16:08,465
> StorageProxy.java:1670 - Failed to apply mutation locally :
> java.lang.IllegalArgumentException: newPosition > limit: (1048634 > 1048576)
> at java.base/java.nio.Buffer.createPositionException(Buffer.java:341)
> at java.base/java.nio.Buffer.position(Buffer.java:316)
> at java.base/java.nio.ByteBuffer.position(ByteBuffer.java:1516)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:321)
> at
> java.base/java.nio.MappedByteBuffer.position(MappedByteBuffer.java:73)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegment.allocate(CommitLogSegment.java:216)
> at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerStandard.allocate(CommitLogSegmentManagerStandard.java:52)
> at org.apache.cassandra.db.commitlog.CommitLog.add(CommitLog.java:307)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.addToCommitLog(CassandraKeyspaceWriteHandler.java:99)
> at
> org.apache.cassandra.db.CassandraKeyspaceWriteHandler.beginWrite(CassandraKeyspaceWriteHandler.java:53)
> at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:612)
> at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:497)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:244)
> at org.apache.cassandra.db.Mutation.apply(Mutation.java:264)
> at
> org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1664)
> at
> org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2624)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:833)
> node1: ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2024-03-14 17:16:08,470
> StorageService.java:636 - Stopping gossiper
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19042) Repair fuzz tests fail with paxos_variant: v2
[
https://issues.apache.org/jira/browse/CASSANDRA-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829196#comment-17829196
]
Ekaterina Dimitrova edited comment on CASSANDRA-19042 at 3/20/24 2:57 PM:
--
This is one of our last 5.0 blockers.
To help expedite the work, I rebased the patch [here
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-19042], just
started min CI run -
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-19042]
If nothing shows up from CI's perspective, I will do a first pass of review,
Blake Eggleston; I hope you can still look at it, too.
was (Author: e.dimitrova):
This is one of our last 5.0 blockers.
To help expedite the work, I rebased the patch [here
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-19042], just
started min CI run -
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-19042]
If nothing shows up I will do a first pass of review, [~bdeggleston], I hope
you can still take a look at it too.
> Repair fuzz tests fail with paxos_variant: v2
> -
>
> Key: CASSANDRA-19042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19042
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Feature/Lightweight Transactions,
> Test/fuzz
>Reporter: Branimir Lambov
>Assignee: David Capwell
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Adding {{paxos_variant: v2}} to the test yaml causes all fuzz repair tests to
> fail with
> {code}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:337)
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:300)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:176)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:147)
> at
> org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1067)
> at org.apache.cassandra.net.Message.payloadSize(Message.java:1114)
> at
> org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:750)
> at org.apache.cassandra.net.Message.serializedSize(Message.java:1094)
> ...
> {code}
> This happens for all three options of {{paxos_state_purging}} and both with
> and without {{storage_compatibility_mode: NONE}}.
> Tests still fail if {{PaxosStartPrepareCleanup}} is changed to use
> {{EndpointState.nullableSerializer}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19042) Repair fuzz tests fail with paxos_variant: v2
[
https://issues.apache.org/jira/browse/CASSANDRA-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829196#comment-17829196
]
Ekaterina Dimitrova edited comment on CASSANDRA-19042 at 3/20/24 2:57 PM:
--
This is one of our last 5.0 blockers.
To help expedite the work, I rebased the patch [here
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-19042], just
started min CI run -
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-19042]
If nothing shows up from CI's perspective, I will do a first pass of review.
[~bdeggleston], I hope you can still look at it, too.
was (Author: e.dimitrova):
This is one of our last 5.0 blockers.
To help expedite the work, I rebased the patch [here
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-19042], just
started min CI run -
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-19042]
If nothing shows up from CI's perspective, I will do a first pass of review,
Blake Eggleston; I hope you can still look at it, too.
> Repair fuzz tests fail with paxos_variant: v2
> -
>
> Key: CASSANDRA-19042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19042
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Feature/Lightweight Transactions,
> Test/fuzz
>Reporter: Branimir Lambov
>Assignee: David Capwell
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Adding {{paxos_variant: v2}} to the test yaml causes all fuzz repair tests to
> fail with
> {code}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:337)
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:300)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:176)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:147)
> at
> org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1067)
> at org.apache.cassandra.net.Message.payloadSize(Message.java:1114)
> at
> org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:750)
> at org.apache.cassandra.net.Message.serializedSize(Message.java:1094)
> ...
> {code}
> This happens for all three options of {{paxos_state_purging}} and both with
> and without {{storage_compatibility_mode: NONE}}.
> Tests still fail if {{PaxosStartPrepareCleanup}} is changed to use
> {{EndpointState.nullableSerializer}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19042) Repair fuzz tests fail with paxos_variant: v2
[
https://issues.apache.org/jira/browse/CASSANDRA-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-19042:
Reviewers: Blake Eggleston, Ekaterina Dimitrova, Ekaterina Dimitrova (was:
Blake Eggleston)
Blake Eggleston, Ekaterina Dimitrova, Ekaterina Dimitrova (was:
Blake Eggleston, Ekaterina Dimitrova)
Status: Review In Progress (was: Patch Available)
> Repair fuzz tests fail with paxos_variant: v2
> -
>
> Key: CASSANDRA-19042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19042
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Feature/Lightweight Transactions,
> Test/fuzz
>Reporter: Branimir Lambov
>Assignee: David Capwell
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Adding {{paxos_variant: v2}} to the test yaml causes all fuzz repair tests to
> fail with
> {code}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:337)
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:300)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:176)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:147)
> at
> org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1067)
> at org.apache.cassandra.net.Message.payloadSize(Message.java:1114)
> at
> org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:750)
> at org.apache.cassandra.net.Message.serializedSize(Message.java:1094)
> ...
> {code}
> This happens for all three options of {{paxos_state_purging}} and both with
> and without {{storage_compatibility_mode: NONE}}.
> Tests still fail if {{PaxosStartPrepareCleanup}} is changed to use
> {{EndpointState.nullableSerializer}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19042) Repair fuzz tests fail with paxos_variant: v2
[
https://issues.apache.org/jira/browse/CASSANDRA-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829196#comment-17829196
]
Ekaterina Dimitrova commented on CASSANDRA-19042:
-
This is one of our last 5.0 blockers.
To help expedite the work, I rebased the patch [here
|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-19042], just
started min CI run -
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=CASSANDRA-19042]
If nothing shows up I will do a first pass of review, [~bdeggleston], I hope
you can still take a look at it too.
> Repair fuzz tests fail with paxos_variant: v2
> -
>
> Key: CASSANDRA-19042
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19042
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Feature/Lightweight Transactions,
> Test/fuzz
>Reporter: Branimir Lambov
>Assignee: David Capwell
>Priority: Normal
> Fix For: 5.0-rc, 5.x
>
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> Adding {{paxos_variant: v2}} to the test yaml causes all fuzz repair tests to
> fail with
> {code}
> java.lang.NullPointerException: null
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:337)
> at
> org.apache.cassandra.gms.EndpointStateSerializer.serializedSize(EndpointState.java:300)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:176)
> at
> org.apache.cassandra.service.paxos.cleanup.PaxosStartPrepareCleanup$RequestSerializer.serializedSize(PaxosStartPrepareCleanup.java:147)
> at
> org.apache.cassandra.net.Message$Serializer.payloadSize(Message.java:1067)
> at org.apache.cassandra.net.Message.payloadSize(Message.java:1114)
> at
> org.apache.cassandra.net.Message$Serializer.serializedSize(Message.java:750)
> at org.apache.cassandra.net.Message.serializedSize(Message.java:1094)
> ...
> {code}
> This happens for all three options of {{paxos_state_purging}} and both with
> and without {{storage_compatibility_mode: NONE}}.
> Tests still fail if {{PaxosStartPrepareCleanup}} is changed to use
> {{EndpointState.nullableSerializer}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19482) Simplify metadata log implementation using custom partitioner
Sam Tunnicliffe created CASSANDRA-19482:
---
Summary: Simplify metadata log implementation using custom
partitioner
Key: CASSANDRA-19482
URL: https://issues.apache.org/jira/browse/CASSANDRA-19482
Project: Cassandra
Issue Type: Improvement
Components: Transactional Cluster Metadata
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
The distributed metadata log table can be simplified by leveraging the fact
that replicas are all responsible for the entire token range. Given this
assumption, we can then use {{ReversedLongLocalPartitioner}} introduced in
CASSANDRA-19391 to make it much easier to append to/read from the tail of the
log, effectively removing the need for the {{Period}} construct. This will also
improve apply to the local metadata log used at startup.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19193) Reimplement ClusterMetadata::writePlacementAllSettled
[ https://issues.apache.org/jira/browse/CASSANDRA-19193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828963#comment-17828963 ] Alex Petrov commented on CASSANDRA-19193: - +1 Do you think it might be worth to add a small simulation where we'd add as many things "in flight" as we can, and then compare settled metadata with one that'd execute items one by one? > Reimplement ClusterMetadata::writePlacementAllSettled > - > > Key: CASSANDRA-19193 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19193 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 5.1-alpha1 > > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 10m > Remaining Estimate: 0h > > This should step through InProgressSequences to determine state when > finished, rather than emulating the logic inline. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19481) The repository 'https://debian.cassandra.apache.org 311x Release' does not have a Release file.
[
https://issues.apache.org/jira/browse/CASSANDRA-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828958#comment-17828958
]
Brandon Williams commented on CASSANDRA-19481:
--
Yeah, that's why I suggested trying to upgrade ca-certificates from a higher
version, but upgrading all the way is even better. Glad you got it working!
> The repository 'https://debian.cassandra.apache.org 311x Release' does not
> have a Release file.
> ---
>
> Key: CASSANDRA-19481
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19481
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Martin Schröder
>Assignee: Brandon Williams
>Priority: Normal
>
> The debian repo for 311 can't be used anymore; apt update fails with
> {color:#00}Ign:10
> [https://debian.cassandra.apache.org|https://debian.cassandra.apache.org/]
> 311x InRelease {color}
> Err:12
> [https://debian.cassandra.apache.org|https://debian.cassandra.apache.org/]
> 311x Release
> server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none
> Reading package lists... Done{color:#b26818} {color}{color:#00} {color}
> E: The repository 'https://debian.cassandra.apache.org 311x Release' does not
> have a Release file.
> N: Updating from such a repository can't be done securely, and is therefore
> disabled by default.
> N: See apt-secure(8) manpage for repository creation and user configuration
> details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19481) The repository 'https://debian.cassandra.apache.org 311x Release' does not have a Release file.
[
https://issues.apache.org/jira/browse/CASSANDRA-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828952#comment-17828952
]
Martin Schröder commented on CASSANDRA-19481:
-
It seems debian stretch doesn't know the ca for the certs of jfrog anymore. We
upgraded the system to debian 11 and anything is well.
> The repository 'https://debian.cassandra.apache.org 311x Release' does not
> have a Release file.
> ---
>
> Key: CASSANDRA-19481
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19481
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Martin Schröder
>Assignee: Brandon Williams
>Priority: Normal
>
> The debian repo for 311 can't be used anymore; apt update fails with
> {color:#00}Ign:10
> [https://debian.cassandra.apache.org|https://debian.cassandra.apache.org/]
> 311x InRelease {color}
> Err:12
> [https://debian.cassandra.apache.org|https://debian.cassandra.apache.org/]
> 311x Release
> server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none
> Reading package lists... Done{color:#b26818} {color}{color:#00} {color}
> E: The repository 'https://debian.cassandra.apache.org 311x Release' does not
> have a Release file.
> N: Updating from such a repository can't be done securely, and is therefore
> disabled by default.
> N: See apt-secure(8) manpage for repository creation and user configuration
> details.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19477) Do not go to disk to get HintsStore.getTotalFileSize
[
https://issues.apache.org/jira/browse/CASSANDRA-19477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Yeschenko updated CASSANDRA-19477:
--
Reviewers: Aleksey Yeschenko
> Do not go to disk to get HintsStore.getTotalFileSize
>
>
> Key: CASSANDRA-19477
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19477
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Hints
>Reporter: Jon Haddad
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 5.x
>
> Attachments: flamegraph.cpu.html
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> When testing a cluster with more requests than it could handle, I noticed
> significant CPU time (25%) spent in HintsStore.getTotalFileSize. Here's what
> I'm seeing from profiling:
> 10% of CPU time spent in HintsDescriptor.fileName which only does this:
>
> {noformat}
> return String.format("%s-%s-%s.hints", hostId, timestamp, version);{noformat}
> At a bare minimum here we should create this string up front with the host
> and version and eliminate 2 of the 3 substitutions, but I think it's probably
> faster to use a StringBuilder and avoid the underlying regular expression
> altogether.
> 12% of the time is spent in org.apache.cassandra.io.util.File.length. It
> looks like this is called once for each hint file on disk for each host we're
> hinting to. In the case of an overloaded cluster, this is significant. It
> would be better if we were to track the file size in memory for each hint
> file and reference that rather than go to the filesystem.
> These fairly small changes should make Cassandra more reliable when under
> load spikes.
> CPU Flame graph attached.
> I only tested this in 4.1 but it looks like this is present up to trunk.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19153) Preclude irrecoverable log corruption in case split-brain situation during leader election with absent seeds
[ https://issues.apache.org/jira/browse/CASSANDRA-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828911#comment-17828911 ] Alex Petrov commented on CASSANDRA-19153: - +1 with minor comment. > Preclude irrecoverable log corruption in case split-brain situation during > leader election with absent seeds > > > Key: CASSANDRA-19153 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19153 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Alex Petrov >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 5.x > > Attachments: ci_summary.html, result_details.tar.gz > > Time Spent: 20m > Remaining Estimate: 0h > > It should be possible to detect a scenario where two partitioned nodes > independently elect themselves as the first CMS nodes in a brand new cluster. > In such a case, metadata changes should not be applied so that the conflict > can be resolved. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19007) Queries with multi-column replica-side filtering can miss rows
[
https://issues.apache.org/jira/browse/CASSANDRA-19007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17828636#comment-17828636
]
Berenguer Blasi commented on CASSANDRA-19007:
-
Just doublechecking: this is _not_ a 5.0 blocker correct?
> Queries with multi-column replica-side filtering can miss rows
> --
>
> Key: CASSANDRA-19007
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19007
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Andres de la Peña
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{SELECT}} queries with multi-column replica-side filtering can miss rows if
> the filtered columns are spread across out-of-sync replicas. This dtest
> reproduces the issue:
> {code:java}
> @Test
> public void testMultiColumnReplicaSideFiltering() throws IOException
> {
> try (Cluster cluster = init(Cluster.build().withNodes(2).start()))
> {
> cluster.schemaChange(withKeyspace("CREATE TABLE %s.t (k int PRIMARY
> KEY, a int, b int)"));
> // insert a split row
> cluster.get(1).executeInternal(withKeyspace("INSERT INTO %s.t(k, a)
> VALUES (0, 1)"));
> cluster.get(2).executeInternal(withKeyspace("INSERT INTO %s.t(k, b)
> VALUES (0, 2)"));
> String select = withKeyspace("SELECT * FROM %s.t WHERE a = 1 AND b =
> 2 ALLOW FILTERING");
> Object[][] initialRows = cluster.coordinator(1).execute(select, ALL);
> assertRows(initialRows, row(0, 1, 2)); // not found!!
> }
> }
> {code}
> This edge case affects queries using {{ALLOW FILTERING}} or any index
> implementation.
> It affects all branches since multi-column replica-side filtering queries
> were introduced, long before 3.0.
> The protection mechanism added by CASSANDRA-8272/8273 won't deal with this
> case, since it only solves single-column conflicts where stale rows could
> resurrect. This bug however doesn't resurrect data, it can only miss rows
> while the replicas are out-of-sync.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19458) Minor bugs in generate.sh -d
[ https://issues.apache.org/jira/browse/CASSANDRA-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-19458: Fix Version/s: 5.0-rc Since Version: 5.0-rc Source Control Link: https://github.com/apache/cassandra/commit/c3a18e36d12a9fa55aa1a0687d6a58b5ebebe768 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Minor bugs in generate.sh -d > > > Key: CASSANDRA-19458 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19458 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 5.0-rc > > > Option -d does not work if you don't skip repetition of tests. It should > error out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19458) Minor bugs in generate.sh -d
[ https://issues.apache.org/jira/browse/CASSANDRA-19458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-19458: Status: Ready to Commit (was: Review In Progress) > Minor bugs in generate.sh -d > > > Key: CASSANDRA-19458 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19458 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > > Option -d does not work if you don't skip repetition of tests. It should > error out. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (8f7c23be85 -> 32b146dc63)
This is an automated email from the ASF dual-hosted git repository. bereng pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 8f7c23be85 Merge branch 'cassandra-5.0' into trunk new c3a18e36d1 Minor bugs in generate.sh -d new 32b146dc63 Merge branch 'cassandra-5.0' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .circleci/config.yml | 81 --- .circleci/config.yml.FREE | 81 --- .circleci/config.yml.PAID | 81 --- .circleci/generate.sh | 47 ++- 4 files changed, 25 insertions(+), 265 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. bereng pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 32b146dc632341124f8da9bef69de5cdf3bd2d0e Merge: 8f7c23be85 c3a18e36d1 Author: Bereng AuthorDate: Wed Mar 20 08:07:39 2024 +0100 Merge branch 'cassandra-5.0' into trunk * cassandra-5.0: Minor bugs in generate.sh -d .circleci/config.yml | 81 --- .circleci/config.yml.FREE | 81 --- .circleci/config.yml.PAID | 81 --- .circleci/generate.sh | 47 ++- 4 files changed, 25 insertions(+), 265 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated: Minor bugs in generate.sh -d
This is an automated email from the ASF dual-hosted git repository. bereng pushed a commit to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cassandra-5.0 by this push: new c3a18e36d1 Minor bugs in generate.sh -d c3a18e36d1 is described below commit c3a18e36d12a9fa55aa1a0687d6a58b5ebebe768 Author: Bereng AuthorDate: Tue Mar 12 08:52:53 2024 +0100 Minor bugs in generate.sh -d patch by Berenguer Blasi; reviewed by Ekaterina Dimitrova for CASSANDRA-19458 --- .circleci/config.yml | 81 --- .circleci/config.yml.FREE | 81 --- .circleci/config.yml.PAID | 81 --- .circleci/generate.sh | 47 ++- 4 files changed, 25 insertions(+), 265 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index 4fce7a0fef..8ac77b306c 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -9499,12 +9499,6 @@ workflows: requires: - start_j11_jvm_dtests_latest_vnode - j11_build -- start_j11_jvm_dtests_latest_vnode_repeat: -type: approval -- j11_jvm_dtests_latest_vnode_repeat: -requires: -- start_j11_jvm_dtests_latest_vnode_repeat -- j11_build - start_j17_jvm_dtests: type: approval - j17_jvm_dtests: @@ -9517,12 +9511,6 @@ workflows: requires: - start_j17_jvm_dtests_latest_vnode - j11_build -- start_j17_jvm_dtests_latest_vnode_repeat: -type: approval -- j17_jvm_dtests_latest_vnode_repeat: -requires: -- start_j17_jvm_dtests_latest_vnode_repeat -- j11_build - start_j11_simulator_dtests: type: approval - j11_simulator_dtests: @@ -9619,18 +9607,6 @@ workflows: requires: - start_j17_utests_latest - j11_build -- start_j11_utests_latest_repeat: -type: approval -- j11_utests_latest_repeat: -requires: -- start_j11_utests_latest_repeat -- j11_build -- start_j17_utests_latest_repeat: -type: approval -- j17_utests_latest_repeat: -requires: -- start_j17_utests_latest_repeat -- j11_build - start_j11_utests_stress: type: approval - j11_utests_stress: @@ -9697,12 +9673,6 @@ workflows: requires: - start_j11_dtests_latest - j11_build -- start_j11_dtests_latest_repeat: -type: approval -- j11_dtests_latest_repeat: -requires: -- start_j11_dtests_latest_repeat -- j11_build - start_j17_dtests: type: approval - j17_dtests: @@ -9721,12 +9691,6 @@ workflows: requires: - start_j17_dtests_latest - j11_build -- start_j17_dtests_latest_repeat: -type: approval -- j17_dtests_latest_repeat: -requires: -- start_j17_dtests_latest_repeat -- j11_build - start_j11_dtests_large: type: approval - j11_dtests_large: @@ -9829,9 +9793,6 @@ workflows: - j11_utests_latest: requires: - j11_build -- j11_utests_latest_repeat: -requires: -- j11_build - j11_simulator_dtests: requires: - j11_build @@ -9841,18 +9802,12 @@ workflows: - j11_jvm_dtests_latest_vnode: requires: - j11_build -- j11_jvm_dtests_latest_vnode_repeat: -requires: -- j11_build - j17_jvm_dtests: requires: - j11_build - j17_jvm_dtests_latest_vnode: requires: - j11_build -- j17_jvm_dtests_latest_vnode_repeat: -requires: -- j11_build - j11_cqlshlib_tests: requires: - j11_build @@ -9874,9 +9829,6 @@ workflows: - j17_utests_latest: requires: - j11_build -- j17_utests_latest_repeat: -requires: -- j11_build - start_utests_long: type: approval - j11_utests_long: @@ -9954,9 +9906,6 @@ workflows: - j11_dtests_latest: requires: - j11_build -- j11_dtests_latest_repeat: -requires: -- j11_build - j17_dtests: requires: - j11_build @@ -9966,9 +9915,6 @@ workflows: - j17_dtests_latest: requires: - j11_build -- j17_dtests_latest_repeat: -requires: -- j11_build - start_j11_dtests_large: type: approval - j11_dtests_large: @@ -10064,12 +10010,6 @@ workflows: requires: - start_j17_jvm_dtests_latest_vnode - j17_build -- start_j17_jvm_dtests_latest_vnode_repeat: -type: approval -- j17_jvm_dtests_latest_vnode_repeat: -requires: -- start_j17_jvm_dtests_latest_vnode_repeat -- j17_build - start_j17_cqlshlib_tests: type: approval
