[jira] [Commented] (CASSANDRA-17140) Broken test_rolling_upgrade - upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x
[
https://issues.apache.org/jira/browse/CASSANDRA-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456901#comment-17456901
]
Berenguer Blasi commented on CASSANDRA-17140:
-
Aha that's sthg new... and weird it doesn't match local repro.
> Broken test_rolling_upgrade -
> upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x
>
>
> Key: CASSANDRA-17140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17140
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Yifan Cai
>Priority: Normal
> Fix For: 4.0.x
>
>
> The tests "test_rolling_upgrade" fail with the below error.
>
> [https://app.circleci.com/pipelines/github/yifan-c/cassandra/279/workflows/6340cd42-0b27-42c2-8418-9f8b56c57bea/jobs/1990]
>
> I am able to alway produce it by running the test locally too.
> {{$ pytest --execute-upgrade-tests-only --upgrade-target-version-only
> --upgrade-version-selection all --cassandra-version=4.0
> upgrade_tests/upgrade_through_versions_test.py::TestUpgrade_indev_3_11_x_To_indev_4_0_x::test_rolling_upgrade}}
>
> {code:java}
> self =
> object at 0x7ffba4242fd0>
> subprocs = [,
> ]
> def _check_on_subprocs(self, subprocs):
> """
> Check on given subprocesses.
>
> If any are not alive, we'll go ahead and terminate any remaining
> alive subprocesses since this test is going to fail.
> """
> subproc_statuses = [s.is_alive() for s in subprocs]
> if not all(subproc_statuses):
> message = "A subprocess has terminated early. Subprocess
> statuses: "
> for s in subprocs:
> message += "{name} (is_alive: {aliveness}),
> ".format(name=s.name, aliveness=s.is_alive())
> message += "attempting to terminate remaining subprocesses now."
> self._terminate_subprocs()
> > raise RuntimeError(message)
> E RuntimeError: A subprocess has terminated early. Subprocess
> statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting
> to terminate remaining subprocesses now.{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17140) Broken test_rolling_upgrade - upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x
[
https://issues.apache.org/jira/browse/CASSANDRA-17140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456867#comment-17456867
]
Ekaterina Dimitrova commented on CASSANDRA-17140:
-
As we talked, I tested also in CircleCI:
*
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1242/workflows/2aa40aad-f174-413c-8c9e-2cb4ba9790ea/jobs/7457]
- this is CASSANDRA-15252 --> fail
*
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1243/workflows/9004656f-9d9d-4457-9e31-143fee8ed167/jobs/7461]
- this is CASSANDRA-14612 --> fail
*
[https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1244/workflows/800e27c1-941e-4cf5-b1b2-76b4b1ad6b2b/jobs/7466]
- this is CASSANDRA-17014 --> fail
* this is CASSANDRA-16959 --> haven't finished yet, will revise it tomorrow
> Broken test_rolling_upgrade -
> upgrade_tests.upgrade_through_versions_test.TestUpgrade_indev_3_0_x_To_indev_4_0_x
>
>
> Key: CASSANDRA-17140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17140
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Yifan Cai
>Priority: Normal
> Fix For: 4.0.x
>
>
> The tests "test_rolling_upgrade" fail with the below error.
>
> [https://app.circleci.com/pipelines/github/yifan-c/cassandra/279/workflows/6340cd42-0b27-42c2-8418-9f8b56c57bea/jobs/1990]
>
> I am able to alway produce it by running the test locally too.
> {{$ pytest --execute-upgrade-tests-only --upgrade-target-version-only
> --upgrade-version-selection all --cassandra-version=4.0
> upgrade_tests/upgrade_through_versions_test.py::TestUpgrade_indev_3_11_x_To_indev_4_0_x::test_rolling_upgrade}}
>
> {code:java}
> self =
> object at 0x7ffba4242fd0>
> subprocs = [,
> ]
> def _check_on_subprocs(self, subprocs):
> """
> Check on given subprocesses.
>
> If any are not alive, we'll go ahead and terminate any remaining
> alive subprocesses since this test is going to fail.
> """
> subproc_statuses = [s.is_alive() for s in subprocs]
> if not all(subproc_statuses):
> message = "A subprocess has terminated early. Subprocess
> statuses: "
> for s in subprocs:
> message += "{name} (is_alive: {aliveness}),
> ".format(name=s.name, aliveness=s.is_alive())
> message += "attempting to terminate remaining subprocesses now."
> self._terminate_subprocs()
> > raise RuntimeError(message)
> E RuntimeError: A subprocess has terminated early. Subprocess
> statuses: Process-1 (is_alive: True), Process-2 (is_alive: False), attempting
> to terminate remaining subprocesses now.{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456841#comment-17456841
]
Brandon Williams commented on CASSANDRA-17133:
--
I agree, and I'm sorry this was missed during review. I think we're probably
already handling this with similar cases somewhere but I'll have to look later.
/cc [~blerer]
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:33 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed its fine because
the other functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
It obviously works when the number is bigint.
{code:java}
select maxTimeuuid(1564830182000) from emp; system.maxtimeuuid(1564830182000)
--
42d31e0f-b5de-11e9-7f7f-7f7f7f7f7f7f
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
case INTEGER:
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
The tounixtimestamp function which wasnt changed as part of this PR also has
the same problem.
{code:java}
// code placeholder
select toUnixTimestamp(123) from emp;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Ambiguous
call to function tounixtimestamp (can be matched by following signatures:
system.tounixtimestamp : (timestamp) -> bigint, system.tounixtimestamp : (date)
-> bigint): use type casts to disambiguate"
{code}
It will definitely be great if we can check for int and cast to bigint, but not
sure where it can be done because the testAssignment function is first called.
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed its fine because
the other functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
case INTEGER:
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
The tounixtimestamp function which wasnt changed as part of this PR also has
the same problem.
{code:java}
// code placeholder
select toUnixTimestamp(123) from emp;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Ambiguous
call to function tounixtimestamp (can be matched by following signatures:
system.tounixtimestamp : (timestamp) -> bigint, system.tounixtimestamp : (date)
-> bigint): use type casts to disambiguate"
{code}
It will definitely be great if we can check for int and cast to bigint, but not
sure where it can be done because the testAssignment function is first called.
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:32 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed its fine because
the other functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
case INTEGER:
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
The tounixtimestamp function which wasnt changed as part of this PR also has
the same problem.
{code:java}
// code placeholder
select toUnixTimestamp(123) from emp;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Ambiguous
call to function tounixtimestamp (can be matched by following signatures:
system.tounixtimestamp : (timestamp) -> bigint, system.tounixtimestamp : (date)
-> bigint): use type casts to disambiguate"
{code}
It will definitely be great if we can check for int and cast to bigint, but not
sure where it can be done because the testAssignment function is first called.
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
case INTEGER:
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
The tounixtimestamp function which wasnt changed as part of this PR also has
the same problem.
{code:java}
// code placeholder
select toUnixTimestamp(123) from emp;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Ambiguous
call to function tounixtimestamp (can be matched by following signatures:
system.tounixtimestamp : (timestamp) -> bigint, system.tounixtimestamp : (date)
-> bigint): use type casts to disambiguate"
{code}
It will definitely be great if we can check for int and cast to bigint, but not
sure where it can be done because the testAssignment function is first called.
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:28 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
case INTEGER:
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
The tounixtimestamp function which wasnt changed as part of this PR also has
the same problem.
{code:java}
// code placeholder
select toUnixTimestamp(123) from emp;
InvalidRequest: Error from server: code=2200 [Invalid query] message="Ambiguous
call to function tounixtimestamp (can be matched by following signatures:
system.tounixtimestamp : (timestamp) -> bigint, system.tounixtimestamp : (date)
-> bigint): use type casts to disambiguate"
{code}
It will definitely be great if we can check for int and cast to bigint, but not
sure where it can be done because the testAssignment function is first called.
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
case INTEGER:
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:24 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
case INTEGER:
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:19 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:17 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
The reason is this logic here,
Because bigint and timestamp gets the WEAKLY_ASSIGNABLE when the parameter is
integer.
https://issues.apache.org/jira/browse/CASSANDRA-17029
{code:java}
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
{code}
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way, I will dig for the code that selects the function
based on the data type.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:16 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way, I will dig for the code that selects the function
based on the data type.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
// code placeholdercase INTEGER:
switch (nt)
{
case BIGINT:
case COUNTER:
case DATE:
case DECIMAL:
case DOUBLE:
case DURATION:
case FLOAT:
case INT:
case SMALLINT:
case TIME:
case TIMESTAMP:
case TINYINT:
case VARINT:
return AssignmentTestable.TestResult.WEAKLY_ASSIGNABLE;
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way, I will dig for the code that selects the function
based on the data type.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian edited comment on CASSANDRA-17133 at 12/10/21, 1:14 AM:
---
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
Its quite possible the user will encounter this, but I assumed that the other
functions work the same way, I will dig for the code that selects the function
based on the data type.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
was (Author: subkanthi):
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456834#comment-17456834
]
Kanthi Subramanian commented on CASSANDRA-17133:
[~brandon.williams] , I made the same change in the unit tests too, please see
here, because support for bigint was added to maxTimeuuid, the logic of picking
the right function throws the ambiguous error.
[https://github.com/apache/cassandra/pull/1275/files]
{code:java}
-- assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1234567) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
++ assertEmpty(execute("SELECT t FROM %s WHERE k = 0 AND t >
maxTimeuuid(1564830182000) AND t < minTimeuuid('2012-11-07 18:18:22-0800')"));
{code}
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Reviewers: Benjamin Lerer, Caleb Rackliffe, David Capwell (was: Benjamin
Lerer, Caleb Rackliffe, David Capwell, Ekaterina Dimitrova)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.x
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17189) Guardrail for page size
[
https://issues.apache.org/jira/browse/CASSANDRA-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456819#comment-17456819
]
Bartlomiej commented on CASSANDRA-17189:
Hi [~brandon.williams] I will have a question, I made the change following
steps from the ticket [^CASSANDRA-17189-trunk.txt] , but I don't know what to
do here:
{code:java}
public DataLimits forPaging(int pageSize)
{
Guardrails.pageSize.guard(pageSize, "?", null);
return new CQLLimits(pageSize, perPartitionLimit, isDistinct);
}
public DataLimits forPaging(int pageSize, ByteBuffer lastReturnedKey, int
lastReturnedKeyRemaining)
{
Guardrails.pageSize.guard(pageSize, "?", null);
return new CQLPagingLimits(pageSize, perPartitionLimit, isDistinct,
lastReturnedKey, lastReturnedKeyRemaining);
} {code}
atm that is how messages look
{code:java}
WARN Quering page size with size 2200 exceeds warning threshold of 2000.
WARN Quering page size with size 2400 exceeds warning threshold of 2000. {code}
problem is that there is no table name in the log(what makes it difficult to
figure out what query exceed the threshold), also guard requires 'what'
argument (it is is place for table name), but I have no idea how to make it
good :) should I pass table name to forPaging ? that doesn't looks good for me
(or no?). Should I extract guard outside forPaging ?(for example in
SelectStatement.execute?) - problem is that there are multiple places that
forPagining is executed.
* I also wonder, should we care about too small value for abort_threshold ? If
someone will set for example 100, cassandra will crash.
Thanks !
> Guardrail for page size
> ---
>
> Key: CASSANDRA-17189
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17189
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Bartlomiej
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.1
>
> Attachments: CASSANDRA-17189-trunk.txt
>
>
> Add guardrail limiting the query page size, for example:
> {code}
> # Guardrail to warn about or reject page sizes greater than threshold.
> # The two thresholds default to -1 to disable.
> page_size:
> warn_threshold: -1
> abort_threshold: -1
> {code}
> Initially this can be based on the specified number of rows used as page
> size, although it would be ideal to also limit the actual size in bytes of
> the returned pages.
> +Additional information for newcomers:+
> # Add the configuration for the new guardrail on page size in the guardrails
> section of cassandra.yaml.
> # Add a getPageSize method in GuardrailsConfig returning a Threshold.Config
> object
> # Implement that method in GuardrailsOptions, which is the default yaml-based
> implementation of GuardrailsConfig
> # Add a Threshold guardrail named pageSize in Guardrails, using the
> previously created config
> # Define JMX-friendly getters and setters for the previously created config
> in GuardrailsMBean
> # Implement the JMX-friendly getters and setters in Guardrails
> # Now that we have the guardrail ready, it’s time to use it. We should search
> for a place to invoke the Guardrails.pageSize#guard method with the page size
> that each query is going to use. The DataLimits#forPaging methods look like
> good candidates for this.
> # Finally, add some tests for the new guardrail. Given that the new guardrail
> is a Threshold, our new test should probably extend ThresholdTester.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17189) Guardrail for page size
[
https://issues.apache.org/jira/browse/CASSANDRA-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456819#comment-17456819
]
Bartlomiej edited comment on CASSANDRA-17189 at 12/9/21, 11:33 PM:
---
Hi [~brandon.williams] I will have a question, I made the change following
steps from the ticket [^CASSANDRA-17189-trunk.txt] , but I don't know what to
do here:
{code:java}
public DataLimits forPaging(int pageSize)
{
Guardrails.pageSize.guard(pageSize, "?", null);
return new CQLLimits(pageSize, perPartitionLimit, isDistinct);
}
public DataLimits forPaging(int pageSize, ByteBuffer lastReturnedKey, int
lastReturnedKeyRemaining)
{
Guardrails.pageSize.guard(pageSize, "?", null);
return new CQLPagingLimits(pageSize, perPartitionLimit, isDistinct,
lastReturnedKey, lastReturnedKeyRemaining);
} {code}
atm that is how messages look
{code:java}
WARN Quering page size with size 2200 exceeds warning threshold of 2000.
WARN Quering page size with size 2400 exceeds warning threshold of 2000. {code}
problem is that there is no table name in the log(what makes it difficult to
figure out what query exceed the threshold), also guard requires 'what'
argument (it is is place for table name), but I have no idea how to make it
good :) should I pass table name to forPaging ? that doesn't looks good for me
(or no?). Should I extract guard outside forPaging ?(for example in
SelectStatement.execute?) - problem is that there are multiple places that
forPagining is executed.
I also wonder, should we care about too small value for abort_threshold ? If
someone will set for example 100, cassandra will crash.
Thanks !
was (Author: bkowalczyyk):
Hi [~brandon.williams] I will have a question, I made the change following
steps from the ticket [^CASSANDRA-17189-trunk.txt] , but I don't know what to
do here:
{code:java}
public DataLimits forPaging(int pageSize)
{
Guardrails.pageSize.guard(pageSize, "?", null);
return new CQLLimits(pageSize, perPartitionLimit, isDistinct);
}
public DataLimits forPaging(int pageSize, ByteBuffer lastReturnedKey, int
lastReturnedKeyRemaining)
{
Guardrails.pageSize.guard(pageSize, "?", null);
return new CQLPagingLimits(pageSize, perPartitionLimit, isDistinct,
lastReturnedKey, lastReturnedKeyRemaining);
} {code}
atm that is how messages look
{code:java}
WARN Quering page size with size 2200 exceeds warning threshold of 2000.
WARN Quering page size with size 2400 exceeds warning threshold of 2000. {code}
problem is that there is no table name in the log(what makes it difficult to
figure out what query exceed the threshold), also guard requires 'what'
argument (it is is place for table name), but I have no idea how to make it
good :) should I pass table name to forPaging ? that doesn't looks good for me
(or no?). Should I extract guard outside forPaging ?(for example in
SelectStatement.execute?) - problem is that there are multiple places that
forPagining is executed.
* I also wonder, should we care about too small value for abort_threshold ? If
someone will set for example 100, cassandra will crash.
Thanks !
> Guardrail for page size
> ---
>
> Key: CASSANDRA-17189
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17189
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Bartlomiej
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.1
>
> Attachments: CASSANDRA-17189-trunk.txt
>
>
> Add guardrail limiting the query page size, for example:
> {code}
> # Guardrail to warn about or reject page sizes greater than threshold.
> # The two thresholds default to -1 to disable.
> page_size:
> warn_threshold: -1
> abort_threshold: -1
> {code}
> Initially this can be based on the specified number of rows used as page
> size, although it would be ideal to also limit the actual size in bytes of
> the returned pages.
> +Additional information for newcomers:+
> # Add the configuration for the new guardrail on page size in the guardrails
> section of cassandra.yaml.
> # Add a getPageSize method in GuardrailsConfig returning a Threshold.Config
> object
> # Implement that method in GuardrailsOptions, which is the default yaml-based
> implementation of GuardrailsConfig
> # Add a Threshold guardrail named pageSize in Guardrails, using the
> previously created config
> # Define JMX-friendly getters and setters for the previously created config
> in GuardrailsMBean
> # Implement the JMX-friendly getters and setters in Guardrails
> # Now that we have the guardrail ready, it’s time to use it. We should search
> for a place to invoke the Guardrails.pageSize#guard method with the page size
> that each query is going to use. The
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456808#comment-17456808
]
Caleb Rackliffe edited comment on CASSANDRA-15234 at 12/9/21, 11:17 PM:
Finished my first pass of review and left comments inline in the PR. (Can take
a look at CCM next...)
was (Author: maedhroz):
Finished my first pass of review and left comments inline in the PR.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.x
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17189) Guardrail for page size
[
https://issues.apache.org/jira/browse/CASSANDRA-17189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bartlomiej updated CASSANDRA-17189:
---
Attachment: CASSANDRA-17189-trunk.txt
> Guardrail for page size
> ---
>
> Key: CASSANDRA-17189
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17189
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Bartlomiej
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.1
>
> Attachments: CASSANDRA-17189-trunk.txt
>
>
> Add guardrail limiting the query page size, for example:
> {code}
> # Guardrail to warn about or reject page sizes greater than threshold.
> # The two thresholds default to -1 to disable.
> page_size:
> warn_threshold: -1
> abort_threshold: -1
> {code}
> Initially this can be based on the specified number of rows used as page
> size, although it would be ideal to also limit the actual size in bytes of
> the returned pages.
> +Additional information for newcomers:+
> # Add the configuration for the new guardrail on page size in the guardrails
> section of cassandra.yaml.
> # Add a getPageSize method in GuardrailsConfig returning a Threshold.Config
> object
> # Implement that method in GuardrailsOptions, which is the default yaml-based
> implementation of GuardrailsConfig
> # Add a Threshold guardrail named pageSize in Guardrails, using the
> previously created config
> # Define JMX-friendly getters and setters for the previously created config
> in GuardrailsMBean
> # Implement the JMX-friendly getters and setters in Guardrails
> # Now that we have the guardrail ready, it’s time to use it. We should search
> for a place to invoke the Guardrails.pageSize#guard method with the page size
> that each query is going to use. The DataLimits#forPaging methods look like
> good candidates for this.
> # Finally, add some tests for the new guardrail. Given that the new guardrail
> is a Threshold, our new test should probably extend ThresholdTester.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456808#comment-17456808
]
Caleb Rackliffe commented on CASSANDRA-15234:
-
Finished my first pass of review and left comments inline in the PR.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.x
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17136) FQL: Enabling via nodetool can trigger disk_failure_mode
[
https://issues.apache.org/jira/browse/CASSANDRA-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17136:
-
Fix Version/s: 4.0.2
(was: 4.0.x)
Since Version: 4.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/aa82e3e4277397dbf630c7b805c5aaca22d660ad
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Test run looks good now, committed. Thanks!
> FQL: Enabling via nodetool can trigger disk_failure_mode
>
>
> Key: CASSANDRA-17136
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17136
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Brendan Cicchi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.2
>
>
> When enabling fullquerylog via nodetool, if there is a non empty directory
> present under the location specified via --path which would trigger an
> java.nio.file.AccessDeniedException during cleaning, the node will trigger
> the disk_failure_policy which by default is stop. This is a fairly easy way
> to offline a cluster if someone executes this in parallel. I don't that think
> the behavior is desirable for enabling via nodetool.
>
> Repro (1 node cluster already up):
> {code:bash}
> mkdir /some/path/dir
> touch /some/path/dir/file
> chown -R user: /some/path/dir # Non Cassandra process user
> chmod 700 /some/path/dir
> nodetool enablefullquerylog --path /some/path
> {code}
> Nodetool will give back this error:
> {code:java}
> error: /some/path/dir/file
> -- StackTrace --
> java.nio.file.AccessDeniedException: /some/path/dir/file
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
> at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> at java.nio.file.Files.delete(Files.java:1126)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:250)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:237)
> at
> org.apache.cassandra.utils.binlog.BinLog.deleteRecursively(BinLog.java:492)
> at
> org.apache.cassandra.utils.binlog.BinLog.cleanDirectory(BinLog.java:477)
> at
> org.apache.cassandra.utils.binlog.BinLog$Builder.build(BinLog.java:436)
> at
> org.apache.cassandra.fql.FullQueryLogger.enable(FullQueryLogger.java:106)
> at
> org.apache.cassandra.service.StorageService.enableFullQueryLogger(StorageService.java:5915)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at
[jira] [Updated] (CASSANDRA-17136) FQL: Enabling via nodetool can trigger disk_failure_mode
[
https://issues.apache.org/jira/browse/CASSANDRA-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17136:
-
Status: Ready to Commit (was: Review In Progress)
> FQL: Enabling via nodetool can trigger disk_failure_mode
>
>
> Key: CASSANDRA-17136
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17136
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Brendan Cicchi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> When enabling fullquerylog via nodetool, if there is a non empty directory
> present under the location specified via --path which would trigger an
> java.nio.file.AccessDeniedException during cleaning, the node will trigger
> the disk_failure_policy which by default is stop. This is a fairly easy way
> to offline a cluster if someone executes this in parallel. I don't that think
> the behavior is desirable for enabling via nodetool.
>
> Repro (1 node cluster already up):
> {code:bash}
> mkdir /some/path/dir
> touch /some/path/dir/file
> chown -R user: /some/path/dir # Non Cassandra process user
> chmod 700 /some/path/dir
> nodetool enablefullquerylog --path /some/path
> {code}
> Nodetool will give back this error:
> {code:java}
> error: /some/path/dir/file
> -- StackTrace --
> java.nio.file.AccessDeniedException: /some/path/dir/file
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
> at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> at java.nio.file.Files.delete(Files.java:1126)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:250)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:237)
> at
> org.apache.cassandra.utils.binlog.BinLog.deleteRecursively(BinLog.java:492)
> at
> org.apache.cassandra.utils.binlog.BinLog.cleanDirectory(BinLog.java:477)
> at
> org.apache.cassandra.utils.binlog.BinLog$Builder.build(BinLog.java:436)
> at
> org.apache.cassandra.fql.FullQueryLogger.enable(FullQueryLogger.java:106)
> at
> org.apache.cassandra.service.StorageService.enableFullQueryLogger(StorageService.java:5915)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
[cassandra-dtest] branch trunk updated: Add test for unclean FQL dir
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 372 Add test for unclean FQL dir
372 is described below
commit 372ace6d8a5b7f4ab3d768e691f5176e7d9f
Author: Brandon Williams
AuthorDate: Mon Dec 6 13:19:03 2021 -0600
Add test for unclean FQL dir
Patch by brandonwilliams; reviewed by bereng for CASSANDRA-17136
---
fqltool_test.py | 21 +
1 file changed, 21 insertions(+)
diff --git a/fqltool_test.py b/fqltool_test.py
index f7e7c1a..cfe590c 100644
--- a/fqltool_test.py
+++ b/fqltool_test.py
@@ -160,6 +160,27 @@ class TestFQLTool(Tester):
rs = session.execute("SELECT * FROM fql_ks.fql_table;")
assert(len(list(rs)) == 1)
+def test_unclean_enable(self):
+"""
+test that fql can be enabled on a dirty directory
+@jira_ticket CASSANDRA-17136
+"""
+self.cluster.populate(1).start()
+node1 = self.cluster.nodelist()[0]
+
+with tempfile.TemporaryDirectory() as temp_dir:
+fqldir = tempfile.mkdtemp(dir=temp_dir)
+baddir = os.path.join(fqldir, 'baddir')
+os.mkdir(baddir)
+badfile = os.path.join(baddir, 'badfile')
+with open(os.path.join(badfile), 'w') as f:
+f.write('bad')
+os.chmod(badfile, 0o000)
+os.chmod(baddir, 0o000)
+node1.nodetool("enablefullquerylog --path={}".format(fqldir))
+# so teardown doesn't fail
+os.chmod(baddir, 0o777)
+
def _run_fqltool_replay(self, node, logdirs, target, queries, results,
replay_ddl_statements=False):
fqltool = self.fqltool(node)
args = [fqltool, "replay", "--target {}".format(target)]
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Don't clean when enabling FQL via JMX
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new e99a8da Don't clean when enabling FQL via JMX
e99a8da is described below
commit e99a8da161ed599c1a22a853c9c7f9caf6c1eb79
Author: Brandon Williams
AuthorDate: Thu Nov 18 10:58:53 2021 -0600
Don't clean when enabling FQL via JMX
Patch by brandonwilliams; reviewed by bereng for CASSANDRA-17136
---
CHANGES.txt | 1 +
src/java/org/apache/cassandra/fql/FullQueryLogger.java | 16
.../org/apache/cassandra/service/StorageService.java | 2 +-
3 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 7e460b6..d60f541 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -65,6 +65,7 @@
* Add a system property to set hostId if not yet initialized (CASSANDRA-14582)
* GossiperTest.testHasVersion3Nodes didn't take into account trunk version
changes, fixed to rely on latest version (CASSANDRA-16651)
Merged from 4.0:
+ * Fix disk failure triggered when enabling FQL on an unclean directory
(CASSANDRA-17136)
* Fixed broken classpath when multiple jars in build directory
(CASSANDRA-17129)
* DebuggableThreadPoolExecutor does not propagate client warnings
(CASSANDRA-17072)
* internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes
have new names. Backward compatibility with the old names added
(CASSANDRA-17141)
diff --git a/src/java/org/apache/cassandra/fql/FullQueryLogger.java
b/src/java/org/apache/cassandra/fql/FullQueryLogger.java
index ba88127..a3c9c1a 100644
--- a/src/java/org/apache/cassandra/fql/FullQueryLogger.java
+++ b/src/java/org/apache/cassandra/fql/FullQueryLogger.java
@@ -107,6 +107,22 @@ public class FullQueryLogger implements
QueryEvents.Listener
QueryEvents.instance.registerListener(this);
}
+public synchronized void enableWithoutClean(Path path, String rollCycle,
boolean blocking, int maxQueueWeight, long maxLogSize, String archiveCommand,
int maxArchiveRetries)
+{
+if (this.binLog != null)
+throw new IllegalStateException("Binlog is already configured");
+this.binLog = new BinLog.Builder().path(path)
+ .rollCycle(rollCycle)
+ .blocking(blocking)
+ .maxQueueWeight(maxQueueWeight)
+ .maxLogSize(maxLogSize)
+ .archiveCommand(archiveCommand)
+ .maxArchiveRetries(maxArchiveRetries)
+ .build(false);
+QueryEvents.instance.registerListener(this);
+}
+
+
static
{
ByteBuf buf = CBUtil.allocator.buffer(0, 0);
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 44d757b..8fde79b 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -6081,7 +6081,7 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
maxArchiveRetries = maxArchiveRetries != Integer.MIN_VALUE ?
maxArchiveRetries : fqlOptions.max_archive_retries;
Preconditions.checkNotNull(path, "cassandra.yaml did not set log_dir
and not set as parameter");
-FullQueryLogger.instance.enable(Paths.get(path), rollCycle, blocking,
maxQueueWeight, maxLogSize, archiveCommand, maxArchiveRetries);
+FullQueryLogger.instance.enableWithoutClean(Paths.get(path),
rollCycle, blocking, maxQueueWeight, maxLogSize, archiveCommand,
maxArchiveRetries);
}
@Override
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated: Don't clean when enabling FQL via JMX
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new aa82e3e Don't clean when enabling FQL via JMX
aa82e3e is described below
commit aa82e3e4277397dbf630c7b805c5aaca22d660ad
Author: Brandon Williams
AuthorDate: Thu Nov 18 10:58:53 2021 -0600
Don't clean when enabling FQL via JMX
Patch by brandonwilliams; reviewed by bereng for CASSANDRA-17136
---
CHANGES.txt | 1 +
src/java/org/apache/cassandra/fql/FullQueryLogger.java | 16
.../org/apache/cassandra/service/StorageService.java | 2 +-
3 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 055a46e..77d2737 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.2
+ * Fix disk failure triggered when enabling FQL on an unclean directory
(CASSANDRA-17136)
* Fixed broken classpath when multiple jars in build directory
(CASSANDRA-17129)
* DebuggableThreadPoolExecutor does not propagate client warnings
(CASSANDRA-17072)
* internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes
have new names. Backward compatibility with the old names added
(CASSANDRA-17141)
diff --git a/src/java/org/apache/cassandra/fql/FullQueryLogger.java
b/src/java/org/apache/cassandra/fql/FullQueryLogger.java
index 9725cea..cb35fcf 100644
--- a/src/java/org/apache/cassandra/fql/FullQueryLogger.java
+++ b/src/java/org/apache/cassandra/fql/FullQueryLogger.java
@@ -107,6 +107,22 @@ public class FullQueryLogger implements
QueryEvents.Listener
QueryEvents.instance.registerListener(this);
}
+public synchronized void enableWithoutClean(Path path, String rollCycle,
boolean blocking, int maxQueueWeight, long maxLogSize, String archiveCommand,
int maxArchiveRetries)
+{
+if (this.binLog != null)
+throw new IllegalStateException("Binlog is already configured");
+this.binLog = new BinLog.Builder().path(path)
+ .rollCycle(rollCycle)
+ .blocking(blocking)
+ .maxQueueWeight(maxQueueWeight)
+ .maxLogSize(maxLogSize)
+ .archiveCommand(archiveCommand)
+ .maxArchiveRetries(maxArchiveRetries)
+ .build(false);
+QueryEvents.instance.registerListener(this);
+}
+
+
static
{
ByteBuf buf = CBUtil.allocator.buffer(0, 0);
diff --git a/src/java/org/apache/cassandra/service/StorageService.java
b/src/java/org/apache/cassandra/service/StorageService.java
index 3c94568..d78e0e8 100644
--- a/src/java/org/apache/cassandra/service/StorageService.java
+++ b/src/java/org/apache/cassandra/service/StorageService.java
@@ -5926,7 +5926,7 @@ public class StorageService extends
NotificationBroadcasterSupport implements IE
maxArchiveRetries = maxArchiveRetries != Integer.MIN_VALUE ?
maxArchiveRetries : fqlOptions.max_archive_retries;
Preconditions.checkNotNull(path, "cassandra.yaml did not set log_dir
and not set as parameter");
-FullQueryLogger.instance.enable(Paths.get(path), rollCycle, blocking,
maxQueueWeight, maxLogSize, archiveCommand, maxArchiveRetries);
+FullQueryLogger.instance.enableWithoutClean(Paths.get(path),
rollCycle, blocking, maxQueueWeight, maxLogSize, archiveCommand,
maxArchiveRetries);
}
@Override
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10023) Emit a metric for number of local read and write calls
[ https://issues.apache.org/jira/browse/CASSANDRA-10023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456735#comment-17456735 ] Brandon Williams commented on CASSANDRA-10023: -- It's nit, but all the 'has' variables and methods would better communicated with 'is' such as hasLocalRequest -> isLocalRequest. +1 > Emit a metric for number of local read and write calls > -- > > Key: CASSANDRA-10023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10023 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Sankalp Kohli >Assignee: Stefan Miklosovic >Priority: Low > Labels: 4.0-feature-freeze-review-requested, lhf > Fix For: 4.x > > Attachments: 10023-trunk-dtests.txt, 10023-trunk.txt, > CASSANDRA-10023.patch > > > Many C* drivers have feature to be replica aware and chose the co-ordinator > which is a replica. We should add a metric which tells us whether all calls > to the co-ordinator are replica aware. > We have seen issues where client thinks they are replica aware when they > forget to add routing key at various places in the code. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16913) Fix incorrect UI bundle URL in content Dockerfile
[ https://issues.apache.org/jira/browse/CASSANDRA-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16913: --- Fix Version/s: 4.0.2 4.1 Source Control Link: https://github.com/apache/cassandra-website/commit/f6385794ff962c11365aebbfed90da6717770bf1 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [f6385794ff962c11365aebbfed90da6717770bf1|https://github.com/apache/cassandra-website/commit/f6385794ff962c11365aebbfed90da6717770bf1]. > Fix incorrect UI bundle URL in content Dockerfile > - > > Key: CASSANDRA-16913 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16913 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Anthony Grasso >Assignee: Michael Semb Wever >Priority: High > Labels: pull-request-available > Fix For: 4.0.2, 4.1 > > > We need to change the UI bundle URL in the content container Dockerfile to > point to a UI bundle that contains the correct site styling. > The URL to the bundle can be updated only after an initial version of the UI > bundle is tagged and released on the cassandra-website. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16913) Fix incorrect UI bundle URL in content Dockerfile
[ https://issues.apache.org/jira/browse/CASSANDRA-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16913: --- Reviewers: Michael Semb Wever Status: Review In Progress (was: Patch Available) > Fix incorrect UI bundle URL in content Dockerfile > - > > Key: CASSANDRA-16913 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16913 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Anthony Grasso >Assignee: Michael Semb Wever >Priority: High > Labels: pull-request-available > > We need to change the UI bundle URL in the content container Dockerfile to > point to a UI bundle that contains the correct site styling. > The URL to the bundle can be updated only after an initial version of the UI > bundle is tagged and released on the cassandra-website. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16913) Fix incorrect UI bundle URL in content Dockerfile
[ https://issues.apache.org/jira/browse/CASSANDRA-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-16913: --- Status: Ready to Commit (was: Review In Progress) > Fix incorrect UI bundle URL in content Dockerfile > - > > Key: CASSANDRA-16913 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16913 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Anthony Grasso >Assignee: Michael Semb Wever >Priority: High > Labels: pull-request-available > > We need to change the UI bundle URL in the content container Dockerfile to > point to a UI bundle that contains the correct site styling. > The URL to the bundle can be updated only after an initial version of the UI > bundle is tagged and released on the cassandra-website. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch trunk updated: CASSANDRA-16913: Updated UI Bundle URL
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new f638579 CASSANDRA-16913: Updated UI Bundle URL f638579 is described below commit f6385794ff962c11365aebbfed90da6717770bf1 Author: Anthony Grasso AuthorDate: Tue Dec 7 22:04:15 2021 +1100 CASSANDRA-16913: Updated UI Bundle URL * Generated ui-bundle.zip and added it to repository under site-ui/build/ * Updated bundle-ui URL in site-content Dockerfile to point to GitHub trunk version of ui-bundle.zip * Updated logic in ./run.sh to use by default the local copy of the ui-bundle.zip located in site-ui/build * Updated README.md to reflect changes to ./run.sh logic patch by Anthony Grasso; reviewed by Michael Semb Wever for CASSANDRA-16913 --- .gitignore | 1 - README.md | 10 +- run.sh | 35 --- site-content/Dockerfile | 5 ++--- site-ui/build/ui-bundle.zip | Bin 0 -> 4740057 bytes 5 files changed, 27 insertions(+), 24 deletions(-) diff --git a/.gitignore b/.gitignore index 9dd1f7b..6aaf298 100644 --- a/.gitignore +++ b/.gitignore @@ -6,6 +6,5 @@ /site-content/site.yaml /site-content/build/ -/site-ui/build/ /site-ui/node_modules/ /site-ui/public/ \ No newline at end of file diff --git a/README.md b/README.md index fda4bb4..8576d66 100644 --- a/README.md +++ b/README.md @@ -69,7 +69,7 @@ To build the website only, run the following command from within the `./cassandr $ ./run.sh website build ``` -This will build the website content using your local copy of the cassandra-website, and the current checked-out branch. Use this command if you want to make a change to a top-level webpage without building the docs for any versions of cassandra. +This will use the current checked-out branch of your cassandra-website local copy to build the website content. In addition, the local copy of the UI bundle will be used to style the website content. Use this command if you want to make a change to a top-level webpage without building the docs for any versions of cassandra. Once building has completed, the HTML content will be in the `./site-content/build/html/` directory ready to be reviewed and committed. @@ -225,9 +225,11 @@ $ ./run.sh \ -u cassandra-website:/local/path/to/cassandra-website/repository ``` -### Generate the website using local copy of the ui-bundle +### Generate the website using your own copy of the ui-bundle -You can use your own *ui-bundle.zip* file containing the information on how to style the website when building it. The *ui-bundle.zip* file can be generated using the `./run.sh` script. See the [Building the Site UI](#building-the-site-ui) section for furher details on how to build the *ui-bundle.zip*. +By default, the local copy of the *ui-bundle.zip* located in site-ui/build/ is used by Antora to style the website when it is built. The `./run.sh` script will pass the location of ui-bundle.zip to the Docker container that executes Antora. + +You can use your own *ui-bundle.zip* file containing the information on how to style the website when building it. The *ui-bundle.zip* file can be generated using the `./run.sh` script. See the [Building the Site UI](#building-the-site-ui) section for further details on how to build the *ui-bundle.zip*. To supply your own *ui-bundle.zip* file when building the website, run the following command. @@ -243,8 +245,6 @@ $ ./run.sh website build -z https://github.com/apache/cassandra-website/archive/ The styling contained in the *ui-bundle.zip* will be applied to docs if they are being rendered as well. -By default, the Docker container used to render the site will reference a GitHub release version of the *ui-bundle.zip*. - ## Building the Site UI To get a list of the tasks that can be executed, run the following commands. diff --git a/run.sh b/run.sh index fb8f635..2edeb25 100755 --- a/run.sh +++ b/run.sh @@ -111,7 +111,7 @@ Options specified multiple times. The valid values for REPOSITORY are only 'cassandra' and 'cassandra-website'. The values for TAGS are split by comma with no spaces. The repository and list of tags are split by a colon ':'. For example: --t cassandra:cassandra-4.0,cassandra-3.11,cassandra-3.0 +-t cassandra:cassandra-4.0.1,cassandra-3.11.11 -u REPOSITORY:URL The location defined by URL of the repository defined by REPOSITORY that will be used as the source content to generate the docs and/or website. This option can be @@ -122,17 +122,17 @@ Options
[jira] [Updated] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17133:
-
Status: Changes Suggested (was: Review In Progress)
Changing the test isn't the answer here, we should still be able to create
timeuuids from ints.
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17142) Limit the maximum hints size per host
[ https://issues.apache.org/jira/browse/CASSANDRA-17142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-17142: -- Test and Documentation Plan: ci; unit test Status: Patch Available (was: Open) > Limit the maximum hints size per host > - > > Key: CASSANDRA-17142 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17142 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Hints >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > The hints system defines a time window, i.e. max_hint_window_in_ms, to store > the hints. > It defines no limit on how much data can be kept during the time window. The > hints can grow excessively and make the node running out of disk. In such > scenario, the operators have to truncate the hints manually. > I'd propose that in addition to the conventional hints window, operators > should be able to define the maximum hints size per host, i.e. > max_hints_size_per_host_in_mb, to provide an another layer of protection. A > node stops to store hints for the down node whenever it reaches to the time > cap or the size cap. In order to not surprise the users, the config should be > disabled by default. It should also be configurable via JMX. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17142) Limit the maximum hints size per host
[
https://issues.apache.org/jira/browse/CASSANDRA-17142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456703#comment-17456703
]
Yifan Cai commented on CASSANDRA-17142:
---
PR: https://github.com/apache/cassandra/pull/1357
CI:
https://app.circleci.com/pipelines/github/yifan-c/cassandra/288/workflows/566bde90-9b69-4743-9e12-a97d4a4c137a
(mostly green)
As mentioned in the description, the PR add the configuration,
{{max_hints_size_per_host_in_mb}}, to limit the total size of the hints per
host. It is off by default.
Thanks [~Ge] for bringing up the Guardrails framework. Read though the CEP
and the first merged prototype. I think ultimately it is a good fit as you
mentioned. But as of now, the implementation is client facing since its
foundation classes depends on ClientState and ClientWarn. It does not fit very
well with those system-internal limits. I can see Guardrails is approaching it
iteratively. With respect to this ticket, we do not have to block it. We can
merge it first and address the required refactoring along with the other
system-internal limits when Guardrails evolves.
> Limit the maximum hints size per host
> -
>
> Key: CASSANDRA-17142
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17142
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
>
> The hints system defines a time window, i.e. max_hint_window_in_ms, to store
> the hints.
> It defines no limit on how much data can be kept during the time window. The
> hints can grow excessively and make the node running out of disk. In such
> scenario, the operators have to truncate the hints manually.
> I'd propose that in addition to the conventional hints window, operators
> should be able to define the maximum hints size per host, i.e.
> max_hints_size_per_host_in_mb, to provide an another layer of protection. A
> node stops to store hints for the down node whenever it reaches to the time
> cap or the size cap. In order to not surprise the users, the config should be
> disabled by default. It should also be configurable via JMX.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17136) FQL: Enabling via nodetool can trigger disk_failure_mode
[
https://issues.apache.org/jira/browse/CASSANDRA-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456591#comment-17456591
]
Brandon Williams commented on CASSANDRA-17136:
--
Heh, circle fails the teardown with this test:
bq. PermissionError: [Errno 13] Permission denied: 'baddir'
I've updated the test to change the permissions back afterward, circle is
[running|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-17136-trunk]
again.
> FQL: Enabling via nodetool can trigger disk_failure_mode
>
>
> Key: CASSANDRA-17136
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17136
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Brendan Cicchi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> When enabling fullquerylog via nodetool, if there is a non empty directory
> present under the location specified via --path which would trigger an
> java.nio.file.AccessDeniedException during cleaning, the node will trigger
> the disk_failure_policy which by default is stop. This is a fairly easy way
> to offline a cluster if someone executes this in parallel. I don't that think
> the behavior is desirable for enabling via nodetool.
>
> Repro (1 node cluster already up):
> {code:bash}
> mkdir /some/path/dir
> touch /some/path/dir/file
> chown -R user: /some/path/dir # Non Cassandra process user
> chmod 700 /some/path/dir
> nodetool enablefullquerylog --path /some/path
> {code}
> Nodetool will give back this error:
> {code:java}
> error: /some/path/dir/file
> -- StackTrace --
> java.nio.file.AccessDeniedException: /some/path/dir/file
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
> at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> at java.nio.file.Files.delete(Files.java:1126)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:250)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:237)
> at
> org.apache.cassandra.utils.binlog.BinLog.deleteRecursively(BinLog.java:492)
> at
> org.apache.cassandra.utils.binlog.BinLog.cleanDirectory(BinLog.java:477)
> at
> org.apache.cassandra.utils.binlog.BinLog$Builder.build(BinLog.java:436)
> at
> org.apache.cassandra.fql.FullQueryLogger.enable(FullQueryLogger.java:106)
> at
> org.apache.cassandra.service.StorageService.enableFullQueryLogger(StorageService.java:5915)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at
[jira] [Updated] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17133:
-
Reviewers: Brandon Williams, Brandon Williams (was: Brandon Williams)
Brandon Williams, Brandon Williams (was: Brandon Williams)
Status: Review In Progress (was: Patch Available)
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17133:
-
Reviewers: Brandon Williams
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456588#comment-17456588
]
Brandon Williams commented on CASSANDRA-17133:
--
[circle|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-17133]
running.
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17181) SchemaCQLHelperTest methods can be simplified
[
https://issues.apache.org/jira/browse/CASSANDRA-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456559#comment-17456559
]
Benjamin Lerer commented on CASSANDRA-17181:
Thanks a lot [~bkowalczyyk]. I will review your patch tomorrow :-)
> SchemaCQLHelperTest methods can be simplified
> -
>
> Key: CASSANDRA-17181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17181
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Snapshots
>Reporter: Benjamin Lerer
>Assignee: Bartlomiej
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.1
>
> Attachments: CASSANDRA-17181-4.0.patch
>
>
> {{SchemaCQLHelperTest}} is used during a snapshot to generate the
> {{schema.cql}} file. The methods accept the following paramaters:
> {{includeDroppedColumns}}, {{internals}} and {{ifNotExists}}.
> Those parameters are in practice always set to true by the calling code and
> therefore can be removed.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17181) SchemaCQLHelperTest methods can be simplified
[
https://issues.apache.org/jira/browse/CASSANDRA-17181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-17181:
---
Workflow: Copy of Cassandra Default Workflow (was: Copy of Cassandra Bug
Workflow)
Issue Type: Improvement (was: Bug)
> SchemaCQLHelperTest methods can be simplified
> -
>
> Key: CASSANDRA-17181
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17181
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Snapshots
>Reporter: Benjamin Lerer
>Assignee: Bartlomiej
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.1
>
> Attachments: CASSANDRA-17181-4.0.patch
>
>
> {{SchemaCQLHelperTest}} is used during a snapshot to generate the
> {{schema.cql}} file. The methods accept the following paramaters:
> {{includeDroppedColumns}}, {{internals}} and {{ifNotExists}}.
> Those parameters are in practice always set to true by the calling code and
> therefore can be removed.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456521#comment-17456521
]
Joey Lynch edited comment on CASSANDRA-14898 at 12/9/21, 4:04 PM:
--
Committed to 3.0, 3.11, 4.0 and trunk. Thank you [~n.v.harikrishna] for your
contribution!
trunk:
[97b47c3b|https://github.com/apache/cassandra/commit/97b47c3b5f845097181130125752bd6efc1e1e47]
4.0:
[e73d05bf|https://github.com/apache/cassandra/commit/e73d05bf858fa195ac2f2778027ef0d2ebcd3abd]
3.11:
[1fce84f9|https://github.com/apache/cassandra/commit/1fce84f9833bd62227dbf8f5d063935457dbc18e]
3.0:
[1911a887|https://github.com/apache/cassandra/commit/1911a887e8316d343c9bfe3aca3f9d143e9f4a61]
was (Author: jolynch):
Committed to 3.0, 3.11, 4.0 and trunk. Thank you [~n.v.harikrishna] for your
contribution!
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Fix For: 3.0.26, 3.11.12, 4.0.2
>
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
> [resizing|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139]
> when the number of sstables is much greater than 16 (the default size of the
> backing {{HashMap}}).
> As we see in the attached flamegraph we spend 50% of our CPU time in these
> {{getSSTable}} calls, of which 36% is spent adding sstables to the HashSet in
> {{View::select}} and 17% is spent just iterating the sstables in the first
> place. A full 16% of CPU time is spent _just resizing the HashMap_. Then
> another 4% is spend calling {{CacheService::findDesc}} which does [a linear
> search|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L475]
> for the sstable generation.
> I believe that this affects at least Cassandra 3.0.17 and trunk, and could be
> pretty easily fixed by either caching the getSSTables call or at the very
> least pre-sizing the {{HashSet}} in {{View::select}} to be the size of the
> sstables map.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joey Lynch updated CASSANDRA-14898:
---
Fix Version/s: 3.0.26
3.11.12
4.0.2
Since Version: 3.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/97b47c3b5f845097181130125752bd6efc1e1e47
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Fix For: 3.0.26, 3.11.12, 4.0.2
>
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
> [resizing|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139]
> when the number of sstables is much greater than 16 (the default size of the
> backing {{HashMap}}).
> As we see in the attached flamegraph we spend 50% of our CPU time in these
> {{getSSTable}} calls, of which 36% is spent adding sstables to the HashSet in
> {{View::select}} and 17% is spent just iterating the sstables in the first
> place. A full 16% of CPU time is spent _just resizing the HashMap_. Then
> another 4% is spend calling {{CacheService::findDesc}} which does [a linear
> search|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L475]
> for the sstable generation.
> I believe that this affects at least Cassandra 3.0.17 and trunk, and could be
> pretty easily fixed by either caching the getSSTables call or at the very
> least pre-sizing the {{HashSet}} in {{View::select}} to be the size of the
> sstables map.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456521#comment-17456521
]
Joey Lynch commented on CASSANDRA-14898:
Committed to 3.0, 3.11, 4.0 and trunk. Thank you [~n.v.harikrishna] for your
contribution!
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Fix For: 3.0.26, 3.11.12, 4.0.2
>
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
> [resizing|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139]
> when the number of sstables is much greater than 16 (the default size of the
> backing {{HashMap}}).
> As we see in the attached flamegraph we spend 50% of our CPU time in these
> {{getSSTable}} calls, of which 36% is spent adding sstables to the HashSet in
> {{View::select}} and 17% is spent just iterating the sstables in the first
> place. A full 16% of CPU time is spent _just resizing the HashMap_. Then
> another 4% is spend calling {{CacheService::findDesc}} which does [a linear
> search|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L475]
> for the sstable generation.
> I believe that this affects at least Cassandra 3.0.17 and trunk, and could be
> pretty easily fixed by either caching the getSSTables call or at the very
> least pre-sizing the {{HashSet}} in {{View::select}} to be the size of the
> sstables map.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch cassandra-3.11
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 1fce84f9833bd62227dbf8f5d063935457dbc18e
Merge: cd73c14 1911a88
Author: Joseph Lynch
AuthorDate: Thu Dec 9 10:24:19 2021 -0500
Merge branch 'cassandra-3.0' into cassandra-3.11
CHANGES.txt| 1 +
conf/cassandra.yaml| 7 +-
.../apache/cassandra/cache/AutoSavingCache.java| 8 +-
src/java/org/apache/cassandra/config/Config.java | 2 +
.../cassandra/config/DatabaseDescriptor.java | 11 ++
.../org/apache/cassandra/db/lifecycle/View.java| 2 +-
.../org/apache/cassandra/service/CacheService.java | 40 --
.../test/microbench/CacheLoaderBench.java | 137
.../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 -
9 files changed, 332 insertions(+), 14 deletions(-)
diff --cc CHANGES.txt
index 6bda91e,085e1ff..6c70235
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,16 -1,5 +1,17 @@@
-3.0.26:
+3.11.12
+ * Upgrade snakeyaml to 1.26 in 3.11 (CASSANDRA=17028)
+ * Add key validation to ssstablescrub (CASSANDRA-16969)
+ * Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851)
+ * Include SASI components to snapshots (CASSANDRA-15134)
+ * Make assassinate more resilient to missing tokens (CASSANDRA-16847)
+ * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies
(CASSANDRA-16854)
+ * Validate SASI tokenizer options before adding index to schema
(CASSANDRA-15135)
+ * Fixup scrub output when no data post-scrub and clear up old use of row,
which really means partition (CASSANDRA-16835)
+ * Fix ant-junit dependency issue (CASSANDRA-16827)
+ * Reduce thread contention in CommitLogSegment and HintsBuffer
(CASSANDRA-16072)
+ * Avoid sending CDC column if not enabled (CASSANDRA-16770)
+Merged from 3.0:
+ * Fix slow keycache load which blocks startup for tables with many sstables
(CASSANDRA-14898)
* Fix rare NPE caused by batchlog replay / node decomission races
(CASSANDRA-17049)
* Allow users to view permissions of the roles they created (CASSANDRA-16902)
* Fix failure handling in inter-node communication (CASSANDRA-16334)
diff --cc test/unit/org/apache/cassandra/db/KeyCacheTest.java
index ada6b5b,9cb06b9..f31df18
--- a/test/unit/org/apache/cassandra/db/KeyCacheTest.java
+++ b/test/unit/org/apache/cassandra/db/KeyCacheTest.java
@@@ -17,33 -17,49 +17,49 @@@
*/
package org.apache.cassandra.db;
+import java.io.IOException;
+ import java.util.ArrayList;
import java.util.Collection;
+ import java.util.Collections;
import java.util.HashMap;
import java.util.Iterator;
+ import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.ExecutionException;
+ import java.util.concurrent.TimeUnit;
import com.google.common.collect.ImmutableList;
-import com.google.common.util.concurrent.Uninterruptibles;
import org.junit.AfterClass;
import org.junit.BeforeClass;
import org.junit.Test;
import org.apache.cassandra.SchemaLoader;
import org.apache.cassandra.Util;
+ import org.apache.cassandra.cache.AutoSavingCache;
+ import org.apache.cassandra.cache.ICache;
import org.apache.cassandra.cache.KeyCacheKey;
-import org.apache.cassandra.config.CFMetaData;
import org.apache.cassandra.config.DatabaseDescriptor;
-import org.apache.cassandra.config.Schema;
import org.apache.cassandra.db.compaction.OperationType;
import org.apache.cassandra.db.compaction.CompactionManager;
import org.apache.cassandra.db.lifecycle.LifecycleTransaction;
import org.apache.cassandra.exceptions.ConfigurationException;
import org.apache.cassandra.io.sstable.format.SSTableReader;
+ import org.apache.cassandra.io.util.DataInputPlus;
import org.apache.cassandra.schema.KeyspaceParams;
import org.apache.cassandra.service.CacheService;
+ import org.apache.cassandra.utils.Pair;
import org.apache.cassandra.utils.concurrent.Refs;
++import org.hamcrest.Matchers;
+ import org.mockito.Mockito;
+ import org.mockito.internal.stubbing.answers.AnswersWithDelay;
import static org.junit.Assert.assertEquals;
-import static org.junit.Assert.assertTrue;
++import static org.junit.Assert.assertNotEquals;
++import static org.junit.Assert.assertThat;
+ import static org.mockito.ArgumentMatchers.any;
+ import static org.mockito.Mockito.doAnswer;
+ import static org.mockito.Mockito.mock;
public class KeyCacheTest
{
@@@ -51,9 -68,11 +68,14 @@@
private static final String COLUMN_FAMILY1 = "Standard1";
private static final String COLUMN_FAMILY2 = "Standard2";
private static final String COLUMN_FAMILY3 = "Standard3";
+private static final String COLUMN_FAMILY4 = "Standard4";
+private static final String COLUMN_FAMILY5 = "Standard5";
+private static final String COLUMN_FAMILY6 =
[cassandra] branch cassandra-3.0 updated: Fix slow keycache load which blocks startup for tables with many sstables.
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch cassandra-3.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-3.0 by this push:
new 1911a88 Fix slow keycache load which blocks startup for tables with
many sstables.
1911a88 is described below
commit 1911a887e8316d343c9bfe3aca3f9d143e9f4a61
Author: Venkata Harikrishna Nukala
AuthorDate: Sat Oct 23 00:03:45 2021 +0530
Fix slow keycache load which blocks startup for tables with many sstables.
Patch by Venkata Harikrishna Nukala; reviewed by Marcus Eriksson and Joseph
Lynch for CASSANDRA-14898
---
CHANGES.txt| 1 +
build.xml | 4 +-
conf/cassandra.yaml| 7 +-
.../apache/cassandra/cache/AutoSavingCache.java| 8 +-
src/java/org/apache/cassandra/config/Config.java | 2 +
.../cassandra/config/DatabaseDescriptor.java | 11 ++
.../org/apache/cassandra/db/lifecycle/View.java| 2 +-
.../org/apache/cassandra/service/CacheService.java | 38 --
.../test/microbench/CacheLoaderBench.java | 137 +
.../unit/org/apache/cassandra/db/KeyCacheTest.java | 135 +++-
10 files changed, 330 insertions(+), 15 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index cf6a956..085e1ff 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
3.0.26:
+ * Fix slow keycache load which blocks startup for tables with many sstables
(CASSANDRA-14898)
* Fix rare NPE caused by batchlog replay / node decomission races
(CASSANDRA-17049)
* Allow users to view permissions of the roles they created (CASSANDRA-16902)
* Fix failure handling in inter-node communication (CASSANDRA-16334)
diff --git a/build.xml b/build.xml
index b42c8ea..41f530d 100644
--- a/build.xml
+++ b/build.xml
@@ -375,8 +375,8 @@
-
-
+
+
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index cc2a369..ec2157b 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -286,7 +286,12 @@ counter_cache_save_period: 7200
# If not set, the default directory is $CASSANDRA_HOME/data/saved_caches.
# saved_caches_directory: /var/lib/cassandra/saved_caches
-# commitlog_sync may be either "periodic" or "batch."
+# Number of seconds the server will wait for each cache (row, key, etc ...) to
load while starting
+# the Cassandra process. Setting this to a negative value is equivalent to
disabling all cache loading on startup
+# while still having the cache during runtime.
+# cache_load_timeout_seconds: 30
+
+# commitlog_sync may be either "periodic" or "batch."
#
# When in batch mode, Cassandra won't ack writes until the commit log
# has been fsynced to disk. It will wait
diff --git a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
index 3da6352..3b7a6b8 100644
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@ -198,8 +198,9 @@ public class AutoSavingCache extends
InstrumentingCache>> futures = new
ArrayDeque>>();
-while (in.available() > 0)
+ArrayDeque>> futures = new ArrayDeque<>();
+long loadByNanos = start +
TimeUnit.SECONDS.toNanos(DatabaseDescriptor.getCacheLoadTimeout());
+while (System.nanoTime() < loadByNanos && in.available() > 0)
{
//ksname and cfname are serialized by the serializers in
CacheService
//That is delegated there because there are serializer
specific conditions
@@ -257,6 +258,7 @@ public class AutoSavingCache extends
InstrumentingCache extends
InstrumentingCache> deserialize(DataInputPlus in, ColumnFamilyStore
cfs) throws IOException;
+
+default void cleanupAfterDeserialize() { }
}
}
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index a835e5a..9e30306 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -254,6 +254,8 @@ public class Config
public volatile int counter_cache_save_period = 7200;
public volatile int counter_cache_keys_to_save = Integer.MAX_VALUE;
+public int cache_load_timeout_seconds = 30;
+
private static boolean isClientMode = false;
private static Supplier overrideLoadConfig = null;
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index b7708cd..1795c98 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -1979,6
[cassandra] branch trunk updated (507c6f7 -> 97b47c3)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 507c6f7 Merge branch 'cassandra-4.0' into trunk new 1911a88 Fix slow keycache load which blocks startup for tables with many sstables. new 1fce84f Merge branch 'cassandra-3.0' into cassandra-3.11 new e73d05b Merge branch 'cassandra-3.11' into cassandra-4.0 new 97b47c3 Merge branch 'cassandra-4.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/cassandra.yaml| 5 + .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 41 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 13 deletions(-) create mode 100644 test/microbench/org/apache/cassandra/test/microbench/CacheLoaderBench.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into cassandra-4.0
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a commit to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit e73d05bf858fa195ac2f2778027ef0d2ebcd3abd Merge: 8cef32a 1fce84f Author: Joseph Lynch AuthorDate: Thu Dec 9 10:27:29 2021 -0500 Merge branch 'cassandra-3.11' into cassandra-4.0 CHANGES.txt| 1 + conf/cassandra.yaml| 5 + .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 41 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 13 deletions(-) diff --cc CHANGES.txt index fb9dfaf,6c70235..055a46e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,31 -1,17 +1,32 @@@ -3.11.12 - * Upgrade snakeyaml to 1.26 in 3.11 (CASSANDRA=17028) +4.0.2 + * Fixed broken classpath when multiple jars in build directory (CASSANDRA-17129) + * DebuggableThreadPoolExecutor does not propagate client warnings (CASSANDRA-17072) + * internode_send_buff_size_in_bytes and internode_recv_buff_size_in_bytes have new names. Backward compatibility with the old names added (CASSANDRA-17141) + * Remove unused configuration parameters from cassandra.yaml (CASSANDRA-17132) + * Queries performed with NODE_LOCAL consistency level do not update request metrics (CASSANDRA-17052) + * Fix multiple full sources can be select unexpectedly for bootstrap streaming (CASSANDRA-16945) + * Fix cassandra.yaml formatting of parameters (CASSANDRA-17131) + * Add backward compatibility for CQLSSTableWriter Date fields (CASSANDRA-17117) + * Push initial client connection messages to trace (CASSANDRA-17038) + * Correct the internode message timestamp if sending node has wrapped (CASSANDRA-16997) + * Avoid race causing us to return null in RangesAtEndpoint (CASSANDRA-16965) + * Avoid rewriting all sstables during cleanup when transient replication is enabled (CASSANDRA-16966) + * Prevent CQLSH from failure on Python 3.10 (CASSANDRA-16987) + * Avoid trying to acquire 0 permits from the rate limiter when taking snapshot (CASSANDRA-16872) + * Upgrade Caffeine to 2.5.6 (CASSANDRA-15153) + * Include SASI components to snapshots (CASSANDRA-15134) + * Fix missed wait latencies in the output of `nodetool tpstats -F` (CASSANDRA-16938) + * Remove all the state pollution between tests in SSTableReaderTest (CASSANDRA-16888) + * Delay auth setup until after gossip has settled to avoid unavailables on startup (CASSANDRA-16783) + * Fix clustering order logic in CREATE MATERIALIZED VIEW (CASSANDRA-16898) + * org.apache.cassandra.db.rows.ArrayCell#unsharedHeapSizeExcludingData includes data twice (CASSANDRA-16900) + * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) +Merged from 3.11: * Add key validation to ssstablescrub (CASSANDRA-16969) * Update Jackson from 2.9.10 to 2.12.5 (CASSANDRA-16851) - * Include SASI components to snapshots (CASSANDRA-15134) * Make assassinate more resilient to missing tokens (CASSANDRA-16847) - * Exclude Jackson 1.x transitive dependency of hadoop* provided dependencies (CASSANDRA-16854) - * Validate SASI tokenizer options before adding index to schema (CASSANDRA-15135) - * Fixup scrub output when no data post-scrub and clear up old use of row, which really means partition (CASSANDRA-16835) - * Fix ant-junit dependency issue (CASSANDRA-16827) - * Reduce thread contention in CommitLogSegment and HintsBuffer (CASSANDRA-16072) - * Avoid sending CDC column if not enabled (CASSANDRA-16770) Merged from 3.0: + * Fix slow keycache load which blocks startup for tables with many sstables (CASSANDRA-14898) * Fix rare NPE caused by batchlog replay / node decomission races (CASSANDRA-17049) * Allow users to view permissions of the roles they created (CASSANDRA-16902) * Fix failure handling in inter-node communication (CASSANDRA-16334) diff --cc conf/cassandra.yaml index 451f9df,d00c3d9..1b0871f --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@@ -382,22 -368,24 +382,27 @@@ counter_cache_save_period: 720 # If not set, the default directory is $CASSANDRA_HOME/data/saved_caches. # saved_caches_directory: /var/lib/cassandra/saved_caches + # Number of seconds the server will wait for each cache (row, key, etc ...) to load while starting + # the Cassandra process. Setting this to a negative value is equivalent to disabling all cache loading on startup + # while still having the cache during runtime. + # cache_load_timeout_seconds: 30 + -# commitlog_sync may be either "periodic" or "batch."
[cassandra] 01/01: Merge branch 'cassandra-4.0' into trunk
This is an automated email from the ASF dual-hosted git repository.
jolynch pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 97b47c3b5f845097181130125752bd6efc1e1e47
Merge: 507c6f7 e73d05b
Author: Joseph Lynch
AuthorDate: Thu Dec 9 10:30:15 2021 -0500
Merge branch 'cassandra-4.0' into trunk
CHANGES.txt| 1 +
conf/cassandra.yaml| 5 +
.../apache/cassandra/cache/AutoSavingCache.java| 8 +-
src/java/org/apache/cassandra/config/Config.java | 2 +
.../cassandra/config/DatabaseDescriptor.java | 11 ++
.../org/apache/cassandra/db/lifecycle/View.java| 2 +-
.../org/apache/cassandra/service/CacheService.java | 41 --
.../test/microbench/CacheLoaderBench.java | 137
.../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 -
9 files changed, 332 insertions(+), 13 deletions(-)
diff --cc src/java/org/apache/cassandra/cache/AutoSavingCache.java
index 0e022d4,430161f..d0b897e
--- a/src/java/org/apache/cassandra/cache/AutoSavingCache.java
+++ b/src/java/org/apache/cassandra/cache/AutoSavingCache.java
@@@ -200,8 -209,9 +200,9 @@@ public class AutoSavingCache>> futures = new
ArrayDeque>>();
- while (in.available() > 0)
+ ArrayDeque>> futures = new ArrayDeque<>();
+ long loadByNanos = start +
TimeUnit.SECONDS.toNanos(DatabaseDescriptor.getCacheLoadTimeout());
-while (System.nanoTime() < loadByNanos && in.available() > 0)
++while (nanoTime() < loadByNanos && in.available() > 0)
{
//tableId and indexName are serialized by the serializers
in CacheService
//That is delegated there because there are serializer
specific conditions
diff --cc src/java/org/apache/cassandra/config/Config.java
index 9fb57797,c03bd96..f08da7a
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@@ -332,8 -311,8 +332,10 @@@ public class Confi
public volatile int counter_cache_save_period = 7200;
public volatile int counter_cache_keys_to_save = Integer.MAX_VALUE;
+ public int cache_load_timeout_seconds = 30;
+
+public Long paxos_cache_size_in_mb = null;
+
private static boolean isClientMode = false;
private static Supplier overrideLoadConfig = null;
diff --cc src/java/org/apache/cassandra/service/CacheService.java
index 3636c13,0a281ad..1725d5f
--- a/src/java/org/apache/cassandra/service/CacheService.java
+++ b/src/java/org/apache/cassandra/service/CacheService.java
@@@ -20,10 -20,16 +20,13 @@@ package org.apache.cassandra.service
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
+ import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
+ import java.util.Map;
import java.util.concurrent.Callable;
+ import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutionException;
-import java.util.concurrent.Future;
-
-import com.google.common.util.concurrent.Futures;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@@ -456,17 -485,12 +484,12 @@@ public class CacheService implements Ca
reader.descriptor.version,
reader.header);
RowIndexEntry entry =
indexSerializer.deserializeForCache(input);
-return Futures.immediateFuture(Pair.create(new
KeyCacheKey(cfs.metadata(), reader.descriptor, key), entry));
+return ImmediateFuture.success(Pair.create(new
KeyCacheKey(cfs.metadata(), reader.descriptor, key), entry));
}
- private SSTableReader findDesc(int generation,
Iterable collection)
+ public void cleanupAfterDeserialize()
{
- for (SSTableReader sstable : collection)
- {
- if (sstable.descriptor.generation == generation)
- return sstable;
- }
- return null;
+ cachedSSTableReaders.clear();
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (8cef32a -> e73d05b)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 8cef32a Flaky CompactionsBytemanTest new 1911a88 Fix slow keycache load which blocks startup for tables with many sstables. new 1fce84f Merge branch 'cassandra-3.0' into cassandra-3.11 new e73d05b Merge branch 'cassandra-3.11' into cassandra-4.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/cassandra.yaml| 5 + .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 41 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 13 deletions(-) create mode 100644 test/microbench/org/apache/cassandra/test/microbench/CacheLoaderBench.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (cd73c14 -> 1fce84f)
This is an automated email from the ASF dual-hosted git repository. jolynch pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from cd73c14 Merge branch 'cassandra-3.0' into cassandra-3.11 new 1911a88 Fix slow keycache load which blocks startup for tables with many sstables. new 1fce84f Merge branch 'cassandra-3.0' into cassandra-3.11 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + conf/cassandra.yaml| 7 +- .../apache/cassandra/cache/AutoSavingCache.java| 8 +- src/java/org/apache/cassandra/config/Config.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 11 ++ .../org/apache/cassandra/db/lifecycle/View.java| 2 +- .../org/apache/cassandra/service/CacheService.java | 40 -- .../test/microbench/CacheLoaderBench.java | 137 .../unit/org/apache/cassandra/db/KeyCacheTest.java | 138 - 9 files changed, 332 insertions(+), 14 deletions(-) create mode 100644 test/microbench/org/apache/cassandra/test/microbench/CacheLoaderBench.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17133) Broken test_timeuuid - upgrade_tests.cql_tests
[
https://issues.apache.org/jira/browse/CASSANDRA-17133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kanthi Subramanian updated CASSANDRA-17133:
---
Test and Documentation Plan:
Created PR here.
https://github.com/apache/cassandra-dtest/pull/171/files
Status: Patch Available (was: In Progress)
> Broken test_timeuuid - upgrade_tests.cql_tests
> --
>
> Key: CASSANDRA-17133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17133
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Semantics
>Reporter: Yifan Cai
>Assignee: Kanthi Subramanian
>Priority: Normal
> Fix For: 4.x
>
>
> Both CircleCI and Jenkins build failed at test_timeuuid with the following
> error.
> {quote}cassandra.InvalidRequest: Error from server: code=2200 [Invalid query]
> message="Ambiguous call to function maxtimeuuid (can be matched by following
> signatures: system.maxtimeuuid : (bigint) -> timeuuid, system.maxtimeuuid :
> (timestamp) -> timeuuid): use type casts to disambiguate"{quote}
> https://app.circleci.com/pipelines/github/yifan-c/cassandra/273/workflows/7a855174-823a-4553-ad09-25623747a58e/jobs/1884/tests#failed-test-0
> https://ci-cassandra.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/1272/tests/
> The change was added in CASSANDRA-17029.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16664) Log JVM Arguments at in-JVM Test Class Initialization
[
https://issues.apache.org/jira/browse/CASSANDRA-16664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16664:
---
Mentor: Paulo Motta
> Log JVM Arguments at in-JVM Test Class Initialization
> -
>
> Key: CASSANDRA-16664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16664
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Caleb Rackliffe
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.0.x, 4.x
>
>
> Normal C* startup flows through {{CassandraDaemon.setup()}}, which calls
> {{logSystemInfo()}}, logging JVM arguments and a number of other useful bits
> of information. The in-JVM dtest startup does not flow through
> {{CassandraDaemon.setup()}}, and therefore does not. It would be useful for
> troubleshooting purposes if {{TestBaseImpl}} logged the JVM arguments before
> moving into executing tests.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16664) Log JVM Arguments at in-JVM Test Class Initialization
[
https://issues.apache.org/jira/browse/CASSANDRA-16664?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-16664:
---
Labels: AdventCalendar2021 lhf (was: )
> Log JVM Arguments at in-JVM Test Class Initialization
> -
>
> Key: CASSANDRA-16664
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16664
> Project: Cassandra
> Issue Type: Improvement
> Components: Test/dtest/java
>Reporter: Caleb Rackliffe
>Priority: Normal
> Labels: AdventCalendar2021, lhf
> Fix For: 4.0.x, 4.x
>
>
> Normal C* startup flows through {{CassandraDaemon.setup()}}, which calls
> {{logSystemInfo()}}, logging JVM arguments and a number of other useful bits
> of information. The in-JVM dtest startup does not flow through
> {{CassandraDaemon.setup()}}, and therefore does not. It would be useful for
> troubleshooting purposes if {{TestBaseImpl}} logged the JVM arguments before
> moving into executing tests.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[ https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456489#comment-17456489 ] Joey Lynch edited comment on CASSANDRA-14898 at 12/9/21, 3:22 PM: -- *trunk* - [Java 11 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/67/workflows/6a98bc38-3021-4763-906b-f84099157ba0], [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/67/workflows/06b843ac-a0f9-4123-b14a-965dc8f22755] 4 dtest failures in: * (J8) test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap * (J8) test_compression_cql_options - compression_test.TestCompression * (J11) test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair * (J11, marked flakey) test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap 1 JVM dtest failure: * (J8): readWriteDuringBootstrapTest - org.apache.cassandra.distributed.test.ring.BootstrapTest *4.0* - [Java 11 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/68/workflows/2fe689eb-1938-42fd-b5cb-abab32cf0d1f] [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/68/workflows/7929a6c9-cc9c-4c5e-b271-cc276fc1d9a1] 1 JVM dtest failure: * (J8, J11) readWriteDuringBootstrapTest - org.apache.cassandra.distributed.test.ring.BootstrapTest *3.11* [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/65/workflows/6e86d386-4057-4d83-aab1-4d3731023f5d] 1 JVM dtest failure: * (J8) bootstrapTest - org.apache.cassandra.distributed.test.BootstrapTest *3.0* [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/64/workflows/5f7095bf-5031-465f-9d82-d574991e21ac] 1 JVM dtest failure: * (J8) bootstrapTest - org.apache.cassandra.distributed.test.BootstrapTest Looking at the tests that failed I don't think any of these look related to the change. A local run of readWriteDuringBootstrapTest passes on 4.0 and bootstrapTest passed on 3.0, plus I'm running a trunk [CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/69/workflows/58cdf28a-8e13-4920-94a2-870d0fe00790] run to see if readWriteDuringBootstrapTest flakes on trunk as well without the patch. I believe this is ready to commit. was (Author: jolynch): *trunk* - [Java 11 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/67/workflows/6a98bc38-3021-4763-906b-f84099157ba0], [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/67/workflows/06b843ac-a0f9-4123-b14a-965dc8f22755] 4 dtest failures in: * (J8) test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap * (J8) test_compression_cql_options - compression_test.TestCompression * (J11) test_multiple_repair - repair_tests.incremental_repair_test.TestIncRepair * (J11, marked flakey) test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap 1 JVM dtest failure: * (J8): readWriteDuringBootstrapTest - org.apache.cassandra.distributed.test.ring.BootstrapTest *4.0* - [Java 11 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/68/workflows/2fe689eb-1938-42fd-b5cb-abab32cf0d1f] [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/68/workflows/7929a6c9-cc9c-4c5e-b271-cc276fc1d9a1] 1 JVM dtest failure: * (J8, J11) readWriteDuringBootstrapTest - org.apache.cassandra.distributed.test.ring.BootstrapTest *3.11* [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/65/workflows/6e86d386-4057-4d83-aab1-4d3731023f5d] 1 JVM dtest failure: * (J8) bootstrapTest - org.apache.cassandra.distributed.test.BootstrapTest *3.0* [Java 8 CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/64/workflows/5f7095bf-5031-465f-9d82-d574991e21ac] 1 JVM dtest failure: * (J8) bootstrapTest - org.apache.cassandra.distributed.test.BootstrapTest Looking at the tests that failed I don't think any of these look related to the change. A local run of IreadWriteDuringBootstrapTest passes on 4.0 and I'm running a trunk [CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/69/workflows/58cdf28a-8e13-4920-94a2-870d0fe00790] run to see if readWriteDuringBootstrapTest flakes on trunk as well without the patch. I believe this is ready to commit. > Key cache loading is very slow when there are many SSTables > --- > > Key: CASSANDRA-14898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14898 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths > Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of > RAM, loading about 8MB of KeyCache with 10k keys in it. >Reporter: Joey Lynch >Assignee: Venkata Harikrishna Nukala >Priority: Low > Labels: Performance, low-hanging-fruit >
[jira] [Commented] (CASSANDRA-17136) FQL: Enabling via nodetool can trigger disk_failure_mode
[
https://issues.apache.org/jira/browse/CASSANDRA-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456498#comment-17456498
]
Brandon Williams commented on CASSANDRA-17136:
--
[circle for
trunk|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-17136-trunk].
> FQL: Enabling via nodetool can trigger disk_failure_mode
>
>
> Key: CASSANDRA-17136
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17136
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Brendan Cicchi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> When enabling fullquerylog via nodetool, if there is a non empty directory
> present under the location specified via --path which would trigger an
> java.nio.file.AccessDeniedException during cleaning, the node will trigger
> the disk_failure_policy which by default is stop. This is a fairly easy way
> to offline a cluster if someone executes this in parallel. I don't that think
> the behavior is desirable for enabling via nodetool.
>
> Repro (1 node cluster already up):
> {code:bash}
> mkdir /some/path/dir
> touch /some/path/dir/file
> chown -R user: /some/path/dir # Non Cassandra process user
> chmod 700 /some/path/dir
> nodetool enablefullquerylog --path /some/path
> {code}
> Nodetool will give back this error:
> {code:java}
> error: /some/path/dir/file
> -- StackTrace --
> java.nio.file.AccessDeniedException: /some/path/dir/file
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
> at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> at java.nio.file.Files.delete(Files.java:1126)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:250)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:237)
> at
> org.apache.cassandra.utils.binlog.BinLog.deleteRecursively(BinLog.java:492)
> at
> org.apache.cassandra.utils.binlog.BinLog.cleanDirectory(BinLog.java:477)
> at
> org.apache.cassandra.utils.binlog.BinLog$Builder.build(BinLog.java:436)
> at
> org.apache.cassandra.fql.FullQueryLogger.enable(FullQueryLogger.java:106)
> at
> org.apache.cassandra.service.StorageService.enableFullQueryLogger(StorageService.java:5915)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
>
[jira] [Comment Edited] (CASSANDRA-17136) FQL: Enabling via nodetool can trigger disk_failure_mode
[
https://issues.apache.org/jira/browse/CASSANDRA-17136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456498#comment-17456498
]
Brandon Williams edited comment on CASSANDRA-17136 at 12/9/21, 3:21 PM:
[circle for
trunk|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-17136-trunk].
I'll check back later to commit if that's good.
was (Author: brandon.williams):
[circle for
trunk|https://app.circleci.com/pipelines/github/driftx/cassandra?branch=CASSANDRA-17136-trunk].
> FQL: Enabling via nodetool can trigger disk_failure_mode
>
>
> Key: CASSANDRA-17136
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17136
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Brendan Cicchi
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x
>
>
> When enabling fullquerylog via nodetool, if there is a non empty directory
> present under the location specified via --path which would trigger an
> java.nio.file.AccessDeniedException during cleaning, the node will trigger
> the disk_failure_policy which by default is stop. This is a fairly easy way
> to offline a cluster if someone executes this in parallel. I don't that think
> the behavior is desirable for enabling via nodetool.
>
> Repro (1 node cluster already up):
> {code:bash}
> mkdir /some/path/dir
> touch /some/path/dir/file
> chown -R user: /some/path/dir # Non Cassandra process user
> chmod 700 /some/path/dir
> nodetool enablefullquerylog --path /some/path
> {code}
> Nodetool will give back this error:
> {code:java}
> error: /some/path/dir/file
> -- StackTrace --
> java.nio.file.AccessDeniedException: /some/path/dir/file
> at
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
> at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
> at
> sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
> at
> sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
> at java.nio.file.Files.delete(Files.java:1126)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:250)
> at
> org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:237)
> at
> org.apache.cassandra.utils.binlog.BinLog.deleteRecursively(BinLog.java:492)
> at
> org.apache.cassandra.utils.binlog.BinLog.cleanDirectory(BinLog.java:477)
> at
> org.apache.cassandra.utils.binlog.BinLog$Builder.build(BinLog.java:436)
> at
> org.apache.cassandra.fql.FullQueryLogger.enable(FullQueryLogger.java:106)
> at
> org.apache.cassandra.service.StorageService.enableFullQueryLogger(StorageService.java:5915)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:72)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:276)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
> at
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
> at
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
> at
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
> at
> javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
> at
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
> at
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
> at
>
[jira] [Updated] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joey Lynch updated CASSANDRA-14898:
---
Test and Documentation Plan:
Pre-commit tests on 3.0, 3.11, 4.0 and trunk.
Included microbench shows reduction from ~140ms / to 20ms / load in keycache
loading with 1000 sstables.
was:Pre-commit tests on 3.0, 3.11, 4.0 and trunk.
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
> [resizing|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139]
> when the number of sstables is much greater than 16 (the default size of the
> backing {{HashMap}}).
> As we see in the attached flamegraph we spend 50% of our CPU time in these
> {{getSSTable}} calls, of which 36% is spent adding sstables to the HashSet in
> {{View::select}} and 17% is spent just iterating the sstables in the first
> place. A full 16% of CPU time is spent _just resizing the HashMap_. Then
> another 4% is spend calling {{CacheService::findDesc}} which does [a linear
> search|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L475]
> for the sstable generation.
> I believe that this affects at least Cassandra 3.0.17 and trunk, and could be
> pretty easily fixed by either caching the getSSTables call or at the very
> least pre-sizing the {{HashSet}} in {{View::select}} to be the size of the
> sstables map.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joey Lynch updated CASSANDRA-14898:
---
Status: Ready to Commit (was: Review In Progress)
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
> [resizing|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139]
> when the number of sstables is much greater than 16 (the default size of the
> backing {{HashMap}}).
> As we see in the attached flamegraph we spend 50% of our CPU time in these
> {{getSSTable}} calls, of which 36% is spent adding sstables to the HashSet in
> {{View::select}} and 17% is spent just iterating the sstables in the first
> place. A full 16% of CPU time is spent _just resizing the HashMap_. Then
> another 4% is spend calling {{CacheService::findDesc}} which does [a linear
> search|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L475]
> for the sstable generation.
> I believe that this affects at least Cassandra 3.0.17 and trunk, and could be
> pretty easily fixed by either caching the getSSTables call or at the very
> least pre-sizing the {{HashSet}} in {{View::select}} to be the size of the
> sstables map.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joey Lynch updated CASSANDRA-14898:
---
Test and Documentation Plan: Pre-commit tests on 3.0, 3.11, 4.0 and trunk.
Status: Patch Available (was: Open)
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
> [resizing|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139]
> when the number of sstables is much greater than 16 (the default size of the
> backing {{HashMap}}).
> As we see in the attached flamegraph we spend 50% of our CPU time in these
> {{getSSTable}} calls, of which 36% is spent adding sstables to the HashSet in
> {{View::select}} and 17% is spent just iterating the sstables in the first
> place. A full 16% of CPU time is spent _just resizing the HashMap_. Then
> another 4% is spend calling {{CacheService::findDesc}} which does [a linear
> search|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L475]
> for the sstable generation.
> I believe that this affects at least Cassandra 3.0.17 and trunk, and could be
> pretty easily fixed by either caching the getSSTables call or at the very
> least pre-sizing the {{HashSet}} in {{View::select}} to be the size of the
> sstables map.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joey Lynch updated CASSANDRA-14898:
---
Reviewers: Joey Lynch, Marcus Eriksson, Joey Lynch (was: Joey Lynch,
Marcus Eriksson)
Joey Lynch, Marcus Eriksson, Joey Lynch (was: Joey Lynch,
Marcus Eriksson)
Status: Review In Progress (was: Patch Available)
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
> [resizing|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139]
> when the number of sstables is much greater than 16 (the default size of the
> backing {{HashMap}}).
> As we see in the attached flamegraph we spend 50% of our CPU time in these
> {{getSSTable}} calls, of which 36% is spent adding sstables to the HashSet in
> {{View::select}} and 17% is spent just iterating the sstables in the first
> place. A full 16% of CPU time is spent _just resizing the HashMap_. Then
> another 4% is spend calling {{CacheService::findDesc}} which does [a linear
> search|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L475]
> for the sstable generation.
> I believe that this affects at least Cassandra 3.0.17 and trunk, and could be
> pretty easily fixed by either caching the getSSTables call or at the very
> least pre-sizing the {{HashSet}} in {{View::select}} to be the size of the
> sstables map.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14898) Key cache loading is very slow when there are many SSTables
[
https://issues.apache.org/jira/browse/CASSANDRA-14898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456489#comment-17456489
]
Joey Lynch commented on CASSANDRA-14898:
*trunk* - [Java 11
CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/67/workflows/6a98bc38-3021-4763-906b-f84099157ba0],
[Java 8
CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/67/workflows/06b843ac-a0f9-4123-b14a-965dc8f22755]
4 dtest failures in:
* (J8) test_bootstrap_with_reset_bootstrap_state - bootstrap_test.TestBootstrap
* (J8) test_compression_cql_options - compression_test.TestCompression
* (J11) test_multiple_repair -
repair_tests.incremental_repair_test.TestIncRepair
* (J11, marked flakey) test_bootstrap_with_reset_bootstrap_state -
bootstrap_test.TestBootstrap
1 JVM dtest failure:
* (J8): readWriteDuringBootstrapTest -
org.apache.cassandra.distributed.test.ring.BootstrapTest
*4.0* - [Java 11
CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/68/workflows/2fe689eb-1938-42fd-b5cb-abab32cf0d1f]
[Java 8
CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/68/workflows/7929a6c9-cc9c-4c5e-b271-cc276fc1d9a1]
1 JVM dtest failure:
* (J8, J11) readWriteDuringBootstrapTest -
org.apache.cassandra.distributed.test.ring.BootstrapTest
*3.11* [Java 8
CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/65/workflows/6e86d386-4057-4d83-aab1-4d3731023f5d]
1 JVM dtest failure:
* (J8) bootstrapTest - org.apache.cassandra.distributed.test.BootstrapTest
*3.0* [Java 8
CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/64/workflows/5f7095bf-5031-465f-9d82-d574991e21ac]
1 JVM dtest failure:
* (J8) bootstrapTest - org.apache.cassandra.distributed.test.BootstrapTest
Looking at the tests that failed I don't think any of these look related to the
change. A local run of IreadWriteDuringBootstrapTest passes on 4.0 and I'm
running a trunk
[CI|https://app.circleci.com/pipelines/github/jolynch/cassandra/69/workflows/58cdf28a-8e13-4920-94a2-870d0fe00790]
run to see if readWriteDuringBootstrapTest flakes on trunk as well without the
patch.
I believe this is ready to commit.
> Key cache loading is very slow when there are many SSTables
> ---
>
> Key: CASSANDRA-14898
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14898
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Local Write-Read Paths
> Environment: AWS i3.2xlarge, 4 physical cores (8 threads), 60GB of
> RAM, loading about 8MB of KeyCache with 10k keys in it.
>Reporter: Joey Lynch
>Assignee: Venkata Harikrishna Nukala
>Priority: Low
> Labels: Performance, low-hanging-fruit
> Attachments: key_cache_load_slow.svg
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> While dealing with a production issue today where some 3.0.17 nodes had close
> to ~8k sstables on disk due to excessive write pressure, we had a few nodes
> crash due to OOM and then they took close to 17 minutes to load the key cache
> and recover. This excessive key cache load significantly increased the
> duration of the outage (to mitigate we just removed the saved key cache
> files). For example here is one example taking 17 minutes to load 10k keys,
> or about 10 keys per second (which is ... very slow):
> {noformat}
> INFO [pool-3-thread-1] 2018-11-15 21:50:21,885 AutoSavingCache.java:190 -
> reading saved cache /mnt/data/cassandra/saved_caches/KeyCache-d.db
> INFO [pool-3-thread-1] 2018-11-15 22:07:16,490 AutoSavingCache.java:166 -
> Completed loading (1014606 ms; 10103 keys) KeyCache cache
> {noformat}
> I've witnessed similar behavior in the past with large LCS clusters, and
> indeed it appears that any time the number of sstables is large, KeyCache
> loading takes a _really_ long time. Today I got a flame graph and I believe
> that I found the issue and I think it's reasonably easy to fix. From what I
> can tell the {{KeyCacheSerializer::deserialize}} [method
> |https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L445]
> which is called for every key is linear in the number of sstables due to the
> [call|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/service/CacheService.java#L459]
> to {{ColumnFamilyStore::getSSTables}} which ends up calling {{View::select}}
> [here|https://github.com/apache/cassandra/blob/06209037ea56b5a2a49615a99f1542d6ea1b2947/src/java/org/apache/cassandra/db/lifecycle/View.java#L139].
> The {{View::select}} call is linear in the number of sstables and causes a
> _lot_ of {{HashSet}}
>
[jira] [Assigned] (CASSANDRA-16913) Fix incorrect UI bundle URL in content Dockerfile
[ https://issues.apache.org/jira/browse/CASSANDRA-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Grasso reassigned CASSANDRA-16913: -- Assignee: Michael Semb Wever (was: Anthony Grasso) > Fix incorrect UI bundle URL in content Dockerfile > - > > Key: CASSANDRA-16913 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16913 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Anthony Grasso >Assignee: Michael Semb Wever >Priority: High > Labels: pull-request-available > > We need to change the UI bundle URL in the content container Dockerfile to > point to a UI bundle that contains the correct site styling. > The URL to the bundle can be updated only after an initial version of the UI > bundle is tagged and released on the cassandra-website. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16913) Fix incorrect UI bundle URL in content Dockerfile
[ https://issues.apache.org/jira/browse/CASSANDRA-16913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456459#comment-17456459 ] Anthony Grasso commented on CASSANDRA-16913: Minor update associated with suggested change in pull request. > Fix incorrect UI bundle URL in content Dockerfile > - > > Key: CASSANDRA-16913 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16913 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Anthony Grasso >Assignee: Anthony Grasso >Priority: High > Labels: pull-request-available > > We need to change the UI bundle URL in the content container Dockerfile to > point to a UI bundle that contains the correct site styling. > The URL to the bundle can be updated only after an initial version of the UI > bundle is tagged and released on the cassandra-website. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16763) Create Cassandra documentation content for new website
[ https://issues.apache.org/jira/browse/CASSANDRA-16763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456456#comment-17456456 ] Anthony Grasso commented on CASSANDRA-16763: Updated trunk, and cassandra-4.0 patches as per feedback in pr#1128 comment Answered questions in pr#1128 comment Updated commit message in cassandra-3.11 patch to match trunk and cassandra-4.0 message. > Create Cassandra documentation content for new website > -- > > Key: CASSANDRA-16763 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16763 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Anthony Grasso >Assignee: Michael Semb Wever >Priority: High > Time Spent: 0.5h > Remaining Estimate: 0h > > We need create the content (asciidoc) to render the Cassandra documentation > using Antora. This work can commence once the following has happened: > * Website and documentation proof of concept is done - CASSANDRA-16029 > * Website design and concept is done - CASSANDRA-16115 > * Website and document tooling is done - CASSANDRA-16066 > * Website UI components are done - CASSANDRA-16762 -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17200) " as first char of a text column (and unique occurrence) during COPY with DELIMITER set to |
[
https://issues.apache.org/jira/browse/CASSANDRA-17200?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17200:
-
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Component/s: CQL/Interpreter
Discovered By: User Report
Fix Version/s: 3.11.x
Severity: Normal
Status: Open (was: Triage Needed)
> " as first char of a text column (and unique occurrence) during COPY with
> DELIMITER set to |
>
>
> Key: CASSANDRA-17200
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17200
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Dominique Hermsdorff
>Priority: Normal
> Fix For: 3.11.x
>
>
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.11.11 | CQL spec 3.4.4 | Native protocol v4]
> During an import using | as delimiter, e.g.:
> COPY cy4secure_company_demo.demo_contacts_clear
> (record_id,state,city,first_name,last_name,email,salary,credit_rating,debt,ssn,liabilities,comments,picture)
> FROM '/var/www/cy4secure/csv/001_demo_contacts_clear_10.csv' WITH
> DELIMITER='|' AND NUMPROCESSES=2 AND MAXBATCHSIZE=1 AND CHUNKSIZE=50 AND
> SKIPROWS=2;
> Yep you can have double-quotes and quotes in a text column, no problem,
> example:
> {{...|Earth
> {color:#FF}"{color}observation{color:#FF}"{color} taken
> {color:#FF}'{color}during{color:#FF}'{color} a day pass
> by...|...}}
> BUT if you have a unique " at the beginning it fails, example:
> ...|{color:#FF}"{color}Earth observation taken during a day
> pass by...|...
> This results in my case with: Failed to import 1 rows: ParseError - Invalid
> row length 12 should be 13, given up without retries
> And the rows part of the chunk are not imported.
> My workaround: filter text values that starts with a ".
> Otherwise, you'll probably want to take a look at it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-10023) Emit a metric for number of local read and write calls
[ https://issues.apache.org/jira/browse/CASSANDRA-10023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17456307#comment-17456307 ] Stefan Miklosovic commented on CASSANDRA-10023: --- hi [~brandon.williams] , would you take a look please? As you are the last one commeting on this ticket. > Emit a metric for number of local read and write calls > -- > > Key: CASSANDRA-10023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-10023 > Project: Cassandra > Issue Type: Improvement > Components: Observability/Metrics >Reporter: Sankalp Kohli >Assignee: Stefan Miklosovic >Priority: Low > Labels: 4.0-feature-freeze-review-requested, lhf > Fix For: 4.x > > Attachments: 10023-trunk-dtests.txt, 10023-trunk.txt, > CASSANDRA-10023.patch > > > Many C* drivers have feature to be replica aware and chose the co-ordinator > which is a replica. We should add a metric which tells us whether all calls > to the co-ordinator are replica aware. > We have seen issues where client thinks they are replica aware when they > forget to add routing key at various places in the code. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17200) " as first char of a text column (and unique occurrence) during COPY with DELIMITER set to |
Dominique Hermsdorff created CASSANDRA-17200:
Summary: " as first char of a text column (and unique occurrence)
during COPY with DELIMITER set to |
Key: CASSANDRA-17200
URL: https://issues.apache.org/jira/browse/CASSANDRA-17200
Project: Cassandra
Issue Type: Bug
Reporter: Dominique Hermsdorff
Connected to Test Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.11 | CQL spec 3.4.4 | Native protocol v4]
During an import using | as delimiter, e.g.:
COPY cy4secure_company_demo.demo_contacts_clear
(record_id,state,city,first_name,last_name,email,salary,credit_rating,debt,ssn,liabilities,comments,picture)
FROM '/var/www/cy4secure/csv/001_demo_contacts_clear_10.csv' WITH
DELIMITER='|' AND NUMPROCESSES=2 AND MAXBATCHSIZE=1 AND CHUNKSIZE=50 AND
SKIPROWS=2;
Yep you can have double-quotes and quotes in a text column, no problem, example:
{{...|Earth
{color:#FF}"{color}observation{color:#FF}"{color} taken
{color:#FF}'{color}during{color:#FF}'{color} a day pass by...|...}}
BUT if you have a unique " at the beginning it fails, example:
...|{color:#FF}"{color}Earth observation taken during a day
pass by...|...
This results in my case with: Failed to import 1 rows: ParseError - Invalid row
length 12 should be 13, given up without retries
And the rows part of the chunk are not imported.
My workaround: filter text values that starts with a ".
Otherwise, you'll probably want to take a look at it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
