[jira] [Assigned] (CASSANDRA-17117) CQLSSTableWriter backwards compatibility fix for Date fields
[
https://issues.apache.org/jira/browse/CASSANDRA-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-17117:
---
Assignee: Doug Rohrer (was: Sam Tunnicliffe)
> CQLSSTableWriter backwards compatibility fix for Date fields
>
>
> Key: CASSANDRA-17117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17117
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Doug Rohrer
>Assignee: Doug Rohrer
>Priority: Normal
> Attachments: CASSANDRA-17117-4.0.patch, CASSANDRA-17117-trunk.patch
>
>
> {{cassandra-all}} library consumers of the CQLSSTableWriter class cannot
> easily create an appropriate instance of the {{LocalDate}} class in order to
> write SSTables including dates, and the current implementation no longer
> accepts a {{int}} value as a valid input as previous versions of the class
> would - we used to use
> {{((AbstractType)columnSpecification.type).decompose(value);}} in order to
> serialize the value, but now we use the type codec's .serialize method.
> Unfortunately, this doesn't work when the consumer can't easily create the
> type needed (serialize, in the case of dates, takes a {{LocalDate}} instance
> which is not easy to construct outside of Cassandra internal code).
> This can be worked around by catching the resulting {{ClassCastException}}
> thrown from the {{serialize}} call and falling back to the older
> {{decompose}} implementation, which would maintain backwards-compatibility
> with other users of the CQLSSTableWriter in cases where they are passing
> integers for date fields, which used to work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17117) CQLSSTableWriter backwards compatibility fix for Date fields
[
https://issues.apache.org/jira/browse/CASSANDRA-17117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-17117:
Authors: Doug Rohrer (was: Sam Tunnicliffe)
> CQLSSTableWriter backwards compatibility fix for Date fields
>
>
> Key: CASSANDRA-17117
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17117
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Doug Rohrer
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Attachments: CASSANDRA-17117-4.0.patch, CASSANDRA-17117-trunk.patch
>
>
> {{cassandra-all}} library consumers of the CQLSSTableWriter class cannot
> easily create an appropriate instance of the {{LocalDate}} class in order to
> write SSTables including dates, and the current implementation no longer
> accepts a {{int}} value as a valid input as previous versions of the class
> would - we used to use
> {{((AbstractType)columnSpecification.type).decompose(value);}} in order to
> serialize the value, but now we use the type codec's .serialize method.
> Unfortunately, this doesn't work when the consumer can't easily create the
> type needed (serialize, in the case of dates, takes a {{LocalDate}} instance
> which is not easy to construct outside of Cassandra internal code).
> This can be worked around by catching the resulting {{ClassCastException}}
> thrown from the {{serialize}} call and falling back to the older
> {{decompose}} implementation, which would maintain backwards-compatibility
> with other users of the CQLSSTableWriter in cases where they are passing
> integers for date fields, which used to work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17085) commit log was switched from non-daemon to daemon threads, which causes the JVM to exit in some case as no non-daemon threads are active
[
https://issues.apache.org/jira/browse/CASSANDRA-17085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-17085:
Status: Ready to Commit (was: Review In Progress)
+1 LGTM
> commit log was switched from non-daemon to daemon threads, which causes the
> JVM to exit in some case as no non-daemon threads are active
>
>
> Key: CASSANDRA-17085
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17085
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.x
>
>
> Right now bootstrap tests are failing every time we run, this work is to
> debug and fix the underling issue.
> Examples:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/1062/workflows/ba3e6395-ef22-4724-8424-0549e65d8cff/jobs/7089
> {code}
> > node3.nodetool('bootstrap resume')
> bootstrap_test.py:1014:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:1005: in nodetool
> return handle_external_tool_process(p, ['nodetool', '-h', 'localhost',
> '-p', str(self.jmx_port)] + shlex.split(cmd))
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> process =
> cmd_args = ['nodetool', '-h', 'localhost', '-p', '7300', 'bootstrap', ...]
> def handle_external_tool_process(process, cmd_args):
> out, err = process.communicate()
> if (out is not None) and isinstance(out, bytes):
> out = out.decode()
> if (err is not None) and isinstance(err, bytes):
> err = err.decode()
> rc = process.returncode
>
> if rc != 0:
> > raise ToolError(cmd_args, rc, out, err)
> E ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost',
> '-p', '7300', 'bootstrap', 'resume'] exited with non-zero status; exit
> status: 1;
> E stderr: nodetool: Failed to connect to 'localhost:7300' -
> EOFException: 'null'.
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:2305: ToolError
> {code}
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/1062/workflows/ba3e6395-ef22-4724-8424-0549e65d8cff/jobs/7087
> {code}
> > node1.start()
> bootstrap_test.py:483:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:895: in start
> node.watch_log_for_alive(self, from_mark=mark)
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:664: in
> watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:592: in watch_log_for
> head=reads[:50], tail="..."+reads[len(reads)-150:]))
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> start = 1635453190.3118386, timeout = 120
> msg = "Missing: ['127.0.0.1:7000.* is now UP'] not found in system.log:\n
> Head: \n Tail: ..."
> node = 'node3'
> @staticmethod
> def raise_if_passed(start, timeout, msg, node=None):
> if start + timeout < time.time():
> > raise TimeoutError.create(start, timeout, msg, node)
> E ccmlib.node.TimeoutError: 28 Oct 2021 20:35:10 [node3] after
> 120.12/120 seconds Missing: ['127.0.0.1:7000.* is now UP'] not found in
> system.log:
> EHead:
> ETail: ...
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17085) commit log was switched from non-daemon to daemon threads, which causes the JVM to exit in some case as no non-daemon threads are active
[
https://issues.apache.org/jira/browse/CASSANDRA-17085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-17085:
Reviewers: Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe)
Sam Tunnicliffe, Sam Tunnicliffe
Status: Review In Progress (was: Patch Available)
> commit log was switched from non-daemon to daemon threads, which causes the
> JVM to exit in some case as no non-daemon threads are active
>
>
> Key: CASSANDRA-17085
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17085
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.x
>
>
> Right now bootstrap tests are failing every time we run, this work is to
> debug and fix the underling issue.
> Examples:
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/1062/workflows/ba3e6395-ef22-4724-8424-0549e65d8cff/jobs/7089
> {code}
> > node3.nodetool('bootstrap resume')
> bootstrap_test.py:1014:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:1005: in nodetool
> return handle_external_tool_process(p, ['nodetool', '-h', 'localhost',
> '-p', str(self.jmx_port)] + shlex.split(cmd))
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> process =
> cmd_args = ['nodetool', '-h', 'localhost', '-p', '7300', 'bootstrap', ...]
> def handle_external_tool_process(process, cmd_args):
> out, err = process.communicate()
> if (out is not None) and isinstance(out, bytes):
> out = out.decode()
> if (err is not None) and isinstance(err, bytes):
> err = err.decode()
> rc = process.returncode
>
> if rc != 0:
> > raise ToolError(cmd_args, rc, out, err)
> E ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost',
> '-p', '7300', 'bootstrap', 'resume'] exited with non-zero status; exit
> status: 1;
> E stderr: nodetool: Failed to connect to 'localhost:7300' -
> EOFException: 'null'.
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:2305: ToolError
> {code}
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/1062/workflows/ba3e6395-ef22-4724-8424-0549e65d8cff/jobs/7087
> {code}
> > node1.start()
> bootstrap_test.py:483:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:895: in start
> node.watch_log_for_alive(self, from_mark=mark)
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:664: in
> watch_log_for_alive
> self.watch_log_for(tofind, from_mark=from_mark, timeout=timeout,
> filename=filename)
> ../env3.6/lib/python3.6/site-packages/ccmlib/node.py:592: in watch_log_for
> head=reads[:50], tail="..."+reads[len(reads)-150:]))
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> start = 1635453190.3118386, timeout = 120
> msg = "Missing: ['127.0.0.1:7000.* is now UP'] not found in system.log:\n
> Head: \n Tail: ..."
> node = 'node3'
> @staticmethod
> def raise_if_passed(start, timeout, msg, node=None):
> if start + timeout < time.time():
> > raise TimeoutError.create(start, timeout, msg, node)
> E ccmlib.node.TimeoutError: 28 Oct 2021 20:35:10 [node3] after
> 120.12/120 seconds Missing: ['127.0.0.1:7000.* is now UP'] not found in
> system.log:
> EHead:
> ETail: ...
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17054) v4+ protocol does not clean up client warnings, which causes leaking the state
[ https://issues.apache.org/jira/browse/CASSANDRA-17054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17433039#comment-17433039 ] Sam Tunnicliffe commented on CASSANDRA-17054: - +1, left a couple of tiny nits, nothing important > v4+ protocol does not clean up client warnings, which causes leaking the state > -- > > Key: CASSANDRA-17054 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17054 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0.x, 4.x > > > If you perform a query in v5, this will cause the STARTUP message to be > handled in the netty thread, but the way this is done is by calling an > internal API to dispatcher which requires the caller to clean up; but we do > not clean up; at this point the netty thread will have a ClientWarn state > populated. If you now perform the same query again, but with the v3 > protocol, this will pick up the state and try to serialize it, causing a > client error (in java as java rejects the output from the server) saying that > v3 may not have client warnings. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16914) Implement Virtual Tables for Auth Caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422695#comment-17422695
]
Sam Tunnicliffe commented on CASSANDRA-16914:
-
Hey [~azotcsit], sorry it took a while to respond, I agree it would be ideal to
deliver the nodetool & VT stuff in the same release.
Basically, I disagree with the intention of these VTs. One of the motivations
behind VTs (maybe the primary one) was to provide an alternative to JMX for
accessing configuration and metrics data. My perspective is that conceptually
these specific tables should exist in order to provide operators with the
ability to configure and control the auth caches without requiring JMX and, by
extension, nodetool. There is some prior art in the system_views.caches table,
which provides read-only metrics info for the counter, row and key caches. It
would be good to expose similar metrics info for the auth caches, but I see
that as a) not super critical as the datasets are so much smaller for auth
data, the hit/miss/occupancy rates are less significant b) outside the scope of
the original issue, which was to provide an operator-friendly way to invalidate
cached data. Perhaps we can file a follow up ticket to expose this data and
other cache config which can be mutated currently via JMX in a separate set of
VTs.
Putting conceptual concerns aside, I don't see there's any real value in being
able to retrieve the precise values that are currently cached from these VTs.
Firstly, there are existing means to obtain that data if required (i.e. using
the LIST X statements and/or querying the underlying tables directly). Second,
knowing that a particular auth datum it is cached _at this time_ doesn't really
benefit an operator. Consider the flow when an invalidation-triggering event
occurs; for instance, if a user has a privilege revoked, checking the VT to see
if it is cached or not is unnecessary work when an operator can simply issue
the idempotent invalidation query. In CASSANDRA-16404, we didn't implement a
set of query commands to look up precisely which values were cached (why would
we?), only the tools to invalidate them, should that be possible. So just
because it's possible to expose the full set of cached data (or even a subset
of it), it doesn't mean we should.
As well as being a somewhat pointless duplication, exposing the entire contents
of the auth tables in this way broadens the surface area of potential security
issues, as you mentioned. That said, permissions do apply to tables in
system_views, so the exposure is perhaps not as broad as you feared, however I
know there are already concerns about allowing read access to system_auth
tables.
My proposal would be to stick to modelling the invalidation commands as
provided by CASSANDRA-16404. Meaning, I would model the tables such that the
primary keys match the cache keys in the auth caches and not include anything
else:
{code}
cassandra@cqlsh> DESCRIBE TABLE system_views.permissions_cache_entries;
/*
Warning: Table system_views.permissions_cache_entries is a virtual table and
cannot be recreated with CQL.
Structure, for reference:
VIRTUAL TABLE system_views.permissions_cache_entries (
role text,
resource text,
PRIMARY KEY (role, resource)
) WITH comment = 'entries in the permissions cache';
*/
cassandra@cqlsh> SELECT * FROM system_views.permissions_cache_entries;
role | resource |
-+--+
user_a | data/test_ks/test_table1 |
user_b | data/test_ks/test_table2 |
user_b | data/test_ks/test_table3 |
user_c | data/test_ks/test_table4 |
# invalidate permissions for a single user on a single resource
cassandra@cqlsh> DELETE FROM system_views.permissions_cache_entries WHERE
role='user_a' AND resource='data/test_ks/test_table1';
# invalidate all permissions for a single user
cassandra@cqlsh> DELETE FROM system_views.permissions_cache_entries WHERE
role='user_b' ;
# invalidate the entire cache
cassandra@cqlsh> TRUNCATE system_views.permissions_cache_entries;
{code}
As I said, I think we could follow up this work with additional tables to expose
* metrics (analogous to system_views.caches)
* configuration options currently only exposed via JMX or yaml
(capacity/validity/updateinterval)
> Implement Virtual Tables for Auth Caches
>
>
> Key: CASSANDRA-16914
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16914
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization, Feature/Virtual Tables
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 4.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {{NodeTool}} commands for Auth Caches invalidation were
[jira] [Updated] (CASSANDRA-16928) InetAddressAndPort to extend InetAddress
[ https://issues.apache.org/jira/browse/CASSANDRA-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16928: Status: Review In Progress (was: Patch Available) > InetAddressAndPort to extend InetAddress > > > Key: CASSANDRA-16928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16928 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Logically InetAddressAndPort encapsulates the same information as a simple > InetAddress. Importantly, InetAddressAndPort cannot easily be unpicked from > its dependencies that prohibit it being shared between classloaders. So our > in-jvm dtest API instead uses SocketAddress. To support a clean and efficient > Streaming API we need to be able to treat InetAddressAndPort as such a > cross-classloader type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16926) Mockable FileSystem
[ https://issues.apache.org/jira/browse/CASSANDRA-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16926: Status: Review In Progress (was: Patch Available) > Mockable FileSystem > --- > > Key: CASSANDRA-16926 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16926 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative file system > implementations so that execution may be consistent between hosts. This > ticket introduces a new File API backed by Path objects, and all File usages > are ported to it. This necessitates the migration away from > RandomAccessReader, but otherwise the change is mostly straight-forward, if > quite expansive. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16927) Mockable Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16927: Status: Review In Progress (was: Patch Available) > Mockable Streaming > -- > > Key: CASSANDRA-16927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16927 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative streaming > implementations, so that execution may be controlled. The ticket encompasses > two sub-tickets: firstly we make InetAddressAndPort extend InetAddress so > that we may 2) introduce a cross-classloader API for establishing a streaming > connection and sending data over it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16925) Task Execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16925: Status: Review In Progress (was: Patch Available) > Task Execution > -- > > Key: CASSANDRA-16925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16925 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative executor > implementations that may be externally controlled. This ticket introduces an > ExecutorFactory, and all executor creation is migrated to this facility. The > codebase’s use of executors is inconsistent, and relatedly there are > divergent Future APIs in use, with Netty, Guava and Java APIs in use in > various places. To improve consistency we introduce our own Future API that > implements all of the above, as well as providing other additional ergonomic > improvements. We introduce a new ExecutorPlus API that extends Executor to > supply this extended Future API. All executors and futures are ported to > these new APIs to improve coherency in the codebase, and to support > consistently mocking these implementations. > DebuggableThreadPoolExecutor and its hierarchy is replaced with > ThreadPoolExecutorBase, ThreadPoolExecutorPlus and extending classes with > consistent names, implementations and configurability. > Additionally the ExecutorFactory takes over the allocation of new threads, > and provides a new InfiniteLoopExecutorfacility for common paradigms. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16924) Blocking Concurrency Primitives
[ https://issues.apache.org/jira/browse/CASSANDRA-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16924: Status: Ready to Commit (was: Review In Progress) +1 from me to FTR > Blocking Concurrency Primitives > --- > > Key: CASSANDRA-16924 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16924 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative implementations of > the blocking concurrency primitives we use on the project. At the same time, > the project is very inconsistent in its usage of these APIs, so this work > includes a number of improvements to the coherency of the codebase. > This ticket introduces new abstractions and standardises old ones, migrating > all blocking concurrency operations besides Futures returned by Executors to > these new APIs. This includes a migration of SimpleCondition to a new > Conditioninterface, new CountDownLatch and Semaphore interfaces, and new > static factory methods for creating these objects (as well as WaitQueue) that > can be intercepted by byte weaving. Additionally the internal Netty Future > implementation is improved to support more general use (though this is only > fully realised in a later ticket), OpOrder is improved to more easily support > mocking WaitQueue. > Finally we standardise the propagation of InterruptedExecption, with the new > UncheckedInterruptedException, so that simulations may be terminated cleanly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16924) Blocking Concurrency Primitives
[ https://issues.apache.org/jira/browse/CASSANDRA-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16924: Status: Review In Progress (was: Patch Available) > Blocking Concurrency Primitives > --- > > Key: CASSANDRA-16924 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16924 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative implementations of > the blocking concurrency primitives we use on the project. At the same time, > the project is very inconsistent in its usage of these APIs, so this work > includes a number of improvements to the coherency of the codebase. > This ticket introduces new abstractions and standardises old ones, migrating > all blocking concurrency operations besides Futures returned by Executors to > these new APIs. This includes a migration of SimpleCondition to a new > Conditioninterface, new CountDownLatch and Semaphore interfaces, and new > static factory methods for creating these objects (as well as WaitQueue) that > can be intercepted by byte weaving. Additionally the internal Netty Future > implementation is improved to support more general use (though this is only > fully realised in a later ticket), OpOrder is improved to more easily support > mocking WaitQueue. > Finally we standardise the propagation of InterruptedExecption, with the new > UncheckedInterruptedException, so that simulations may be terminated cleanly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16995) Add tests for Resource fromName/getName
[
https://issues.apache.org/jira/browse/CASSANDRA-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422583#comment-17422583
]
Sam Tunnicliffe commented on CASSANDRA-16995:
-
Adding an extra fixver costs nothing, so I've done it for CASSANDRA-16977
> Add tests for Resource fromName/getName
> ---
>
> Key: CASSANDRA-16995
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16995
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> As a part of CASSANDRA-16977 we identified a problem functions
> ({{FunctionResource.fromName}}) parsing logic. We agreed that
> {{Resource.fromName/}}{{Resource.getName}} should be symmetrical. I'd like to
> ensure that by having a corresponding unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16977) ArrayIndexOutOfBoundsException in FunctionResource#fromName
[
https://issues.apache.org/jira/browse/CASSANDRA-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16977:
Fix Version/s: 3.0.26
> ArrayIndexOutOfBoundsException in FunctionResource#fromName
> ---
>
> Key: CASSANDRA-16977
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16977
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Authorization
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1
>
>
> {{FunctionResource}} can't handle functions with 0 args where it throws:
> {noformat}
> 2021-04-08 10:45:40,984 ErrorMessage.java:387 - Unexpected exception during
> request
> java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.cassandra.auth.FunctionResource.fromName(FunctionResource.java:178)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16995) Add tests for Resource fromName/getName
[
https://issues.apache.org/jira/browse/CASSANDRA-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421298#comment-17421298
]
Sam Tunnicliffe commented on CASSANDRA-16995:
-
__ __It would be good to add a link/comment on CASSANDRA-16977 about the 3.0
backport.
> Add tests for Resource fromName/getName
> ---
>
> Key: CASSANDRA-16995
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16995
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> As a part of CASSANDRA-16977 we identified a problem functions
> ({{FunctionResource.fromName}}) parsing logic. We agreed that
> {{Resource.fromName/}}{{Resource.getName}} should be symmetrical. I'd like to
> ensure that by having a corresponding unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16995) Add tests for Resource fromName/getName
[
https://issues.apache.org/jira/browse/CASSANDRA-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421296#comment-17421296
]
Sam Tunnicliffe commented on CASSANDRA-16995:
-
+1, the failing upgrade test (
{{dtest-upgrade.upgrade_internal_auth_test.TestAuthUpgrade.test_upgrade_to_30}}
) also failed on the most recent 3.0 nightly, so doesn't look related.
> Add tests for Resource fromName/getName
> ---
>
> Key: CASSANDRA-16995
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16995
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> As a part of CASSANDRA-16977 we identified a problem functions
> ({{FunctionResource.fromName}}) parsing logic. We agreed that
> {{Resource.fromName/}}{{Resource.getName}} should be symmetrical. I'd like to
> ensure that by having a corresponding unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16995) Add tests for Resource fromName/getName
[
https://issues.apache.org/jira/browse/CASSANDRA-16995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420537#comment-17420537
]
Sam Tunnicliffe commented on CASSANDRA-16995:
-
The patch LGTM, but I agree that it could also be applied to earlier versions.
Being a test only change, it's really no big deal but in theory it could be
applied back as far as 3.0.
> Add tests for Resource fromName/getName
> ---
>
> Key: CASSANDRA-16995
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16995
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 4.x
>
>
> As a part of CASSANDRA-16977 we identified a problem functions
> ({{FunctionResource.fromName}}) parsing logic. We agreed that
> {{Resource.fromName/}}{{Resource.getName}} should be symmetrical. I'd like to
> ensure that by having a corresponding unit test.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16927) Mockable Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16927: Test and Documentation Plan: new and existing utests & dtests Status: Patch Available (was: Open) Branch: https://github.com/beobal/cassandra/tree/16927-trunk PR: https://github.com/beobal/cassandra/pull/7 Note: the branch dependency order is out of whack at the end of these phase 1 issues: 16923 <- 16924 <- 16925 <- 16926 <- *16928* <- 16927 > Mockable Streaming > -- > > Key: CASSANDRA-16927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16927 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative streaming > implementations, so that execution may be controlled. The ticket encompasses > two sub-tickets: firstly we make InetAddressAndPort extend InetAddress so > that we may 2) introduce a cross-classloader API for establishing a streaming > connection and sending data over it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16928) InetAddressAndPort to extend InetAddress
[ https://issues.apache.org/jira/browse/CASSANDRA-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe reassigned CASSANDRA-16928: --- Assignee: Benedict Elliott Smith > InetAddressAndPort to extend InetAddress > > > Key: CASSANDRA-16928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16928 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > Logically InetAddressAndPort encapsulates the same information as a simple > InetAddress. Importantly, InetAddressAndPort cannot easily be unpicked from > its dependencies that prohibit it being shared between classloaders. So our > in-jvm dtest API instead uses SocketAddress. To support a clean and efficient > Streaming API we need to be able to treat InetAddressAndPort as such a > cross-classloader type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16927) Mockable Streaming
[ https://issues.apache.org/jira/browse/CASSANDRA-16927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16927: Change Category: Quality Assurance Complexity: Normal Component/s: Legacy/Streaming and Messaging Reviewers: Sam Tunnicliffe Assignee: Benedict Elliott Smith Status: Open (was: Triage Needed) > Mockable Streaming > -- > > Key: CASSANDRA-16927 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16927 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Streaming and Messaging >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative streaming > implementations, so that execution may be controlled. The ticket encompasses > two sub-tickets: firstly we make InetAddressAndPort extend InetAddress so > that we may 2) introduce a cross-classloader API for establishing a streaming > connection and sending data over it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16928) InetAddressAndPort to extend InetAddress
[ https://issues.apache.org/jira/browse/CASSANDRA-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16928: Authors: Benedict Elliott Smith Test and Documentation Plan: existing and new utests & dtests Status: Patch Available (was: Open) Branch: https://github.com/beobal/cassandra/tree/16928-trunk PR: https://github.com/beobal/cassandra/pull/8 Note: the branch dependency order is out of whack at the end of these phase 1 issues: 16923 <- 16924 <- 16925 <- 16926 <- *16928* <- 16927 > InetAddressAndPort to extend InetAddress > > > Key: CASSANDRA-16928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16928 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Priority: Normal > > Logically InetAddressAndPort encapsulates the same information as a simple > InetAddress. Importantly, InetAddressAndPort cannot easily be unpicked from > its dependencies that prohibit it being shared between classloaders. So our > in-jvm dtest API instead uses SocketAddress. To support a clean and efficient > Streaming API we need to be able to treat InetAddressAndPort as such a > cross-classloader type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16928) InetAddressAndPort to extend InetAddress
[ https://issues.apache.org/jira/browse/CASSANDRA-16928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16928: Change Category: Quality Assurance Complexity: Normal Component/s: Legacy/Core Reviewers: Sam Tunnicliffe Status: Open (was: Triage Needed) > InetAddressAndPort to extend InetAddress > > > Key: CASSANDRA-16928 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16928 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Priority: Normal > > Logically InetAddressAndPort encapsulates the same information as a simple > InetAddress. Importantly, InetAddressAndPort cannot easily be unpicked from > its dependencies that prohibit it being shared between classloaders. So our > in-jvm dtest API instead uses SocketAddress. To support a clean and efficient > Streaming API we need to be able to treat InetAddressAndPort as such a > cross-classloader type. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9384) Update jBCrypt dependency to version 0.4
[ https://issues.apache.org/jira/browse/CASSANDRA-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419777#comment-17419777 ] Sam Tunnicliffe commented on CASSANDRA-9384: Sure, sounds fair enough to me. > Update jBCrypt dependency to version 0.4 > > > Key: CASSANDRA-9384 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9384 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Dinesh Joshi >Priority: Normal > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x > > Time Spent: 10m > Remaining Estimate: 0h > > https://bugzilla.mindrot.org/show_bug.cgi?id=2097 > Although the bug tracker lists it as NEW/OPEN, the release notes for 0.4 > indicate that this is now fixed, so we should update. > Thanks to [~Bereng] for identifying the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-9384) Update jBCrypt dependency to version 0.4
[ https://issues.apache.org/jira/browse/CASSANDRA-9384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419761#comment-17419761 ] Sam Tunnicliffe commented on CASSANDRA-9384: bq. So I am wondering who is this feature actually good for The original implementation hardcoded the number of rounds to 2^10, but that was made configurable in CASSANDRA-8085 and the defaults discussed in CASSANDRA-14678. Maybe we could have constrained the value a bit more, instead of just allowing anything jBcrypt supported, but that ship has sailed now. > Update jBCrypt dependency to version 0.4 > > > Key: CASSANDRA-9384 > URL: https://issues.apache.org/jira/browse/CASSANDRA-9384 > Project: Cassandra > Issue Type: Bug >Reporter: Sam Tunnicliffe >Assignee: Dinesh Joshi >Priority: Normal > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x > > Time Spent: 10m > Remaining Estimate: 0h > > https://bugzilla.mindrot.org/show_bug.cgi?id=2097 > Although the bug tracker lists it as NEW/OPEN, the release notes for 0.4 > indicate that this is now fixed, so we should update. > Thanks to [~Bereng] for identifying the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16990) Update jbcrypt library to 0.4 from 0.3m to resolve CVE-2015-0886
[ https://issues.apache.org/jira/browse/CASSANDRA-16990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419724#comment-17419724 ] Sam Tunnicliffe commented on CASSANDRA-16990: - [~stefan.miklosovic] this seems to be a dupe of CASSANDRA-9384 > Update jbcrypt library to 0.4 from 0.3m to resolve CVE-2015-0886 > > > Key: CASSANDRA-16990 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16990 > Project: Cassandra > Issue Type: Task > Components: Dependencies >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1 > > > We are using jbcrypto of version 0.3m across all versions, this version of > the library was never changed since 1.1.2. > In 0.3m they found out this (1) and (2, 3 for better explanation / reference) > I think we are affected by this, it is possible to set 31 rounds here (4) > which would hit the same same logic afteward these tickets are talking about. > 1) [https://nvd.nist.gov/vuln/detail/CVE-2015-0886] > 2) [http://www.mindrot.org/projects/jBCrypt/news/rel04.html] > 3) [https://bugzilla.mindrot.org/show_bug.cgi?id=2097] > 4) > [https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/auth/CassandraRoleManager.java#L105-L117] > I hence propose to update the library to 0.4 where this is fixed. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16977) ArrayIndexOutOfBoundsException in FunctionResource#fromName
[
https://issues.apache.org/jira/browse/CASSANDRA-16977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17418004#comment-17418004
]
Sam Tunnicliffe commented on CASSANDRA-16977:
-
{{DropFunctionStatement}} does accept a function without parens, but this is
really because it's only constructing a {{FunctionName}}, which does not have
parameters. Wherever a {{FunctionResource}} is required, the brackets are used.
IMO, {{fromName}} and {{getName}} ought to be symmetrical, otherwise we're at
risk of running into problems where something is serialized in one format but
read in another. i.e. entries in the permissions tables use {{getName}} and so
always include brackets.
> ArrayIndexOutOfBoundsException in FunctionResource#fromName
> ---
>
> Key: CASSANDRA-16977
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16977
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Authorization
>Reporter: Berenguer Blasi
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 3.11.x, 4.0.x, 4.x
>
>
> {{FunctionResource}} can't handle functions with 0 args where it throws:
> {noformat}
> 2021-04-08 10:45:40,984 ErrorMessage.java:387 - Unexpected exception during
> request
> java.lang.ArrayIndexOutOfBoundsException: 1
> at
> org.apache.cassandra.auth.FunctionResource.fromName(FunctionResource.java:178)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16923) System Clock
[ https://issues.apache.org/jira/browse/CASSANDRA-16923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416855#comment-17416855 ] Sam Tunnicliffe edited comment on CASSANDRA-16923 at 9/20/21, 12:28 PM: Branch: https://github.com/beobal/cassandra/tree/16923-trunk PR: https://github.com/beobal/cassandra/pull/3 was (Author: beobal): [16923-trunk|https://github.com/beobal/cassandra/tree/16923-trunk] > System Clock > > > Key: CASSANDRA-16923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16923 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 and other project testing goals, it is necessary to support > supplying a non-standard time source that can be externally controlled. This > ticket modifies existing abstractions and migrates all calls to > System.currentTimeMillis() and System.nanoTime() to these APIs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16926) Mockable FileSystem
[ https://issues.apache.org/jira/browse/CASSANDRA-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416866#comment-17416866 ] Sam Tunnicliffe edited comment on CASSANDRA-16926 at 9/20/21, 12:28 PM: Branch: https://github.com/beobal/cassandra/tree/16926-trunk PR: https://github.com/beobal/cassandra/pull/6 was (Author: beobal): [16926-trunk|https://github.com/beobal/cassandra/tree/16926-trunk] > Mockable FileSystem > --- > > Key: CASSANDRA-16926 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16926 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative file system > implementations so that execution may be consistent between hosts. This > ticket introduces a new File API backed by Path objects, and all File usages > are ported to it. This necessitates the migration away from > RandomAccessReader, but otherwise the change is mostly straight-forward, if > quite expansive. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16925) Task Execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416862#comment-17416862 ] Sam Tunnicliffe edited comment on CASSANDRA-16925 at 9/20/21, 12:28 PM: Branch: https://github.com/beobal/cassandra/tree/16925-trunk PR: https://github.com/beobal/cassandra/pull/5 was (Author: beobal): [16925-trunk|https://github.com/beobal/cassandra/tree/16925-trunk] > Task Execution > -- > > Key: CASSANDRA-16925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16925 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative executor > implementations that may be externally controlled. This ticket introduces an > ExecutorFactory, and all executor creation is migrated to this facility. The > codebase’s use of executors is inconsistent, and relatedly there are > divergent Future APIs in use, with Netty, Guava and Java APIs in use in > various places. To improve consistency we introduce our own Future API that > implements all of the above, as well as providing other additional ergonomic > improvements. We introduce a new ExecutorPlus API that extends Executor to > supply this extended Future API. All executors and futures are ported to > these new APIs to improve coherency in the codebase, and to support > consistently mocking these implementations. > DebuggableThreadPoolExecutor and its hierarchy is replaced with > ThreadPoolExecutorBase, ThreadPoolExecutorPlus and extending classes with > consistent names, implementations and configurability. > Additionally the ExecutorFactory takes over the allocation of new threads, > and provides a new InfiniteLoopExecutorfacility for common paradigms. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16924) Blocking Concurrency Primitives
[ https://issues.apache.org/jira/browse/CASSANDRA-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416858#comment-17416858 ] Sam Tunnicliffe edited comment on CASSANDRA-16924 at 9/20/21, 12:28 PM: Branch: https://github.com/beobal/cassandra/tree/16924-trunk PR: https://github.com/beobal/cassandra/pull/4 was (Author: beobal): [16924-trunk|https://github.com/beobal/cassandra/tree/16924-trunk] > Blocking Concurrency Primitives > --- > > Key: CASSANDRA-16924 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16924 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative implementations of > the blocking concurrency primitives we use on the project. At the same time, > the project is very inconsistent in its usage of these APIs, so this work > includes a number of improvements to the coherency of the codebase. > This ticket introduces new abstractions and standardises old ones, migrating > all blocking concurrency operations besides Futures returned by Executors to > these new APIs. This includes a migration of SimpleCondition to a new > Conditioninterface, new CountDownLatch and Semaphore interfaces, and new > static factory methods for creating these objects (as well as WaitQueue) that > can be intercepted by byte weaving. Additionally the internal Netty Future > implementation is improved to support more general use (though this is only > fully realised in a later ticket), OpOrder is improved to more easily support > mocking WaitQueue. > Finally we standardise the propagation of InterruptedExecption, with the new > UncheckedInterruptedException, so that simulations may be terminated cleanly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16926) Mockable FileSystem
[ https://issues.apache.org/jira/browse/CASSANDRA-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16926: Test and Documentation Plan: new and existing utests & dtests Status: Patch Available (was: Open) [16926-trunk|https://github.com/beobal/cassandra/tree/16926-trunk] > Mockable FileSystem > --- > > Key: CASSANDRA-16926 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16926 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative file system > implementations so that execution may be consistent between hosts. This > ticket introduces a new File API backed by Path objects, and all File usages > are ported to it. This necessitates the migration away from > RandomAccessReader, but otherwise the change is mostly straight-forward, if > quite expansive. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16926) Mockable FileSystem
[ https://issues.apache.org/jira/browse/CASSANDRA-16926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16926: Change Category: Quality Assurance Complexity: Normal Component/s: Legacy/Core Reviewers: Aleksey Yeschenko, Sam Tunnicliffe Assignee: Benedict Elliott Smith Status: Open (was: Triage Needed) > Mockable FileSystem > --- > > Key: CASSANDRA-16926 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16926 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative file system > implementations so that execution may be consistent between hosts. This > ticket introduces a new File API backed by Path objects, and all File usages > are ported to it. This necessitates the migration away from > RandomAccessReader, but otherwise the change is mostly straight-forward, if > quite expansive. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16925) Task Execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16925: Test and Documentation Plan: new and existing utests & dtests Status: Patch Available (was: Open) [16925-trunk|https://github.com/beobal/cassandra/tree/16925-trunk] > Task Execution > -- > > Key: CASSANDRA-16925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16925 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative executor > implementations that may be externally controlled. This ticket introduces an > ExecutorFactory, and all executor creation is migrated to this facility. The > codebase’s use of executors is inconsistent, and relatedly there are > divergent Future APIs in use, with Netty, Guava and Java APIs in use in > various places. To improve consistency we introduce our own Future API that > implements all of the above, as well as providing other additional ergonomic > improvements. We introduce a new ExecutorPlus API that extends Executor to > supply this extended Future API. All executors and futures are ported to > these new APIs to improve coherency in the codebase, and to support > consistently mocking these implementations. > DebuggableThreadPoolExecutor and its hierarchy is replaced with > ThreadPoolExecutorBase, ThreadPoolExecutorPlus and extending classes with > consistent names, implementations and configurability. > Additionally the ExecutorFactory takes over the allocation of new threads, > and provides a new InfiniteLoopExecutorfacility for common paradigms. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16925) Task Execution
[ https://issues.apache.org/jira/browse/CASSANDRA-16925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16925: Change Category: Quality Assurance Complexity: Normal Component/s: Legacy/Core Reviewers: Sam Tunnicliffe Assignee: Benedict Elliott Smith Status: Open (was: Triage Needed) > Task Execution > -- > > Key: CASSANDRA-16925 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16925 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative executor > implementations that may be externally controlled. This ticket introduces an > ExecutorFactory, and all executor creation is migrated to this facility. The > codebase’s use of executors is inconsistent, and relatedly there are > divergent Future APIs in use, with Netty, Guava and Java APIs in use in > various places. To improve consistency we introduce our own Future API that > implements all of the above, as well as providing other additional ergonomic > improvements. We introduce a new ExecutorPlus API that extends Executor to > supply this extended Future API. All executors and futures are ported to > these new APIs to improve coherency in the codebase, and to support > consistently mocking these implementations. > DebuggableThreadPoolExecutor and its hierarchy is replaced with > ThreadPoolExecutorBase, ThreadPoolExecutorPlus and extending classes with > consistent names, implementations and configurability. > Additionally the ExecutorFactory takes over the allocation of new threads, > and provides a new InfiniteLoopExecutorfacility for common paradigms. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16924) Blocking Concurrency Primitives
[ https://issues.apache.org/jira/browse/CASSANDRA-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16924: Test and Documentation Plan: new and existing utests/dtests Status: Patch Available (was: Open) [16924-trunk|https://github.com/beobal/cassandra/tree/16924-trunk] > Blocking Concurrency Primitives > --- > > Key: CASSANDRA-16924 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16924 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative implementations of > the blocking concurrency primitives we use on the project. At the same time, > the project is very inconsistent in its usage of these APIs, so this work > includes a number of improvements to the coherency of the codebase. > This ticket introduces new abstractions and standardises old ones, migrating > all blocking concurrency operations besides Futures returned by Executors to > these new APIs. This includes a migration of SimpleCondition to a new > Conditioninterface, new CountDownLatch and Semaphore interfaces, and new > static factory methods for creating these objects (as well as WaitQueue) that > can be intercepted by byte weaving. Additionally the internal Netty Future > implementation is improved to support more general use (though this is only > fully realised in a later ticket), OpOrder is improved to more easily support > mocking WaitQueue. > Finally we standardise the propagation of InterruptedExecption, with the new > UncheckedInterruptedException, so that simulations may be terminated cleanly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16924) Blocking Concurrency Primitives
[ https://issues.apache.org/jira/browse/CASSANDRA-16924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16924: Change Category: Quality Assurance Complexity: Normal Component/s: Legacy/Core Reviewers: Sam Tunnicliffe Assignee: Benedict Elliott Smith Status: Open (was: Triage Needed) > Blocking Concurrency Primitives > --- > > Key: CASSANDRA-16924 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16924 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 it is necessary to support alternative implementations of > the blocking concurrency primitives we use on the project. At the same time, > the project is very inconsistent in its usage of these APIs, so this work > includes a number of improvements to the coherency of the codebase. > This ticket introduces new abstractions and standardises old ones, migrating > all blocking concurrency operations besides Futures returned by Executors to > these new APIs. This includes a migration of SimpleCondition to a new > Conditioninterface, new CountDownLatch and Semaphore interfaces, and new > static factory methods for creating these objects (as well as WaitQueue) that > can be intercepted by byte weaving. Additionally the internal Netty Future > implementation is improved to support more general use (though this is only > fully realised in a later ticket), OpOrder is improved to more easily support > mocking WaitQueue. > Finally we standardise the propagation of InterruptedExecption, with the new > UncheckedInterruptedException, so that simulations may be terminated cleanly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16923) System Clock
[ https://issues.apache.org/jira/browse/CASSANDRA-16923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16923: Test and Documentation Plan: New and existing dtests, no new functionality Status: Patch Available (was: Open) [16923-trunk|https://github.com/beobal/cassandra/tree/16923-trunk] > System Clock > > > Key: CASSANDRA-16923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16923 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 and other project testing goals, it is necessary to support > supplying a non-standard time source that can be externally controlled. This > ticket modifies existing abstractions and migrates all calls to > System.currentTimeMillis() and System.nanoTime() to these APIs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16923) System Clock
[ https://issues.apache.org/jira/browse/CASSANDRA-16923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16923: Change Category: Quality Assurance Complexity: Normal Component/s: Legacy/Core Reviewers: Aleksey Yeschenko, Sam Tunnicliffe Assignee: Benedict Elliott Smith Status: Open (was: Triage Needed) > System Clock > > > Key: CASSANDRA-16923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16923 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Core >Reporter: Benedict Elliott Smith >Assignee: Benedict Elliott Smith >Priority: Normal > > To support CEP-10 and other project testing goals, it is necessary to support > supplying a non-standard time source that can be externally controlled. This > ticket modifies existing abstractions and migrates all calls to > System.currentTimeMillis() and System.nanoTime() to these APIs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16914) Implement Virtual Tables for Auth Caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17414827#comment-17414827
]
Sam Tunnicliffe commented on CASSANDRA-16914:
-
The VTs here are more fine grained than they need to be, i.e. than the caches
themselves and their nodetool equivalents.
For instance, we don't cache or enable the invalidation of individual
permissions, so modelling the perms cache table with individual permissions as
clustering columns is unnecessary. The underlying cache impl and {{nodetool
invalidatepermissionscache}} work on the level of role + resource, so I would
suggest modelling the VT likewise with Role as partition key and resource as
the clustering column.
Whether the actual permissions themselves even need to be included in the VT is
a valid question IMO. It could be argued that the purpose of this set of VTs is
not to expose the full set of cached data, but to provide a way to invalidate
cached items, i.e. these VTs represent the _commands_ rather than the
_contents_. If we agree with that, then only the cache keys need to be exposed
via the VTs. The counter argument would be that a difference in the models of
the base auth tables and the cache VTs could be confusing (though there are
already differences in your proposal).
> Implement Virtual Tables for Auth Caches
>
>
> Key: CASSANDRA-16914
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16914
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization, Feature/Virtual Tables
>Reporter: Aleksei Zotov
>Assignee: Aleksei Zotov
>Priority: Low
> Fix For: 4.x
>
>
> {{NodeTool}} commands for Auth Caches invalidation were implemented as a part
> of CASSANDRA-16404 ticket. While discussing that ticket it was agreed that
> there is a need to develop the same kind of functionality through Vitrual
> Tables. Unfortunately, VT did not have {{TRUNCATE}} and {{DELETE}} support.
> And CASSANDRA-16806 was created for that reason. Once it is completed,
> further work can be started.
> The goal of this ticket is to create VTs for the following caches:
> * {{CredentialsCache}}
> * {{JmxPermissionsCache}}
> * {{NetworkPermissionsCache}}
> * {{PermissionsCache}}
> * {{RolesCache}}
> The VTs should support reading from and modification of the in the Auth
> Caches.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16940) Confusing ProtocolException msg Invalid or unsupported protocol version (4)
[
https://issues.apache.org/jira/browse/CASSANDRA-16940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17413025#comment-17413025
]
Sam Tunnicliffe commented on CASSANDRA-16940:
-
will do
> Confusing ProtocolException msg Invalid or unsupported protocol version (4)
> ---
>
> Key: CASSANDRA-16940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16940
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client
>Reporter: Brad Schoening
>Priority: Normal
>
> The following warning was seen frequently after upgrading from 3.0.x to
> 3.11.11 in the cassandra.log:
> {noformat}
> ProtocolException: Invalid or unsupported protocol version (4); supported
> versions are (3/v3, 4/v4, 5/v5-beta){noformat}
> It is at best unclear, or maybe a bug in the code throwing this exception
> stating version '4' not supported but 4/v4 is.
> from org/apache/cassandra/transport/ProtocolVersion.java
> public static String invalidVersionMessage(int version)
> { return String.format("Invalid or unsupported protocol version (%d);
> supported versions are (%s)", version, String.join(", ",
> ProtocolVersion.supportedVersions())); }
> We later found invalid IP addresses in the system.peers table and once
> removed, this exception went away.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16404:
Fix Version/s: (was: 4.x)
4.1
Source Control Link:
https://github.com/apache/cassandra/commit/f9aa19e3b116c0078019e9382d1a6c4bb050f113
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed to C* trunk in
{{[f9aa19e3|https://github.com/apache/cassandra/commit/f9aa19e3b116c0078019e9382d1a6c4bb050f113]}}
and to dtests in
{{[0ef8be46|https://github.com/apache/cassandra-dtest/commit/0ef8be46f8f729c80662a03fd515b6fe108531c8]}}
and
{{[1f5aefdc|https://github.com/apache/cassandra-dtest/commit/1f5aefdc23b5cd27dea056d119ff5d9c9801030a]}}.
Thanks for the patches and for your patience [~azotcsit]
> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.1
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like
> KeyCache, RowCache and CounterCache.
> Being able to invalidate auth caches as well can come in handy in situations
> where, critical backend auth changes may need to be in effect right away for
> all the connections, especially in configurations where cache validity is
> chosen to be for a longer duration. An example can be that an authenticated
> user "User1" is no longer authorized to access a table resource "table1" and
> it is vital that this change is reflected right away, without having to wait
> for cache expiry/refresh to trigger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16404: Reviewers: Benjamin Lerer, Sam Tunnicliffe, Sumanth Pasupuleti (was: Benjamin Lerer, Sam Tunnicliffe) Status: Review In Progress (was: Patch Available) > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 1h > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16404: Status: Ready to Commit (was: Review In Progress) > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 1h > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409013#comment-17409013 ] Sam Tunnicliffe commented on CASSANDRA-16404: - [~azotcsit], I was and it looks good to me. Apologies for both the long wait and the typo, I've rebased and launched CI runs and commit both the patch and test when those complete. ||branch||dtest||Circle CI||Apache CI|| |[16404-trunk|https://github.com/beobal/cassandra/tree/16404-trunk]|[16404|https://github.com/beobal/cassandra-dtest/tree/16404]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16404-trunk]|[apache|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1093] > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 1h > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16783) Delay auth setup to after gossip has settled to avoid unavailables on startup
[ https://issues.apache.org/jira/browse/CASSANDRA-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16783: Status: Ready to Commit (was: Review In Progress) LGTM, +1 > Delay auth setup to after gossip has settled to avoid unavailables on startup > - > > Key: CASSANDRA-16783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16783 > Project: Cassandra > Issue Type: Bug > Components: Local/Startup and Shutdown >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0.x, 4.x > > > We should delay trying to do auth setup until after gossip has settled to > avoid getting unavailables from trying to read the auth tables before knowing > of any neighbours -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16783) Delay auth setup to after gossip has settled to avoid unavailables on startup
[ https://issues.apache.org/jira/browse/CASSANDRA-16783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16783: Reviewers: Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe) Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe) Status: Review In Progress (was: Patch Available) > Delay auth setup to after gossip has settled to avoid unavailables on startup > - > > Key: CASSANDRA-16783 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16783 > Project: Cassandra > Issue Type: Bug > Components: Local/Startup and Shutdown >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0.x, 4.x > > > We should delay trying to do auth setup until after gossip has settled to > avoid getting unavailables from trying to read the auth tables before knowing > of any neighbours -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16885) Add GPG key for s...@apache.org to to dist/cassandra/KEYS
[ https://issues.apache.org/jira/browse/CASSANDRA-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16885: Reviewers: Michael Semb Wever > Add GPG key for s...@apache.org to to dist/cassandra/KEYS > - > > Key: CASSANDRA-16885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16885 > Project: Cassandra > Issue Type: Task > Components: Packaging >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: NA > > Attachments: 16885.patch > > > I need to add my public key to the project's KEYS file in order to > sign/publish release artifacts. > Fingerprint: F8B7FD00E05C932991A2CD6150EE103D162C5A55 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16885) Add GPG key for s...@apache.org to to dist/cassandra/KEYS
[ https://issues.apache.org/jira/browse/CASSANDRA-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16885: Attachment: 16885.patch > Add GPG key for s...@apache.org to to dist/cassandra/KEYS > - > > Key: CASSANDRA-16885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16885 > Project: Cassandra > Issue Type: Task > Components: Packaging >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: NA > > Attachments: 16885.patch > > > I need to add my public key to the project's KEYS file in order to > sign/publish release artifacts. > Fingerprint: F8B7FD00E05C932991A2CD6150EE103D162C5A55 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16885) Add GPG key for s...@apache.org to to dist/cassandra/KEYS
[ https://issues.apache.org/jira/browse/CASSANDRA-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16885: Test and Documentation Plan: Merge, import KEYS file, check sig. Status: Patch Available (was: Open) This is a new, previously unused key so doesn't yet belong to a web of trust. > Add GPG key for s...@apache.org to to dist/cassandra/KEYS > - > > Key: CASSANDRA-16885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16885 > Project: Cassandra > Issue Type: Task > Components: Packaging >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: NA > > > I need to add my public key to the project's KEYS file in order to > sign/publish release artifacts. > Fingerprint: F8B7FD00E05C932991A2CD6150EE103D162C5A55 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16885) Add GPG key for s...@apache.org to to dist/cassandra/KEYS
[ https://issues.apache.org/jira/browse/CASSANDRA-16885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16885: Change Category: Operability Complexity: Normal Fix Version/s: NA Status: Open (was: Triage Needed) > Add GPG key for s...@apache.org to to dist/cassandra/KEYS > - > > Key: CASSANDRA-16885 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16885 > Project: Cassandra > Issue Type: Task > Components: Packaging >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: NA > > > I need to add my public key to the project's KEYS file in order to > sign/publish release artifacts. > Fingerprint: F8B7FD00E05C932991A2CD6150EE103D162C5A55 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16885) Add GPG key for s...@apache.org to to dist/cassandra/KEYS
Sam Tunnicliffe created CASSANDRA-16885: --- Summary: Add GPG key for s...@apache.org to to dist/cassandra/KEYS Key: CASSANDRA-16885 URL: https://issues.apache.org/jira/browse/CASSANDRA-16885 Project: Cassandra Issue Type: Task Components: Packaging Reporter: Sam Tunnicliffe Assignee: Sam Tunnicliffe I need to add my public key to the project's KEYS file in order to sign/publish release artifacts. Fingerprint: F8B7FD00E05C932991A2CD6150EE103D162C5A55 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16721) Repaired data tracking on a read coordinator is susceptible to races between local and remote requests
[
https://issues.apache.org/jira/browse/CASSANDRA-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404655#comment-17404655
]
Sam Tunnicliffe commented on CASSANDRA-16721:
-
Looks good to me, modulo a
[few|https://github.com/apache/cassandra/pull/1160/files#r695998655],
[trivial|https://github.com/apache/cassandra/pull/1160/files#r696006742],
[nits|https://github.com/apache/cassandra/pull/1160/files#r696008838] (sorry, I
forgot it was a PR not just a branch). I'm also a fan of Alex's suggestion to
replace {{TEST_FORCE_ASYNC_LOCAL_READS}} with some ByteBuddy manipulation in
the test.
All of the above can be fixed (or not) on commit, so +1 from me too & thanks!
> Repaired data tracking on a read coordinator is susceptible to races between
> local and remote requests
> --
>
> Key: CASSANDRA-16721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16721
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Sam Tunnicliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> At read time on a coordinator which is also a replica, the local and remote
> reads can race such that the remote responses are received while the local
> read is executing. If the remote responses are mismatching, triggering a
> {{DigestMismatchException}} and subsequent round of full data reads and read
> repair, the local runnable may find the {{isTrackingRepairedStatus}} flag
> flipped mid-execution. If this happens after a certain point in execution,
> it would mean
> that the RepairedDataInfo instance in use is the singleton null object
> {{RepairedDataInfo.NULL_REPAIRED_DATA_INFO}}. If this happens, it can lead to
> an NPE when calling {{RepairedDataInfo::extend}} when the local results are
> iterated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16721) Repaired data tracking on a read coordinator is susceptible to races between local and remote requests
[
https://issues.apache.org/jira/browse/CASSANDRA-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16721:
Reviewers: Alex Petrov, Caleb Rackliffe, Sam Tunnicliffe, Sam Tunnicliffe
(was: Alex Petrov, Caleb Rackliffe, Sam Tunnicliffe)
Alex Petrov, Caleb Rackliffe, Sam Tunnicliffe, Sam Tunnicliffe
(was: Caleb Rackliffe, Sam Tunnicliffe)
Status: Review In Progress (was: Patch Available)
> Repaired data tracking on a read coordinator is susceptible to races between
> local and remote requests
> --
>
> Key: CASSANDRA-16721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16721
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Sam Tunnicliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> At read time on a coordinator which is also a replica, the local and remote
> reads can race such that the remote responses are received while the local
> read is executing. If the remote responses are mismatching, triggering a
> {{DigestMismatchException}} and subsequent round of full data reads and read
> repair, the local runnable may find the {{isTrackingRepairedStatus}} flag
> flipped mid-execution. If this happens after a certain point in execution,
> it would mean
> that the RepairedDataInfo instance in use is the singleton null object
> {{RepairedDataInfo.NULL_REPAIRED_DATA_INFO}}. If this happens, it can lead to
> an NPE when calling {{RepairedDataInfo::extend}} when the local results are
> iterated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16877) High priority internode messages which exceed the large message threshold are dropped
[
https://issues.apache.org/jira/browse/CASSANDRA-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16877:
Fix Version/s: (was: 4.0.x)
4.0.1
Since Version: 4.0-alpha1
Source Control Link:
https://github.com/apache/cassandra/commit/b8242730918c2e8edec83aeafeeae8255378125d
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks. Agreed about the tests, so committed (with one nit addressed and one
swerved) to 4.0 in {{b8242730918c2e8edec83aeafeeae8255378125d}} and merged up
to trunk.
> High priority internode messages which exceed the large message threshold are
> dropped
> -
>
> Key: CASSANDRA-16877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16877
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0.1
>
>
> Currently, there is an assumption that internode messages whose verb has
> priority P0 will always fit within a single messaging frame. While this is
> usually the case, on occasion it is possible that this assumption does not
> hold. One example is gossip messages during the startup shadow round, where
> in very large clusters the digest ack can contain all states for every peer.
> In this scenario, the respondent fails to send the ack which may lead to the
> shadow round and, ultimately, the startup failing.
>
> We could tweak the shadow round acks to minimise the message size, but a more
> robust solution would be to permit high priority messages to be sent on the
> large messages connection when necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16877) High priority internode messages which exceed the large message threshold are dropped
[ https://issues.apache.org/jira/browse/CASSANDRA-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16877: Status: Ready to Commit (was: Review In Progress) > High priority internode messages which exceed the large message threshold are > dropped > - > > Key: CASSANDRA-16877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16877 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0.x > > > Currently, there is an assumption that internode messages whose verb has > priority P0 will always fit within a single messaging frame. While this is > usually the case, on occasion it is possible that this assumption does not > hold. One example is gossip messages during the startup shadow round, where > in very large clusters the digest ack can contain all states for every peer. > In this scenario, the respondent fails to send the ack which may lead to the > shadow round and, ultimately, the startup failing. > > We could tweak the shadow round acks to minimise the message size, but a more > robust solution would be to permit high priority messages to be sent on the > large messages connection when necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16877) High priority internode messages which exceed the large message threshold are dropped
[
https://issues.apache.org/jira/browse/CASSANDRA-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16877:
Test and Documentation Plan: New unit test in patch
Status: Patch Available (was: In Progress)
This can be a fairly serious problem when cluster sizes run into hundreds of
nodes. Following a restart, the shadow round can begin to fail as the
respondants can't serialize the {{GossipDigestAck}}, leaving the sender unable
to start. Even if the shadow round is skipped, the same problem often occurs
when the starting node first sends a regular {{GossipDygestSyn}}. This may be
an even worse scenario from an availability perspective as the restarted node
will not establish contact with peers and will see the rest of the ring as
down.
The reason I didn't add more detail to the nospam log message is that doing so
always feels to be of limited utility to me. With nospam you're probably only
seeing a tiny portion of the actual events, so you really need to enable the
trace/debug logging anyway. If people have a strong opinion to the contrary
though, I can always change this.
||branch||Circle CI||Apache CI||
|[16877-4.0|https://github.com/beobal/cassandra/tree/16877-4.0]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16877-4.0]|[apache|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1058]|
> High priority internode messages which exceed the large message threshold are
> dropped
> -
>
> Key: CASSANDRA-16877
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16877
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0.x
>
>
> Currently, there is an assumption that internode messages whose verb has
> priority P0 will always fit within a single messaging frame. While this is
> usually the case, on occasion it is possible that this assumption does not
> hold. One example is gossip messages during the startup shadow round, where
> in very large clusters the digest ack can contain all states for every peer.
> In this scenario, the respondent fails to send the ack which may lead to the
> shadow round and, ultimately, the startup failing.
>
> We could tweak the shadow round acks to minimise the message size, but a more
> robust solution would be to permit high priority messages to be sent on the
> large messages connection when necessary.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16877) High priority internode messages which exceed the large message threshold are dropped
[ https://issues.apache.org/jira/browse/CASSANDRA-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16877: Bug Category: Parent values: Availability(12983)Level 1 values: Process Crash(12992) (was: Parent values: Availability(12983)Level 1 values: Cluster Crash(12993)) > High priority internode messages which exceed the large message threshold are > dropped > - > > Key: CASSANDRA-16877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16877 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0.x > > > Currently, there is an assumption that internode messages whose verb has > priority P0 will always fit within a single messaging frame. While this is > usually the case, on occasion it is possible that this assumption does not > hold. One example is gossip messages during the startup shadow round, where > in very large clusters the digest ack can contain all states for every peer. > In this scenario, the respondent fails to send the ack which may lead to the > shadow round and, ultimately, the startup failing. > > We could tweak the shadow round acks to minimise the message size, but a more > robust solution would be to permit high priority messages to be sent on the > large messages connection when necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16877) High priority internode messages which exceed the large message threshold are dropped
[ https://issues.apache.org/jira/browse/CASSANDRA-16877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16877: Bug Category: Parent values: Availability(12983)Level 1 values: Cluster Crash(12993) Complexity: Normal Discovered By: Adhoc Test Fix Version/s: 4.0.x Severity: Critical Status: Open (was: Triage Needed) > High priority internode messages which exceed the large message threshold are > dropped > - > > Key: CASSANDRA-16877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16877 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Gossip >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 4.0.x > > > Currently, there is an assumption that internode messages whose verb has > priority P0 will always fit within a single messaging frame. While this is > usually the case, on occasion it is possible that this assumption does not > hold. One example is gossip messages during the startup shadow round, where > in very large clusters the digest ack can contain all states for every peer. > In this scenario, the respondent fails to send the ack which may lead to the > shadow round and, ultimately, the startup failing. > > We could tweak the shadow round acks to minimise the message size, but a more > robust solution would be to permit high priority messages to be sent on the > large messages connection when necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16877) High priority internode messages which exceed the large message threshold are dropped
Sam Tunnicliffe created CASSANDRA-16877: --- Summary: High priority internode messages which exceed the large message threshold are dropped Key: CASSANDRA-16877 URL: https://issues.apache.org/jira/browse/CASSANDRA-16877 Project: Cassandra Issue Type: Bug Components: Cluster/Gossip Reporter: Sam Tunnicliffe Assignee: Sam Tunnicliffe Currently, there is an assumption that internode messages whose verb has priority P0 will always fit within a single messaging frame. While this is usually the case, on occasion it is possible that this assumption does not hold. One example is gossip messages during the startup shadow round, where in very large clusters the digest ack can contain all states for every peer. In this scenario, the respondent fails to send the ack which may lead to the shadow round and, ultimately, the startup failing. We could tweak the shadow round acks to minimise the message size, but a more robust solution would be to permit high priority messages to be sent on the large messages connection when necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16721) Repaired data tracking on a read coordinator is susceptible to races between local and remote requests
[
https://issues.apache.org/jira/browse/CASSANDRA-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401177#comment-17401177
]
Sam Tunnicliffe commented on CASSANDRA-16721:
-
Your approach is a great improvement, makes a lot more sense for
{{ReadExecutionController}} to handle {{RepairedDataInfo}}. The only
non-obvious thing I noticed is that the sub-controllers for index reads should
probably not inherit the tracking flag, just always make it false. It should be
safe and not inefficient either way, but it just doesn't make sense for a read
of the index table to be set to track.
> Repaired data tracking on a read coordinator is susceptible to races between
> local and remote requests
> --
>
> Key: CASSANDRA-16721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16721
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Sam Tunnicliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 4.0.x
>
>
> At read time on a coordinator which is also a replica, the local and remote
> reads can race such that the remote responses are received while the local
> read is executing. If the remote responses are mismatching, triggering a
> {{DigestMismatchException}} and subsequent round of full data reads and read
> repair, the local runnable may find the {{isTrackingRepairedStatus}} flag
> flipped mid-execution. If this happens after a certain point in execution,
> it would mean
> that the RepairedDataInfo instance in use is the singleton null object
> {{RepairedDataInfo.NULL_REPAIRED_DATA_INFO}}. If this happens, it can lead to
> an NPE when calling {{RepairedDataInfo::extend}} when the local results are
> iterated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401041#comment-17401041 ] Sam Tunnicliffe commented on CASSANDRA-16404: - bq. I see 4 failures, but they do not seem to be related to these changes. Yep, LGTM too. Good stuff! bq. Am I right that currently there are no dtests for JMXPermissionsCache/JmxPermissionsCache MBeans? Shall I try to come up with a test for that (basically I'd like to get familiar with dtests)? Can I use this ticket or need to create another one? Yes, you are correct there, so feel free to use this as an opportunity to get comfortable with dtests. Seeing as this isn't an urgent issue, I'd say it's fine to take our time adding new tests here rather than opening another Jira. If you create a dtest branch based off mine, just open a PR against trunk with it when you're ready and I'll review/merge there. If you have any questions, just ask. bq. It needs to be renamed to match the class name. d'oh, good catch. Fixed in my branch. > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 50m > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17400403#comment-17400403
]
Sam Tunnicliffe commented on CASSANDRA-16404:
-
Rather than going back to the old bean names, we have a precedent of aliasing
and deprecating them. I've pushed a additional commit to my branch which fixes
the original test failure in this way. I've also extended the dtest to cover
both the old and new bean name (for {{NetworkAuthTest}} only, it didn't seem
worth adding new tests for the JMX cache just for this).
Also (temporarily) modified circle config to use my own dtest branch & started
a new apache ci job
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1032]
> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like
> KeyCache, RowCache and CounterCache.
> Being able to invalidate auth caches as well can come in handy in situations
> where, critical backend auth changes may need to be in effect right away for
> all the connections, especially in configurations where cache validity is
> chosen to be for a longer duration. An example can be that an authenticated
> user "User1" is no longer authorized to access a table resource "table1" and
> it is vital that this change is reflected right away, without having to wait
> for cache expiry/refresh to trigger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17398082#comment-17398082 ] Sam Tunnicliffe commented on CASSANDRA-16404: - Thanks [~azotcsit], looks good to me. I've squashed & rebased, with one tiny tweak to the new error message, and kicked off CI runs. Assuming they look good, I'll commit when they're done. ||branch||Circle CI||Apache CI|| |[16404-trunk|https://github.com/beobal/cassandra/tree/16404-trunk]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16404-trunk]|[apache|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1011] > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 50m > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17387348#comment-17387348
]
Sam Tunnicliffe commented on CASSANDRA-16404:
-
bq. role_permissions table relies on this method while storing existing
permissions. So we cannot easily change it
this is correct. The only way around it would be to accept a CQL format string
and parse it on the server, but to be honest I don't believe it's really worth
it. So I'd say we should stick with the internal representation for now, seeing
as it's very much a poweruser feature anyway.
That one niggle aside, the CLI looks pretty good to me. There's just one small
issue whereby supplying invalid types for function params throws an exception,
which we could catch and present more nicely.
{code}
❯ bin/nodetool invalidatepermissionscache --functions-in-keyspace foo
--function bar\[x^y\] user
error: org.apache.cassandra.db.marshal.x
-- StackTrace --
java.lang.ClassNotFoundException: org.apache.cassandra.db.marshal.x
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at
org.apache.cassandra.utils.FBUtilities.classForName(FBUtilities.java:716)
at
org.apache.cassandra.db.marshal.TypeParser.getAbstractType(TypeParser.java:357)
at org.apache.cassandra.db.marshal.TypeParser.parse(TypeParser.java:89)
at
org.apache.cassandra.auth.FunctionResource.argsListFromString(FunctionResource.java:338)
at
org.apache.cassandra.auth.FunctionResource.fromName(FunctionResource.java:190)
at org.apache.cassandra.auth.Resources.fromName(Resources.java:60)
at
org.apache.cassandra.auth.PermissionsCache.invalidatePermissions(PermissionsCache.java:48)
...
...
...
{code}
Otherwise, I think we're pretty much done with this. Thanks for all the good
work [~azotcsit]
> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like
> KeyCache, RowCache and CounterCache.
> Being able to invalidate auth caches as well can come in handy in situations
> where, critical backend auth changes may need to be in effect right away for
> all the connections, especially in configurations where cache validity is
> chosen to be for a longer duration. An example can be that an authenticated
> user "User1" is no longer authorized to access a table resource "table1" and
> it is vital that this change is reflected right away, without having to wait
> for cache expiry/refresh to trigger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16796) Clear pending ranges for a SHUTDOWN peer
[ https://issues.apache.org/jira/browse/CASSANDRA-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16796: Fix Version/s: (was: 4.0.x) (was: 3.11.x) (was: 3.0.x) 4.0.1 3.11.11 3.0.25 Since Version: 3.0.0 Source Control Link: https://github.com/apache/cassandra/commit/fbb20b9162b73c4de8a82cf4ffdde3304e904603 Resolution: Fixed Status: Resolved (was: Ready to Commit) Thanks, cleaned up those unused imports and committed to 3.0 and merged to 3.11 -> 4.0 -> trunk. > Clear pending ranges for a SHUTDOWN peer > > > Key: CASSANDRA-16796 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16796 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.25, 3.11.11, 4.0.1 > > > If a node involved in a MOVE operation should fail, peers can sometimes > maintain pending ranges for it even when it has left the ring and/or been > replaced (in practice until the peer is next bounced). This in turn can lead > to bogus unavailable responses to clients if a replica for the any of the > pending ranges should go down. > If the moving node crashes hard, a subsequent replacement will correctly fail > as long as cassandra.consistent.rangemovement is set to true because the new > node will learn the MOVING status from the remaining peers. A graceful > shutdown, however, causes that status to be replaced with SHUTDOWN, but > doesn't update TokenMetadata, so pending ranges remain for the down node, > even after it has been removed from the ring. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16796) Clear pending ranges for a SHUTDOWN peer
[ https://issues.apache.org/jira/browse/CASSANDRA-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16796: Status: Ready to Commit (was: Review In Progress) > Clear pending ranges for a SHUTDOWN peer > > > Key: CASSANDRA-16796 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16796 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x > > > If a node involved in a MOVE operation should fail, peers can sometimes > maintain pending ranges for it even when it has left the ring and/or been > replaced (in practice until the peer is next bounced). This in turn can lead > to bogus unavailable responses to clients if a replica for the any of the > pending ranges should go down. > If the moving node crashes hard, a subsequent replacement will correctly fail > as long as cassandra.consistent.rangemovement is set to true because the new > node will learn the MOVING status from the remaining peers. A graceful > shutdown, however, causes that status to be replaced with SHUTDOWN, but > doesn't update TokenMetadata, so pending ranges remain for the down node, > even after it has been removed from the ring. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16796) Clear pending ranges for a SHUTDOWN peer
[
https://issues.apache.org/jira/browse/CASSANDRA-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17384092#comment-17384092
]
Sam Tunnicliffe commented on CASSANDRA-16796:
-
Thanks [~maedhroz]
bq. So just to make things explicit (for me, a non-gossip expert), notifying
subscribers in onChange() means we hit updateNormalTokens(), which removes the
endpoint from the "moving endpoints" set and eventually removes the pending
ranges?
Yes, that's right. The shutting down node is essentially returning to a
{{NORMAL}} state, so we just make sure that all
relevant parties (i.e. {{TokenMetadata}} ) are aware.
bq. What guarantees that a PendingRangeTask has run by the time
gossipShutdownUpdatesTokenMetadata() verifies there are no longer pending range
for node 2?
Good catch, I haven't seen any failures yet, but this definitely has the
potential for flakiness. I've added a log
statement at {{DEBUG}} at the end of {{Gossiper::markAsShutdown}} to gate the
test on, plus a blocking wait in the
assertion to ensure the PRT is complete before we check.
Circle runs are in the same place as before, new ASF CI jobs here:
[3.0|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/961],
[4.0|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/962]
> Clear pending ranges for a SHUTDOWN peer
>
>
> Key: CASSANDRA-16796
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16796
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> If a node involved in a MOVE operation should fail, peers can sometimes
> maintain pending ranges for it even when it has left the ring and/or been
> replaced (in practice until the peer is next bounced). This in turn can lead
> to bogus unavailable responses to clients if a replica for the any of the
> pending ranges should go down.
> If the moving node crashes hard, a subsequent replacement will correctly fail
> as long as cassandra.consistent.rangemovement is set to true because the new
> node will learn the MOVING status from the remaining peers. A graceful
> shutdown, however, causes that status to be replaced with SHUTDOWN, but
> doesn't update TokenMetadata, so pending ranges remain for the down node,
> even after it has been removed from the ring.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383381#comment-17383381
]
Sam Tunnicliffe edited comment on CASSANDRA-16404 at 7/19/21, 2:45 PM:
---
bq. Sam Tunnicliffe Does that sound reasonable to you?
Yes SGTM, thanks
[~azotcsit] I think it's only {{invalidatepermissionscache}} that warrants any
syntax change.
bq. "-u" might be bit confusing
You're right, I had overlooked that when sketching out that syntax.
Fortunately, we don't really need the {{-u}} flag as I think we can assume that
the non-option argument is the grantee, which is consistent with the other
commands anyway.
bq. if we do not have enough information from input to construct "Resource"
then we'll have to iterate through all cached records because "Resource" is a
part of key
I'd be tempted to enforce that through validation in the tool itself, although
iterating on the server side would still be required to invalidate all
parmissions for a user. So for instance, an invocation like this should be
rejected:
{code} bin/nodetool invalidatepermissionscache user_a -t some_table {code}
With the correct version being:
{code} bin/nodetool invalidatepermissionscache user_a -k some_keyspace -t
some_table {code}
Handling function resources is going to be a bit ugly as the arg types are
significant and so need to be included. The CQL representation is nicer for
the user but not very shell friendly whereas the internal representation is
very verbose. e.g. for a function with 2 int args:
{code} function ks.foo(int, int)
functions/ks/foo[org.apache.cassandra.db.marshal.Int32Type^org.apache.cassandra.db.marshal.Int32Type]
{code}
Seeing as we probably want a way to represent "all functions in a given
keyspace" anyway, maybe we should simplify and just make that the only option.
i.e. don't allow invalidation of permissions on specific functions, only all
functions in a particular keyspace.
What does everyone think?
was (Author: beobal):
bq. Sam Tunnicliffe Does that sound reasonable to you?
Yes SGTM, thanks
[~azotcsit] I think it's only {{invalidatepermissionscache}} that warrants any
syntax change.
bq. "-u" might be bit confusing
You're right, I had overlooked that when sketching out that syntax. , we don't
really need the {{-u}} flag as I think we can assume that the non-option
argument is the grantee, which is consistent with the other commands anyway.
bq. if we do not have enough information from input to construct "Resource"
then we'll have to iterate through all cached records because "Resource" is a
part of key
I'd be tempted to enforce that through validation in the tool itself, although
iterating on the server side would still be required to invalidate all
parmissions for a user. So for instance, an invocation like this should be
rejected:
{code} bin/nodetool invalidatepermissionscache user_a -t some_table {code}
With the correct version being:
{code} bin/nodetool invalidatepermissionscache user_a -k some_keyspace -t
some_table {code}
Handling function resources is going to be a bit ugly as the arg types are
significant and so need to be included. The CQL representation is nicer for
the user but not very shell friendly whereas the internal representation is
very verbose. e.g. for a function with 2 int args:
{code} function ks.foo(int, int)
functions/ks/foo[org.apache.cassandra.db.marshal.Int32Type^org.apache.cassandra.db.marshal.Int32Type]
{code}
Seeing as we probably want a way to represent "all functions in a given
keyspace" anyway, maybe we should simplify and just make that the only option.
i.e. don't allow invalidation of permissions on specific functions, only all
functions in a particular keyspace.
What does everyone think?
> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like
> KeyCache, RowCache and CounterCache.
> Being able to invalidate auth caches as well can come in handy in situations
> where, critical backend auth changes may need to be in effect right away for
> all the connections, especially in configurations where cache validity is
> chosen to be for a longer duration. An example can be that an authenticated
> user "User1" is no longer authorized to access a table resource "table1" and
> it is vital that this change is reflected right away, without having to
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17383381#comment-17383381
]
Sam Tunnicliffe commented on CASSANDRA-16404:
-
bq. Sam Tunnicliffe Does that sound reasonable to you?
Yes SGTM, thanks
[~azotcsit] I think it's only {{invalidatepermissionscache}} that warrants any
syntax change.
bq. "-u" might be bit confusing
You're right, I had overlooked that when sketching out that syntax. , we don't
really need the {{-u}} flag as I think we can assume that the non-option
argument is the grantee, which is consistent with the other commands anyway.
bq. if we do not have enough information from input to construct "Resource"
then we'll have to iterate through all cached records because "Resource" is a
part of key
I'd be tempted to enforce that through validation in the tool itself, although
iterating on the server side would still be required to invalidate all
parmissions for a user. So for instance, an invocation like this should be
rejected:
{code} bin/nodetool invalidatepermissionscache user_a -t some_table {code}
With the correct version being:
{code} bin/nodetool invalidatepermissionscache user_a -k some_keyspace -t
some_table {code}
Handling function resources is going to be a bit ugly as the arg types are
significant and so need to be included. The CQL representation is nicer for
the user but not very shell friendly whereas the internal representation is
very verbose. e.g. for a function with 2 int args:
{code} function ks.foo(int, int)
functions/ks/foo[org.apache.cassandra.db.marshal.Int32Type^org.apache.cassandra.db.marshal.Int32Type]
{code}
Seeing as we probably want a way to represent "all functions in a given
keyspace" anyway, maybe we should simplify and just make that the only option.
i.e. don't allow invalidation of permissions on specific functions, only all
functions in a particular keyspace.
What does everyone think?
> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like
> KeyCache, RowCache and CounterCache.
> Being able to invalidate auth caches as well can come in handy in situations
> where, critical backend auth changes may need to be in effect right away for
> all the connections, especially in configurations where cache validity is
> chosen to be for a longer duration. An example can be that an authenticated
> user "User1" is no longer authorized to access a table resource "table1" and
> it is vital that this change is reflected right away, without having to wait
> for cache expiry/refresh to trigger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380539#comment-17380539
]
Sam Tunnicliffe commented on CASSANDRA-16404:
-
Yep, I completely recognise your points here [~blerer], but I suppose my
general counter argument would be if not now, when?
{quote}I also want to raise the fact that so far the Virtual Tables that we
expose do not support updates. The mechanism is there but has not really been
used (TRUNCATE is not supported at at).
{quote}
This is a brand new, low risk feature which is not blocking a release and nor
is it particularly urgent. That seems to me to make it an ideal way to explore
& flesh out that area.
{quote}Will it not make sense to support both approach until Virtual Tables
provide the same level of functionalities that JMX and NodeTool or an extended
version?
{quote}
That is quite an ambitious goal. My concern is that if we take the approach of
splitting each task like this into distinct tickets, then pragmatism will get
the better of us as it has in the past. That is, the desire for something that
"does the job" today trumps the better/newer, but ultimately redundant, stuff.
i.e. if there's a way to acheive this through nodetool/JMX, the incentive to
build the VT version largely disappears.
All that said, like I said before I do appreciate that a lot of work has
already gone into this ticket so it's absolutely my bad for not bringing
forward this suggestion earlier. Aside from the syntactic stuff I mentioned in
my previous comment, this generally LGTM, so I certainly wouldn't dig my heels
in over it should we choose to stick with the current approach.
> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like
> KeyCache, RowCache and CounterCache.
> Being able to invalidate auth caches as well can come in handy in situations
> where, critical backend auth changes may need to be in effect right away for
> all the connections, especially in configurations where cache validity is
> chosen to be for a longer duration. An example can be that an authenticated
> user "User1" is no longer authorized to access a table resource "table1" and
> it is vital that this change is reflected right away, without having to wait
> for cache expiry/refresh to trigger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380401#comment-17380401 ] Sam Tunnicliffe commented on CASSANDRA-16404: - Thanks [~azotcsit], and apologies again for dropping in that suggestion so late. > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Aleksei Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 50m > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[
https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379900#comment-17379900
]
Sam Tunnicliffe commented on CASSANDRA-16404:
-
I feel that the {{invalidatepermissionscache}} nodetool command could use a
little syntactic sugar as operators have
previously not had any reason to be aware of the resource hierarchy. Perhaps as
a first option we could support invalidation of all permissions for a supplied
user, with the option to specify resources when required.
That said, the format of resource names is not something I'd expect most
operators to be familiar with. So while
allowing a qualified resource name is good, perhaps we ought to base the cli
syntax more closely on CQL GRANT/REVOKE
statements. e.g.:
{code}bin/nodetool invalidatepermissionscache -u [ -k |
-t | -f | -m
| -r ] {code}
Stepping back a bit... I appreciate that a lot of effort has already gone into
this ticket, but I wonder if we ought to
approach this slightly differently. Nodetool is, to some extent, deprecated,
with access to system information via CQL &
virtual tables being the preferred alternative. Perhaps we would be better off
implementing this feature with a set of
virtual tables to represent the caches.
{code}
cqlsh:system_views> SELECT * FROM permissions_cache;
role | resource| permissions
-+-+--
user_a | data/test_ks/test_table | {'SELECT', 'MODIFY'}
cqlsh:system_views> SELECT * FROM network_permissions_cache;
role | datacenters
-+
user_a | {'dc1', 'dc2'}
user_b | {}
{code}
The wrinkle I see would be in restricting the set of modifications permitted
against the virtual tables. i.e. A user,
with the requisite permissions, should be able to DELETE from and TRUNCATE a
*_cache table, but not perform any INSERT
or UPDATE modifications. It would require some special casing probably along
the lines of what we have in
{{ClientState::preventSystemKSSchemaModification}} but I don't think that would
be too onerous. Seeing as this is
targetting 4.x it seems slightly wrong to be adding brand new nodetool/JMX
operations.
> Provide a nodetool way of invalidating auth caches
> --
>
> Key: CASSANDRA-16404
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16404
> Project: Cassandra
> Issue Type: Improvement
> Components: Feature/Authorization
>Reporter: Sumanth Pasupuleti
>Assignee: Aleksei Zotov
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We currently have nodetool commands to invalidate certain caches like
> KeyCache, RowCache and CounterCache.
> Being able to invalidate auth caches as well can come in handy in situations
> where, critical backend auth changes may need to be in effect right away for
> all the connections, especially in configurations where cache validity is
> chosen to be for a longer duration. An example can be that an authenticated
> user "User1" is no longer authorized to access a table resource "table1" and
> it is vital that this change is reflected right away, without having to wait
> for cache expiry/refresh to trigger.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16796) Clear pending ranges for a SHUTDOWN peer
[ https://issues.apache.org/jira/browse/CASSANDRA-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16796: Description: If a node involved in a MOVE operation should fail, peers can sometimes maintain pending ranges for it even when it has left the ring and/or been replaced (in practice until the peer is next bounced). This in turn can lead to bogus unavailable responses to clients if a replica for the any of the pending ranges should go down. If the moving node crashes hard, a subsequent replacement will correctly fail as long as cassandra.consistent.rangemovement is set to true because the new node will learn the MOVING status from the remaining peers. A graceful shutdown, however, causes that status to be replaced with SHUTDOWN, but doesn't update TokenMetadata, so pending ranges remain for the down node, even after it has been removed from the ring. was: If a node involved in a MOVE operation should fail, peers can sometimes maintain pending ranges even when it has left the ring and/or been replaced (in practice until the peer is next bounced). This in turn can lead to bogus unavailable responses to clients if a replica for the any of the pending ranges should go down. If the moving node crashes hard, a subsequent replacement will correctly fail as long as cassandra.consistent.rangemovement is set to true because the new node will learn the MOVING status from the remaining peers. A graceful shutdown, however, causes that status to be replaced with SHUTDOWN, but doesn't update TokenMetadata, so pending ranges remain for the down node, even after it has been removed from the ring. > Clear pending ranges for a SHUTDOWN peer > > > Key: CASSANDRA-16796 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16796 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x > > > If a node involved in a MOVE operation should fail, peers can sometimes > maintain pending ranges for it even when it has left the ring and/or been > replaced (in practice until the peer is next bounced). This in turn can lead > to bogus unavailable responses to clients if a replica for the any of the > pending ranges should go down. > If the moving node crashes hard, a subsequent replacement will correctly fail > as long as cassandra.consistent.rangemovement is set to true because the new > node will learn the MOVING status from the remaining peers. A graceful > shutdown, however, causes that status to be replaced with SHUTDOWN, but > doesn't update TokenMetadata, so pending ranges remain for the down node, > even after it has been removed from the ring. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16796) Clear pending ranges for a SHUTDOWN peer
[ https://issues.apache.org/jira/browse/CASSANDRA-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16796: Test and Documentation Plan: new dtest added Status: Patch Available (was: Open) ||branch||Circle CI|| |[16796-3.0|https://github.com/beobal/cassandra/tree/16796-3.0]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16796-3.0]| |[16796-3.11|https://github.com/beobal/cassandra/tree/16796-3.11]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16796-3.11]| |[16796-4.0|https://github.com/beobal/cassandra/tree/16796-4.0]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16796-4.0]| |[16796-trunk|https://github.com/beobal/cassandra/tree/16796-trunk]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16796-trunk]| The 3.0/3.11 & 4.0/trunk patches are basically the same, so I've only kicked off ci-c jobs for [3.0|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/936] and [4.0|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/936] > Clear pending ranges for a SHUTDOWN peer > > > Key: CASSANDRA-16796 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16796 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x > > > If a node involved in a MOVE operation should fail, peers can sometimes > maintain pending ranges even when it has left the ring and/or been replaced > (in practice until the peer is next bounced). This in turn can lead to bogus > unavailable responses to clients if a replica for the any of the pending > ranges should go down. > If the moving node crashes hard, a subsequent replacement will correctly fail > as long as cassandra.consistent.rangemovement is set to true because the new > node will learn the MOVING status from the remaining peers. A graceful > shutdown, however, causes that status to be replaced with SHUTDOWN, but > doesn't update TokenMetadata, so pending ranges remain for the down node, > even after it has been removed from the ring. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16796) Clear pending ranges for a SHUTDOWN peer
[ https://issues.apache.org/jira/browse/CASSANDRA-16796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16796: Bug Category: Parent values: Availability(12983)Level 1 values: Unavailable(12994) Complexity: Normal Discovered By: User Report Fix Version/s: 4.0.x 3.11.x 3.0.x Severity: Normal Status: Open (was: Triage Needed) > Clear pending ranges for a SHUTDOWN peer > > > Key: CASSANDRA-16796 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16796 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Membership >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > Fix For: 3.0.x, 3.11.x, 4.0.x > > > If a node involved in a MOVE operation should fail, peers can sometimes > maintain pending ranges even when it has left the ring and/or been replaced > (in practice until the peer is next bounced). This in turn can lead to bogus > unavailable responses to clients if a replica for the any of the pending > ranges should go down. > If the moving node crashes hard, a subsequent replacement will correctly fail > as long as cassandra.consistent.rangemovement is set to true because the new > node will learn the MOVING status from the remaining peers. A graceful > shutdown, however, causes that status to be replaced with SHUTDOWN, but > doesn't update TokenMetadata, so pending ranges remain for the down node, > even after it has been removed from the ring. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16796) Clear pending ranges for a SHUTDOWN peer
Sam Tunnicliffe created CASSANDRA-16796: --- Summary: Clear pending ranges for a SHUTDOWN peer Key: CASSANDRA-16796 URL: https://issues.apache.org/jira/browse/CASSANDRA-16796 Project: Cassandra Issue Type: Bug Components: Cluster/Membership Reporter: Sam Tunnicliffe Assignee: Sam Tunnicliffe If a node involved in a MOVE operation should fail, peers can sometimes maintain pending ranges even when it has left the ring and/or been replaced (in practice until the peer is next bounced). This in turn can lead to bogus unavailable responses to clients if a replica for the any of the pending ranges should go down. If the moving node crashes hard, a subsequent replacement will correctly fail as long as cassandra.consistent.rangemovement is set to true because the new node will learn the MOVING status from the remaining peers. A graceful shutdown, however, causes that status to be replaced with SHUTDOWN, but doesn't update TokenMetadata, so pending ranges remain for the down node, even after it has been removed from the ring. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17375378#comment-17375378 ] Sam Tunnicliffe commented on CASSANDRA-16404: - bq. Sam Tunnicliffe Your are familiar with that part of the code. Would you have some time for a review? Probably not for a few days, but I will try to take a look this week if that's any use. > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Alexey Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 40m > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-12988) make the consistency level for user-level auth reads and writes configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-12988:
---
Assignee: Sam Tunnicliffe
> make the consistency level for user-level auth reads and writes configurable
>
>
> Key: CASSANDRA-12988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12988
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Assignee: Sam Tunnicliffe
>Priority: Low
> Fix For: 4.x
>
>
> Most reads for the auth-related tables execute at {{LOCAL_ONE}}. We'd like to
> make it configurable, with the default still being {{LOCAL_ONE}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12988) make the consistency level for user-level auth reads and writes configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-12988:
Authors: (was: Sam Tunnicliffe)
> make the consistency level for user-level auth reads and writes configurable
>
>
> Key: CASSANDRA-12988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12988
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Assignee: Sam Tunnicliffe
>Priority: Low
> Fix For: 4.x
>
>
> Most reads for the auth-related tables execute at {{LOCAL_ONE}}. We'd like to
> make it configurable, with the default still being {{LOCAL_ONE}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-12988) make the consistency level for user-level auth reads and writes configurable
[
https://issues.apache.org/jira/browse/CASSANDRA-12988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe reassigned CASSANDRA-12988:
---
Assignee: (was: Sam Tunnicliffe)
> make the consistency level for user-level auth reads and writes configurable
>
>
> Key: CASSANDRA-12988
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12988
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Jason Brown
>Priority: Low
> Fix For: 4.x
>
>
> Most reads for the auth-related tables execute at {{LOCAL_ONE}}. We'd like to
> make it configurable, with the default still being {{LOCAL_ONE}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16735) Adding columns via ALTER TABLE can generate corrupt sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16735: Status: Ready to Commit (was: Review In Progress) LGTM +1 > Adding columns via ALTER TABLE can generate corrupt sstables > > > Key: CASSANDRA-16735 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16735 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair, Local/SSTable >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-rc2, 3.0.x, 3.11.x, 4.0-rc > > > This is similar to CASSANDRA-13004 and was caused by CASSANDRA-15899 > Basically the column placeholders introduced in 15899 can get read-repaired > in to the memtable and later flushed to disk and in some cases this can > conflict with the actual column (if the actual column is a collection for > example) and cause CorruptSSTableExceptions. > Fix is probably to just revert 15899, at least until if and when we find a > solution that we can rely on. Will post that + test next week. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16735) Adding columns via ALTER TABLE can generate corrupt sstables
[ https://issues.apache.org/jira/browse/CASSANDRA-16735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16735: Reviewers: Alex Petrov, Sam Tunnicliffe, Sam Tunnicliffe (was: Alex Petrov, Sam Tunnicliffe) Alex Petrov, Sam Tunnicliffe, Sam Tunnicliffe (was: Alex Petrov, Sam Tunnicliffe) Status: Review In Progress (was: Patch Available) > Adding columns via ALTER TABLE can generate corrupt sstables > > > Key: CASSANDRA-16735 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16735 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Repair, Local/SSTable >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 4.0-rc2, 3.0.x, 3.11.x, 4.0-rc > > > This is similar to CASSANDRA-13004 and was caused by CASSANDRA-15899 > Basically the column placeholders introduced in 15899 can get read-repaired > in to the memtable and later flushed to disk and in some cases this can > conflict with the actual column (if the actual column is a collection for > example) and cause CorruptSSTableExceptions. > Fix is probably to just revert 15899, at least until if and when we find a > solution that we can rely on. Will post that + test next week. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16722) ClientMetrics should be initialised in CQLConnectionTest
[
https://issues.apache.org/jira/browse/CASSANDRA-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16722:
Fix Version/s: (was: 4.0-rc)
4.0
Since Version: NA
Source Control Link:
https://github.com/apache/cassandra/commit/546792169e7df90c905139d1a275056bbef850d4
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks, committed with one additional tweak to remove a race in
{{CQLConnectionTest}} when the server closes the channel.
> ClientMetrics should be initialised in CQLConnectionTest
>
>
> Key: CASSANDRA-16722
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16722
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0
>
>
> Without a call to {{ClientMetrics.instance.init}}, client protocol errors
> cause an NPE when the server attempts to mark the relevant meter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16722) ClientMetrics should be initialised in CQLConnectionTest
[
https://issues.apache.org/jira/browse/CASSANDRA-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16722:
Test and Documentation Plan: test only fix
Status: Patch Available (was: Open)
||branch||Circle CI|Apache CI|
|[16722-4.0.0|https://github.com/beobal/cassandra/tree/16722-4.0.0]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16722-4.0.0]|[apache|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/849]|
> ClientMetrics should be initialised in CQLConnectionTest
>
>
> Key: CASSANDRA-16722
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16722
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Without a call to {{ClientMetrics.instance.init}}, client protocol errors
> cause an NPE when the server attempts to mark the relevant meter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16722) ClientMetrics should be initialised in CQLConnectionTest
[
https://issues.apache.org/jira/browse/CASSANDRA-16722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16722:
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Low Hanging Fruit
Discovered By: Unit Test
Fix Version/s: 4.0-rc
Severity: Low
Status: Open (was: Triage Needed)
This only affects test code and only appears to have caused a single failure of
the 4.0.0 nightlies.
https://ci-cassandra.apache.org/job/Cassandra-4.0.0/3/testReport/
> ClientMetrics should be initialised in CQLConnectionTest
>
>
> Key: CASSANDRA-16722
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16722
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0-rc
>
>
> Without a call to {{ClientMetrics.instance.init}}, client protocol errors
> cause an NPE when the server attempts to mark the relevant meter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16722) ClientMetrics should be initialised in CQLConnectionTest
Sam Tunnicliffe created CASSANDRA-16722:
---
Summary: ClientMetrics should be initialised in CQLConnectionTest
Key: CASSANDRA-16722
URL: https://issues.apache.org/jira/browse/CASSANDRA-16722
Project: Cassandra
Issue Type: Bug
Components: Test/unit
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
Without a call to {{ClientMetrics.instance.init}}, client protocol errors cause
an NPE when the server attempts to mark the relevant meter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16721) Repaired data tracking on a read coordinator is susceptible to races between local and remote requests
[
https://issues.apache.org/jira/browse/CASSANDRA-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16721:
Test and Documentation Plan: New dtests added
Status: Patch Available (was: In Progress)
||branch||Circle CI|Apache CI|
|[16721-4.0|https://github.com/beobal/cassandra/tree/16721-4.0]|[circle|https://circleci.com/gh/beobal/cassandra?branch=16721-4.0]|[ci-c|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/848]|
> Repaired data tracking on a read coordinator is susceptible to races between
> local and remote requests
> --
>
> Key: CASSANDRA-16721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16721
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0.x
>
>
> At read time on a coordinator which is also a replica, the local and remote
> reads can race such that the remote responses are received while the local
> read is executing. If the remote responses are mismatching, triggering a
> {{DigestMismatchException}} and subsequent round of full data reads and read
> repair, the local runnable may find the {{isTrackingRepairedStatus}} flag
> flipped mid-execution. If this happens after a certain point in execution,
> it would mean
> that the RepairedDataInfo instance in use is the singleton null object
> {{RepairedDataInfo.NULL_REPAIRED_DATA_INFO}}. If this happens, it can lead to
> an NPE when calling {{RepairedDataInfo::extend}} when the local results are
> iterated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16721) Repaired data tracking on a read coordinator is susceptible to races between local and remote requests
[
https://issues.apache.org/jira/browse/CASSANDRA-16721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16721:
Bug Category: Parent values: Availability(12983)Level 1 values: Response
Crash(12991)
Complexity: Normal
Discovered By: User Report
Fix Version/s: 4.0.x
Severity: Normal
Status: Open (was: Triage Needed)
> Repaired data tracking on a read coordinator is susceptible to races between
> local and remote requests
> --
>
> Key: CASSANDRA-16721
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16721
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: Normal
> Fix For: 4.0.x
>
>
> At read time on a coordinator which is also a replica, the local and remote
> reads can race such that the remote responses are received while the local
> read is executing. If the remote responses are mismatching, triggering a
> {{DigestMismatchException}} and subsequent round of full data reads and read
> repair, the local runnable may find the {{isTrackingRepairedStatus}} flag
> flipped mid-execution. If this happens after a certain point in execution,
> it would mean
> that the RepairedDataInfo instance in use is the singleton null object
> {{RepairedDataInfo.NULL_REPAIRED_DATA_INFO}}. If this happens, it can lead to
> an NPE when calling {{RepairedDataInfo::extend}} when the local results are
> iterated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16721) Repaired data tracking on a read coordinator is susceptible to races between local and remote requests
Sam Tunnicliffe created CASSANDRA-16721:
---
Summary: Repaired data tracking on a read coordinator is
susceptible to races between local and remote requests
Key: CASSANDRA-16721
URL: https://issues.apache.org/jira/browse/CASSANDRA-16721
Project: Cassandra
Issue Type: Bug
Components: Consistency/Coordination
Reporter: Sam Tunnicliffe
Assignee: Sam Tunnicliffe
At read time on a coordinator which is also a replica, the local and remote
reads can race such that the remote responses are received while the local read
is executing. If the remote responses are mismatching, triggering a
{{DigestMismatchException}} and subsequent round of full data reads and read
repair, the local runnable may find the {{isTrackingRepairedStatus}} flag
flipped mid-execution. If this happens after a certain point in execution, it
would mean
that the RepairedDataInfo instance in use is the singleton null object
{{RepairedDataInfo.NULL_REPAIRED_DATA_INFO}}. If this happens, it can lead to
an NPE when calling {{RepairedDataInfo::extend}} when the local results are
iterated.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16581) Failure to execute queries should emit a KPI other than read timeout/unavailable so it can be alerted/tracked
[
https://issues.apache.org/jira/browse/CASSANDRA-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355045#comment-17355045
]
Sam Tunnicliffe commented on CASSANDRA-16581:
-
sorry for the holdup, the trunk fix LGTM now except that the change to
{{FrameEncoder::write}} hasn't been reverted yet. Aside from that, I think this
good to go once the 4.0 branch is updated. Thanks!
> Failure to execute queries should emit a KPI other than read
> timeout/unavailable so it can be alerted/tracked
> -
>
> Key: CASSANDRA-16581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16581
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Observability/Metrics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-rc
>
>
> When we are unable to parse a message we do not have a way to detect this
> from a monitoring point of view so can get into situations where we believe
> the database is fine but the clients are on-fire. This case popped up in the
> 2.1 to 3.0 upgrade as paging state wasn’t mixed-mode safe.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16581) Failure to execute queries should emit a KPI other than read timeout/unavailable so it can be alerted/tracked
[
https://issues.apache.org/jira/browse/CASSANDRA-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348687#comment-17348687
]
Sam Tunnicliffe edited comment on CASSANDRA-16581 at 5/20/21, 6:48 PM:
---
Oh, I totally missed that in {{SimpleClient}}, sorry. In that case, I
definitely think we need to revisit how the test is
implemented. That assertion in {{FrameEncoder}} is important, not only for the
client protocol, but for internode which
shares the class.
There's quite a lot of machinery in {{CQLConnectionTest}} to give control over
the exact bytes that the v5 pipeline
classes have to deal with. Unfortunately, retro fitting that to support v4 and
earlier is not a trivial task. I've
pushed an alternative reworking of the test
[here|https://github.com/beobal/cassandra/tree/CASSANDRA-16581-4.0].
Instead of changing the encoders/decoders this adds a test implementation of
{{Message.Request}} which we can prime with some garbage bytes, so that when
it's decoded, a \{{ProtocolError}} is guaranteed. This required a tiny bit of
refactoring in {{Envelope}} to make the v4 encoding ask-dont-tell like v5, but
it gives the test code control over the actual bytes that get decoded on the
server rather than having to bypass the checks in the outbound pipeline.
A nice side effect of this is that the v5 test is now exercising the CQL
protocol decoding, rather than hitting the CRC
errors in the framing layer. There are quite a few tests in
{{CQLConnectionTest}} which do verify that, so we should be
good there. This also means that the channel-closing behaviour is the same
between the v4 & v5 tests.
Now the message pipelines and failure behaviours are aligned across versions,
we don't need {{WrappedSimpleClient}} any
more and because the test cases are identical, I've made the test
{{@Parameterized}} so it will automatically cover all
supported versions.
was (Author: beobal):
Oh, I totally missed that in {{SimpleClient}}, sorry. In that case, I
definitely think we need to revisit how the test is
implemented. That assertion in {{FrameEncoder}} is important, not only for the
client protocol, but for internode which
shares the class.
There's quite a lot of machinery in {{CQLConnectionTest}} to give control over
the exact bytes that the v5 pipeline
classes have to deal with. Unfortunately, retro fitting that to support v4 and
earlier is not a trivial task. I've
pushed an alternative reworking of the test
[here|https://github.com/beobal/cassandra/tree/CASSANDRA-16581-4.0].
Instead of changing the encoders/decoders this adds a test implementation of
{{Message.Request}} which we can prime with some garbage bytes, so that when
it's decoded, a{{ProtocolError}} is guaranteed. This required a tiny bit of
refactoring in {{Envelope}} to make the v4 encoding ask-dont-tell like v5, but
it gives the test code control over the actual bytes that get decoded on the
server rather than having to bypass the checks in the outbound pipeline.
A nice side effect of this is that the v5 test is now exercising the CQL
protocol decoding, rather than hitting the CRC
errors in the framing layer. There are quite a few tests in
{{CQLConnectionTest}} which do verify that, so we should be
good there. This also means that the channel-closing behaviour is the same
between the v4 & v5 tests.
Now the message pipelines and failure behaviours are aligned across versions,
we don't need {{WrappedSimpleClient}} any
more and because the test cases are identical, I've made the test
{{@Parameterized}} so it will automatically cover all
supported versions.
> Failure to execute queries should emit a KPI other than read
> timeout/unavailable so it can be alerted/tracked
> -
>
> Key: CASSANDRA-16581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16581
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Observability/Metrics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-rc
>
>
> When we are unable to parse a message we do not have a way to detect this
> from a monitoring point of view so can get into situations where we believe
> the database is fine but the clients are on-fire. This case popped up in the
> 2.1 to 3.0 upgrade as paging state wasn’t mixed-mode safe.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16581) Failure to execute queries should emit a KPI other than read timeout/unavailable so it can be alerted/tracked
[
https://issues.apache.org/jira/browse/CASSANDRA-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348687#comment-17348687
]
Sam Tunnicliffe edited comment on CASSANDRA-16581 at 5/20/21, 6:47 PM:
---
Oh, I totally missed that in {{SimpleClient}}, sorry. In that case, I
definitely think we need to revisit how the test is
implemented. That assertion in {{FrameEncoder}} is important, not only for the
client protocol, but for internode which
shares the class.
There's quite a lot of machinery in {{CQLConnectionTest}} to give control over
the exact bytes that the v5 pipeline
classes have to deal with. Unfortunately, retro fitting that to support v4 and
earlier is not a trivial task. I've
pushed an alternative reworking of the test
[here|https://github.com/beobal/cassandra/tree/CASSANDRA-16581-4.0].
Instead of changing the encoders/decoders this adds a test implementation of
{{Message.Request}} which we can prime with some garbage bytes, so that when
it's decoded, a{{ProtocolError}} is guaranteed. This required a tiny bit of
refactoring in {{Envelope}} to make the v4 encoding ask-dont-tell like v5, but
it gives the test code control over the actual bytes that get decoded on the
server rather than having to bypass the checks in the outbound pipeline.
A nice side effect of this is that the v5 test is now exercising the CQL
protocol decoding, rather than hitting the CRC
errors in the framing layer. There are quite a few tests in
{{CQLConnectionTest}} which do verify that, so we should be
good there. This also means that the channel-closing behaviour is the same
between the v4 & v5 tests.
Now the message pipelines and failure behaviours are aligned across versions,
we don't need {{WrappedSimpleClient}} any
more and because the test cases are identical, I've made the test
{{@Parameterized}} so it will automatically cover all
supported versions.
was (Author: beobal):
Oh, I totally missed that in {{SimpleClient}}, sorry. In that case, I
definitely think we need to revisit how the test is
implemented. That assertion in {{FrameEncoder}} is important, not only for the
client protocol, but for internode which
shares the class.
There's quite a lot of machinery in {{CQLConnectionTest}} to give control over
the exact bytes that the v5 pipeline
classes have to deal with. Unfortunately, retro fitting that to support v4 and
earlier is not a trivial task. I've
pushed an alternative reworking of the test
[here|https://github.com/beobal/cassandra/tree/CASSANDRA-16581-4.0].
Instead of changing the encoders/decoders this adds a test implementation of
{{Message.Request}} which we can prime with some garbage bytes, so that when
it's decoded, a
{{ProtocolError}} is guaranteed. This required a tiny bit of refactoring in
{{Envelope}} to make the v4 encoding
ask-dont-tell like v5, but it gives the test code control over the actual bytes
that get decoded on the server rather
than having to bypass the checks in the outbound pipeline.
A nice side effect of this is that the v5 test is now exercising the CQL
protocol decoding, rather than hitting the CRC
errors in the framing layer. There are quite a few tests in
{{CQLConnectionTest}} which do verify that, so we should be
good there. This also means that the channel-closing behaviour is the same
between the v4 & v5 tests.
Now the message pipelines and failure behaviours are aligned across versions,
we don't need {{WrappedSimpleClient}} any
more and because the test cases are identical, I've made the test
{{@Parameterized}} so it will automatically cover all
supported versions.
> Failure to execute queries should emit a KPI other than read
> timeout/unavailable so it can be alerted/tracked
> -
>
> Key: CASSANDRA-16581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16581
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Observability/Metrics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-rc
>
>
> When we are unable to parse a message we do not have a way to detect this
> from a monitoring point of view so can get into situations where we believe
> the database is fine but the clients are on-fire. This case popped up in the
> 2.1 to 3.0 upgrade as paging state wasn’t mixed-mode safe.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16581) Failure to execute queries should emit a KPI other than read timeout/unavailable so it can be alerted/tracked
[
https://issues.apache.org/jira/browse/CASSANDRA-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16581:
Reviewers: Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe)
Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe)
Status: Review In Progress (was: Patch Available)
Oh, I totally missed that in {{SimpleClient}}, sorry. In that case, I
definitely think we need to revisit how the test is
implemented. That assertion in {{FrameEncoder}} is important, not only for the
client protocol, but for internode which
shares the class.
There's quite a lot of machinery in {{CQLConnectionTest}} to give control over
the exact bytes that the v5 pipeline
classes have to deal with. Unfortunately, retro fitting that to support v4 and
earlier is not a trivial task. I've
pushed an alternative reworking of the test
[here|https://github.com/beobal/cassandra/tree/CASSANDRA-16581-4.0].
Instead of changing the encoders/decoders this adds a test implementation of
{{Message.Request}} which we can prime with some garbage bytes, so that when
it's decoded, a
{{ProtocolError}} is guaranteed. This required a tiny bit of refactoring in
{{Envelope}} to make the v4 encoding
ask-dont-tell like v5, but it gives the test code control over the actual bytes
that get decoded on the server rather
than having to bypass the checks in the outbound pipeline.
A nice side effect of this is that the v5 test is now exercising the CQL
protocol decoding, rather than hitting the CRC
errors in the framing layer. There are quite a few tests in
{{CQLConnectionTest}} which do verify that, so we should be
good there. This also means that the channel-closing behaviour is the same
between the v4 & v5 tests.
Now the message pipelines and failure behaviours are aligned across versions,
we don't need {{WrappedSimpleClient}} any
more and because the test cases are identical, I've made the test
{{@Parameterized}} so it will automatically cover all
supported versions.
> Failure to execute queries should emit a KPI other than read
> timeout/unavailable so it can be alerted/tracked
> -
>
> Key: CASSANDRA-16581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16581
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Observability/Metrics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-rc
>
>
> When we are unable to parse a message we do not have a way to detect this
> from a monitoring point of view so can get into situations where we believe
> the database is fine but the clients are on-fire. This case popped up in the
> 2.1 to 3.0 upgrade as paging state wasn’t mixed-mode safe.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16581) Failure to execute queries should emit a KPI other than read timeout/unavailable so it can be alerted/tracked
[
https://issues.apache.org/jira/browse/CASSANDRA-16581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16581:
Status: Changes Suggested (was: Review In Progress)
Thanks, looks good to me except for a couple of minor things.
h3. ProtocolVersion
I'd suggest renaming {{DSE_VERSIONS}} to something like
{{KNOWN_INVALID_VERSIONS}}
h3. FrameEncoder
This change seems wrong, if we receive anything other than a {{Payload}} here,
it's an error. Is this some leftover debugging perhaps?
> Failure to execute queries should emit a KPI other than read
> timeout/unavailable so it can be alerted/tracked
> -
>
> Key: CASSANDRA-16581
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16581
> Project: Cassandra
> Issue Type: Bug
> Components: Messaging/Client, Observability/Metrics
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-rc
>
>
> When we are unable to parse a message we do not have a way to detect this
> from a monitoring point of view so can get into situations where we believe
> the database is fine but the clients are on-fire. This case popped up in the
> 2.1 to 3.0 upgrade as paging state wasn’t mixed-mode safe.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16613) ProtocolVersion.V4 is still used in places in the code
[
https://issues.apache.org/jira/browse/CASSANDRA-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16613:
Status: Ready to Commit (was: Review In Progress)
+1 LGTM. If I remember right, {{ViewComplexTest}} has long been flaky due to
timeouts on anything less than the XLarge plan. It would be nice if it wasn't
necessary, but raising the timeout is the most pragmatic solution so I'm fine
with it. Also +1 to changing the in-jvm dtests to use {{CURRENT}}, I can't
think of any reason that isn't the right thing to do.
> ProtocolVersion.V4 is still used in places in the code
> --
>
> Key: CASSANDRA-16613
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16613
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-rc2, 4.0-rc
>
>
> While working on CASSANDRA-16567, [~adelapena] observed that
> _ProtocolVersion.V4_ is used in _ViewTest_.
> I decided to do a quick grep and observed a list of places where we still
> refer to V4 and it seems at least in many of the tests that was left not
> intentionally.
> This ticket is to verify the usage of _ProtocolVersion.V4_ in the codebase
> and bump it to V5 or default version, similar to what was done in
> CASSANDRA-16567, wherever there is a need.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16613) ProtocolVersion.V4 is still used in places in the code
[ https://issues.apache.org/jira/browse/CASSANDRA-16613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16613: Reviewers: Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe) Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe) Status: Review In Progress (was: Patch Available) > ProtocolVersion.V4 is still used in places in the code > -- > > Key: CASSANDRA-16613 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16613 > Project: Cassandra > Issue Type: Bug >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-rc2, 4.0-rc > > > While working on CASSANDRA-16567, [~adelapena] observed that > _ProtocolVersion.V4_ is used in _ViewTest_. > I decided to do a quick grep and observed a list of places where we still > refer to V4 and it seems at least in many of the tests that was left not > intentionally. > This ticket is to verify the usage of _ProtocolVersion.V4_ in the codebase > and bump it to V5 or default version, similar to what was done in > CASSANDRA-16567, wherever there is a need. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-15269:
Reviewers: Sam Tunnicliffe, Sam Tunnicliffe (was: Sam Tunnicliffe)
Sam Tunnicliffe, Sam Tunnicliffe
Status: Review In Progress (was: Patch Available)
Sorry for the extremely late feedback on this, I was completely unaware of the
ticket. Unfortunately, protocol v5 has been finalised so a change like this one
would need to target v6.
That said, I'm not really sure that a new exception code is necessary here.
Technically, what is occurring _is_ a problem with function execution, that
function being a {{NumericOperationFunction}}, which is a subclass of
{{OperationFunction}} and ultimately {{AbstractFunction}}. So I don't think the
{{FUNCTION_FAILURE}} response is actually incorrect. My suggestion would be
that we fix the class cast exception, perhaps by having {{OEE}} extend {{FEE},
{{OperationFcts::getFunctionNameFromOperator}} can be used to obtain a
{{FunctionName}} for the operation.
> Cassandra fails to process OperationExecutionException which causes
> ClassCastException
> --
>
> Key: CASSANDRA-15269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15269
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Liudmila Kornilova
>Assignee: Liudmila Kornilova
>Priority: Normal
>
> While working on CASSANDRA-15232 I noticed that OperationExecutionException
> is not processed correctly.
> How to reproduce the issue:
> 1. {{create table d (numerator decimal primary key, denominator decimal);}}
> 2. {{insert into d (numerator, denominator) values
> (123456789112345678921234567893123456, 2);}}
> 3. {{select numerator % denominator from d;}}
> What happens:
> 1. remainder operation throws ArithmeticException (BigDecimal:1854)
> 2. The exception is wrapped in OperationExecutionException
> 3. ClassCastException appears (OperationExecutionException cannot be cast to
> FunctionExecutionException at ErrorMessage.java:280)
> What should happen:
> OperationExecutionException with message "the operation 'decimal % decimal'
> failed: Division impossible" should be delivered to user
> Note that after fixing CASSANDRA-15232 {{select numerator % denominator from
> d;}} will produce correct result of remainder operation.
> Currently I am not aware of other cases when OperationExecutionException may
> be treated as FunctionExecutionException
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-15269:
Status: Changes Suggested (was: Review In Progress)
> Cassandra fails to process OperationExecutionException which causes
> ClassCastException
> --
>
> Key: CASSANDRA-15269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15269
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Liudmila Kornilova
>Assignee: Liudmila Kornilova
>Priority: Normal
>
> While working on CASSANDRA-15232 I noticed that OperationExecutionException
> is not processed correctly.
> How to reproduce the issue:
> 1. {{create table d (numerator decimal primary key, denominator decimal);}}
> 2. {{insert into d (numerator, denominator) values
> (123456789112345678921234567893123456, 2);}}
> 3. {{select numerator % denominator from d;}}
> What happens:
> 1. remainder operation throws ArithmeticException (BigDecimal:1854)
> 2. The exception is wrapped in OperationExecutionException
> 3. ClassCastException appears (OperationExecutionException cannot be cast to
> FunctionExecutionException at ErrorMessage.java:280)
> What should happen:
> OperationExecutionException with message "the operation 'decimal % decimal'
> failed: Division impossible" should be delivered to user
> Note that after fixing CASSANDRA-15232 {{select numerator % denominator from
> d;}} will produce correct result of remainder operation.
> Currently I am not aware of other cases when OperationExecutionException may
> be treated as FunctionExecutionException
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15269) Cassandra fails to process OperationExecutionException which causes ClassCastException
[
https://issues.apache.org/jira/browse/CASSANDRA-15269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338860#comment-17338860
]
Sam Tunnicliffe edited comment on CASSANDRA-15269 at 5/4/21, 9:14 AM:
--
Sorry for the extremely late feedback on this, I was completely unaware of the
ticket. Unfortunately, protocol v5 has been finalised so a change like this one
would need to target v6.
That said, I'm not really sure that a new exception code is necessary here.
Technically, what is occurring _is_ a problem with function execution, that
function being a {{NumericOperationFunction}}, which is a subclass of
{{OperationFunction}} and ultimately {{AbstractFunction}}. So I don't think the
{{FUNCTION_FAILURE}} response is actually incorrect. My suggestion would be
that we fix the class cast exception, perhaps by having {{OEE}} extend {{FEE}}.
{{OperationFcts::getFunctionNameFromOperator}} can be used to obtain a
{{FunctionName}} for the operation.
was (Author: beobal):
Sorry for the extremely late feedback on this, I was completely unaware of the
ticket. Unfortunately, protocol v5 has been finalised so a change like this one
would need to target v6.
That said, I'm not really sure that a new exception code is necessary here.
Technically, what is occurring _is_ a problem with function execution, that
function being a {{NumericOperationFunction}}, which is a subclass of
{{OperationFunction}} and ultimately {{AbstractFunction}}. So I don't think the
{{FUNCTION_FAILURE}} response is actually incorrect. My suggestion would be
that we fix the class cast exception, perhaps by having {{OEE}} extend {{FEE},
{{OperationFcts::getFunctionNameFromOperator}} can be used to obtain a
{{FunctionName}} for the operation.
> Cassandra fails to process OperationExecutionException which causes
> ClassCastException
> --
>
> Key: CASSANDRA-15269
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15269
> Project: Cassandra
> Issue Type: Bug
> Components: CQL/Interpreter
>Reporter: Liudmila Kornilova
>Assignee: Liudmila Kornilova
>Priority: Normal
>
> While working on CASSANDRA-15232 I noticed that OperationExecutionException
> is not processed correctly.
> How to reproduce the issue:
> 1. {{create table d (numerator decimal primary key, denominator decimal);}}
> 2. {{insert into d (numerator, denominator) values
> (123456789112345678921234567893123456, 2);}}
> 3. {{select numerator % denominator from d;}}
> What happens:
> 1. remainder operation throws ArithmeticException (BigDecimal:1854)
> 2. The exception is wrapped in OperationExecutionException
> 3. ClassCastException appears (OperationExecutionException cannot be cast to
> FunctionExecutionException at ErrorMessage.java:280)
> What should happen:
> OperationExecutionException with message "the operation 'decimal % decimal'
> failed: Division impossible" should be delivered to user
> Note that after fixing CASSANDRA-15232 {{select numerator % denominator from
> d;}} will produce correct result of remainder operation.
> Currently I am not aware of other cases when OperationExecutionException may
> be treated as FunctionExecutionException
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16360) CRC32 is inefficient on x86
[
https://issues.apache.org/jira/browse/CASSANDRA-16360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17337296#comment-17337296
]
Sam Tunnicliffe commented on CASSANDRA-16360:
-
{quote}Do you think it is worth to make this part now or we should wait and
make the full change later on?
{quote}
Personally, I would apply the YAGNI principle and defer the work until I was
ready to start one of the pieces which actually requires it. That said, if you
have a surplus of free time and a burning desire to do it, I certainly wouldn't
stand in your way :)
> CRC32 is inefficient on x86
> ---
>
> Key: CASSANDRA-16360
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16360
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Client
>Reporter: Avi Kivity
>Assignee: Alexey Zotov
>Priority: Normal
> Labels: protocolv6
> Fix For: 4.x
>
>
> The client/server protocol specifies CRC24 and CRC32 as the checksum
> algorithm (cql_protocol_V5_framing.asc). Those however are expensive to
> compute; this affects both the client and the server.
>
> A better checksum algorithm is CRC32C, which has hardware support on x86 (as
> well as other modern architectures).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
